Empirical study on the performance stability of named entity recognition model across domains
Published 1 January 2006Open access
Hong Lei Guo, Li Zhang, Zhong Su
Citations12
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper provides an empirical study on the impact of training data size and domain information on the performance stability of named entity recognition models and presents an informative sample selection method for building high quality and stable namedentity recognition models across domains.
Abstract
When a machine learning-based named entity recognition system is employed in a new domain, its performance usually degrades. In this paper, we provide an empirical study on the impact of training data size and domain information on the performance stability of named entity recognition models. We present an informative sample selection method for building high quality and stable named entity recognition models across domains. Experimental results show that the performance of the named entity recognition model is enhanced significantly after being trained with these informative samples.
Keywords
Computer Science
arXiv (Cornell University)Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
1,574 Citations2002Erik F. Tjong Kim Sang
Elsevier eBooksHeterogeneous Uncertainty Sampling for Supervised Learning
1,142 Citations1994David Lewis, Jason Catlett
This work test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program) and finds that the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.
Machine LearningAn Algorithm that Learns What's in a Name
784 Citations1999Daniel M. Bikel, Richard Schwartz +1 more
IdentiFinderTM, a hidden Markov model that learns to recognize and classify names, dates, times, and numerical quantities, is evaluated and is competitive with approaches based on handcrafted rules on mixed case text and superior on text where case information is not available.
Employing EM and Pool-Based Active Learning for Text Classification
681 Citations1998Andrew McCallum, Kamal Nigam
The Query-by-Committee method of active learning is modified to use the unlabeled pool for explicitly estimating document density when selecting examples for labeling, and the combination of EM and active learning requires only slightly more than half as many labeled training examples to achieve the same accuracy as either the improved active learning or EM alone.
A maximum entropy approach to named entity recognition
467 Citations1999Ralph Grishman, Andrew Borthwick
This thesis describes a novel statistical named-entity recognition system known as MENE (Maximum Entropy Named Entity), and demonstrates the trans-lingual portability of the system, which was competitive with the best systems built by native Japanese speakers despite the fact that the author speaks no Japanese.
Named entity recognition through classifier combination
403 Citations2003Radu Florian, Abe Ittycheriah +2 more
A classifier-combination experimental framework for named entity recognition in which four diverse classifiers (robust linear classifier, maximum entropy, transformation-based learning, and hidden Markov model) are combined under different conditions is presented.
Efficient support vector classifiers for named entity recognition
324 Citations2002Hideki Isozaki, Hideto Kazawa
It is shown that an NE recognizer based on Support Vector Machines (SVMs) gives better scores than conventional systems, but off-the-shelf SVM classifiers are too inefficient for this task.
Multi-criteria-based active learning for named entity recognition
233 Citations2004Dan Shen, Jie Zhang +3 more
A multi-criteria-based active learning approach is proposed and effectively applied to named entity recognition and includes all the criteria using two selection strategies, both of which result in less labeling cost than single-criterion-based method.
Computational LinguisticsChinese Word Segmentation and Named Entity Recognition: A Pragmatic Approach
221 Citations2005Jianfeng Gao, Mu Li +2 more
A pragmatic mathematical framework in which segmenting known words and detecting unknown words of different types can be performed simultaneously in a unified way is proposed and implemented in an adaptive Chinese word segmenter, called MSRSeg, which is described in detail.
Named entity recognition with character-level models
195 Citations2003Dan Klein, Joseph Smarr +2 more
Two named-entity recognition models which use characters and character n-grams either exclusively or as an important part of their data representation are discussed, both of which are a character-level HMM with minimal context information and a maximum-entropy conditional markov model with substantially richer context features.
Text chunking based on a generalization of winnow
165 Citations2002Tong Zhang, Fred J. Damerau +1 more
A simple named entity extractor using AdaBoost
80 Citations2003Xavier Carreras, Lluı́s Màrquez +1 more
The system presented here consists of a replication, with some minor changes, of the system that obtained the best results in the CoNLL-2002 NEE task, and can be considered as a benchmark of the state–of–the– art technology for the current edition, and will allow also to make comparisons about the training corpora of both editions.
Description of the Kent Ridge Digital Labs System Used for MUC-7
55 Citations1998Senxiuyuan Yu, Shuanhu Bai +1 more
This work aims to build a single simple framework for tasks in text information extraction, for which, to a certain extent, the required information can be resolved locally, and is statistics-based.
Chinese named entity recognition based on multiple features
55 Citations2005Youzheng Wu, Jun Zhao +2 more
From the experimental results on People's Daily testing data, the proposed Hybrid Model is better than the models which only use one kind of features, and the experiments on MET-2 testing data also confirm the above conclusion.
Memory-based named entity recognition using unannotated data
40 Citations2003Fien De Meulder, Walter Daelemans
The memory-based learner Timbl (Daelemans et al., 2002) was used to find names in English and German newspaper text to show that gazetteers are not beneficial in the English case, while they are beneficial for the German data.
HowtogetaChineseName(Entity)
28 Citations2003Hongyan Jing, Radu Florian +3 more
It is demonstrated that classifier combination is an effective technique of improving system performance: experiments over a large annotated corpus of fine-grained entity types exhibit a 10% relative reduction in F-measure error.
Lecture notes in computer scienceChinese Named Entity Recognition Based on Multilevel Linguistic Features
17 Citations2005Honglei Guo, Jianmin Jiang +2 more
A Chinese named entity recognition system that employs the Robust Risk Minimization (RRM) classification method and incorporates the advantages of character-based and word-based models is presented.