Design challenges and misconceptions in named entity recognition
Published 1 January 2009Open access
Lev Ratinov, Dan Roth
Citations1,476
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system are analyzed, and several solutions to these challenges are developed.
Abstract
We analyze some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system. In particular, we address issues such as the representation of text chunks, the inference approach needed to combine local NER decisions, the sources of prior knowledge and how to use them within an NER system. In the process of comparing several solutions to these challenges we reach some surprising conclusions, as well as develop an NER system that achieves 90.8 F1 score on the CoNLL-2003 NER shared task, the best reported result for this dataset.
Keywords
Computer ScienceDecision Sciences
Proceedings of the IEEEA tutorial on hidden Markov models and selected applications in speech recognition
22,785 Citations1989L. R. Rabiner
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Incorporating non-local information into information extraction systems by Gibbs sampling
3,035 Citations2005Jenny Rose Finkel, Trond Grenager +1 more
By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorporate non-local structure while preserving tractable inference.
Class-based n -gram models of natural language
2,893 Citations1992Peter F. Brown, P.V. deSouza +3 more
This work addresses the problem of predicting a word from previous words in a sample of text and discusses n-gram models based on classes of words, finding that these models are able to extract classes that have the flavor of either syntactically based groupings or semanticallybased groupings, depending on the nature of the underlying statistics.
Feature-rich part-of-speech tagging with a cyclic dependency network
2,851 Citations2003Kristina Toutanova, Dan Klein +2 more
A new part-of-speech tagger is presented that demonstrates the following ideas: explicit use of both preceding and following tag contexts via a dependency network representation, broad use of lexical features, and effective use of priors in conditional loglinear models.
Discriminative training methods for hidden Markov models
1,889 Citations2002Michael Collins
Experimental results on part-of-speech tagging and base noun phrase chunking are given, in both cases showing improvements over results for a maximum-entropy tagger.
Predicting good probabilities with supervised learning
1,617 Citations2005Alexandru Niculescu-Mizil, Rich Caruana
The relationship between the predictions made by different learning algorithms and true posterior probabilities is examined, showing that maximum margin methods such as boosted trees and boosted stumps push probability mass away from 0 and 1 yielding a characteristic sigmoid shaped distortion in the predicted probabilities.
arXiv (Cornell University)Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
1,574 Citations2002Erik F. Tjong Kim Sang
Artificial IntelligenceUnsupervised named-entity extraction from the Web: An experimental study
1,128 Citations2005Oren Etzioni, Michael Cafarella +6 more
An overview of KnowItAll's novel architecture and design principles is presented, emphasizing its distinctive ability to extract information without any hand-labeled training examples, and three distinct ways to address this challenge are presented and evaluated.
Large margin classification using the perceptron algorithm
1,099 Citations1998Yoav Freund, Robert E. Schapire
Learning dictionaries for information extraction by multi-level bootstrapping
688 Citations1999Ellen Riloff, Rosie Jones
A multilevel bootstrapping algorithm is presented that generates both the semantic lexicon and extraction patterns simultaneously simultaneously and produces high-quality dictionaries for several semantic categories.
Representing text chunks
449 Citations1999Erik F. Tjong Kim Sang, Jorn Veenstra
It is shown that the the data representation choice has a minor influence on chunking performance, however, equipped with the most suitable data representation, the memory-based learning chunker was able to improve the best published chunking results for a standard data set.
RECERCAT (Consorci de Serveis Universitaris de Catalunya)Simple Semi-supervised Dependency Parsing
444 Citations2008Terry Koo, Xavier Carreras +1 more
This work focuses on the problem of lexical representation, introducing features that incorporate word clusters derived from a large unannotated corpus, and shows that the cluster-based features yield substantial gains in performance across a wide range of conditions.
Named entity recognition through classifier combination
403 Citations2003Radu Florian, Abe Ittycheriah +2 more
A classifier-combination experimental framework for named entity recognition in which four diverse classifiers (robust linear classifier, maximum entropy, transformation-based learning, and hidden Markov model) are combined under different conditions is presented.
DSpace@MIT (Massachusetts Institute of Technology)Semi-Supervised Learning for Natural Language
304 Citations2005Percy Liang
This thesis focuses on two segmentation tasks, named-entity recognition and Chinese word segmentation, and shows that features derived from unlabeled data substantially improves performance, both in terms of reducing the amount of labeled data needed to achieve a certain performance level and in termsof reducing the error using a fixed amount of labeling data.
Exploiting Wikipedia as External Knowledge for Named Entity Recognition
277 Citations2007Jun’ichi Kazama, Kentaro Torisawa
It is demonstrated using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER.
Name Tagging with Word Clusters and Discriminative Training
270 Citations2004S.L. Miller, Jethran Guinness +1 more
A technique for augmenting annotated training data with hierarchical word clusters that are automatically derived from a large unannotated corpus that achieves a 25% reduction in error over the state-of-the-art HMM trained on the same material.
Named entity recognition with a maximum entropy approach
227 Citations2003Hai Leong Chieu, Hwee Tou Ng
The named entity recognition (NER) task involves identifying noun phrases that are names, and assigning a class to each name.
Exploiting dictionaries in named entity extraction
224 Citations2004William W. Cohen, Sunita Sarawagi
A semi-Markov extraction process is formalized, which is based on sequentially classifying segments of several adjacent words, rather than single words, and provides a more natural formulation of the NER problem than sequential word classification.
A high-performance semi-supervised learning method for text chunking
223 Citations2005Rie Kubota Ando, Tong Zhang
A novel semi-supervised method that employs a learning paradigm which is to find "what good classifiers are like" by learning from thousands of automatically generated auxiliary classification problems on unlabeled data, which produces performance higher than the previous best results.
An effective two-stage model for exploiting non-local dependencies in named entity recognition
174 Citations2006Vijay Krishnan, Christopher D. Manning
This paper shows that a simple two-stage approach to handle non-local dependencies in Named Entity Recognition (NER) can outperform existing approaches that handleNon- local dependencies, while being much more computationally efficient.
Semi-Supervised Sequential Labeling and Segmentation Using Giga-Word Scale Unlabeled Data
130 Citations2008Jun Suzuki, Hideki Isozaki
Evidence that the use of more unlabeled data in semi-supervised learning can improve the performance of Natural Language Processing tasks, such as part-of-speech tagging, syntactic chunking, and named entity recognition is provided.
A proposal to automatically build and maintain gazetteers for Named Entity Recognition by using Wikipedia
119 Citations2006Antonio Toral, Rafael Muñoz
Part of speech tagging using a network of linear separators
103 Citations1998Dan Roth, Dmitry Zelenko
An architecture and an on-line learning algorithm are presented that utilizes this mistake-driven algorithm for multi-class prediction-selecting the part of speech of a word and it is shown that the algorithm performs comparably to the best known algorithms for POS.
Shallow parsing using specialized hmms
90 Citations2002Antonio Molina, Ferran Plà
A robust risk minimization based named entity recognition system
80 Citations2003Tong Zhang, David E. Johnson
It is shown that the system performance can be enhanced significantly with some relative simple token-based features that are available for many languages, and more sophisticated linguistic features provide much less improvement than might be expected.
Lecture notes in computer scienceVoting Between Multiple Data Representations for Text Chunking
55 Citations2005Hong Shen, Anoop Sarkar
This paper focuses on choosing specific data representations combined with simple majority voting and achieves high scores for arbitrary phrase identification and Base NP identification on the community standard CoNLL-2000 data set.
A New Perceptron Algorithm for Sequence Labeling with Non-Local Features
38 Citations2007Jun’ichi Kazama, Kentaro Torisawa
This work proposes a new perceptron algorithm that can use non-local features whose values are determined from the sequence and the labels and presents experimental results from the CoNLL 2003 named entity recognition (NER) task.
Learning a perceptron-based named entity chunker via online recognition feedback
32 Citations2003Xavier Carreras, Lluı́s Màrquez +1 more
A novel approach for the problem of Named Entity Recognition and Classification (NERC) is presented, in the context of the CoNLL-2003 Shared Task.
International Conference on Semantic Computing (ICSC 2007)Semantic Interpretation of the Web without the Semantic Web: Toward Business-Aware Web Processors
31 Citations2007Peter Z. Yeh, Daniel R. Farina +1 more
A family of systems that are developing to make systematic use of the Web in support of decision-makers, which are called business-aware web processors, use semantic representations but do not assume that these representations will come from the Semantic Web.
International Conference on Semantic Computing (ICSC 2007)Modeling Discriminative Global Inference
7 Citations2007Nicholas Rizzolo, Dan Roth
Finding Good Sequential Model Structures using Output Transformations
1 Citations2007Edward Loper
This paper shows how the notion of output transformation can be used to explore a variety of alternative model structures, thus allowing us to capture longer-distance consistencies while avoiding unnecessary increases to the model’s parameter space.