Named Entity Recognition: Fallacies, challenges and opportunities
Computer Standards & InterfacesPublished 17 October 2012
Mónica Marrero, Julián Urbano, Sonia Sánchez-Cuadrado, Jorge Morato, Juan Miguel Gómez-Berbís
Citations235
SJR quartileQ1
SJR score1.14
SNIP1.76
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is argued that the Named Entity Recognition task is actually far from solved and the consequences for the development and evaluation of tools are shown.
Abstract
Named Entity Recognition serves as the basis for many other areas in Information Management. However, it is unclear what the meaning of Named Entity is, and yet there is a general belief that Named Entity Recognition is a solved task. In this paper we analyze the evolution of the field from a theoretical and practical point of view. We argue that the task is actually far from solved and show the consequences for the development and evaluation of tools. We discuss topics for further research with the goal of bringing the task back to the research scenario.
Keywords
Computer Science
now publishers, Inc. eBooksOpinion Mining and Sentiment Analysis
6,745 Citations2008Bo Pang, Lillian Lee
Foundations and Trends® in Information RetrievalOpinion Mining and Sentiment Analysis
6,204 Citations2008Bo Pang, Lillian Lee
This survey covers techniques and approaches that promise to directly enable opinion-oriented information-seeking systems and focuses on methods that seek to address the new challenges raised by sentiment-aware applications, as compared to those that are already present in more traditional fact-based analysis.
Naming and Necessity
5,279 Citations1972Saul A. Kripke
Lingvisticae InvestigationesA survey of named entity recognition and classification
2,488 Citations2007David R. Nadeau, Satoshi Sekine
Observations about languages, named entity types, domains and textual genres studied in the literature, along with other critical aspects of NERC such as features and evaluation methods, are reported.
Extracting product features and opinions from reviews
1,720 Citations2005Ana-Maria Popescu, Oren Etzioni
Opine is introduced, an unsupervised information-extraction system which mines reviews in order to build a model of important product features, their evaluation by reviewers, and their relative quality across products.
Kluwer international series in engineering and computer scienceOntology Learning for the Semantic Web
1,678 Citations2002Alexander Maedche
Message Understanding Conference-6
1,365 Citations1996Ralph Grishman, Beth Sundheim
MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted and the motivations for the new format.
BioinformaticsGENIA corpus—a semantically annotated corpus for bio-textmining
1,243 Citations2003JD Kim, Tomoko Ohta +2 more
Artificial IntelligenceUnsupervised named-entity extraction from the Web: An experimental study
1,128 Citations2005Oren Etzioni, Michael Cafarella +6 more
An overview of KnowItAll's novel architecture and design principles is presented, emphasizing its distinctive ability to extract information without any hand-labeled training examples, and three distinct ways to address this challenge are presented and evaluated.
IEEE Intelligent SystemsOntology learning for the Semantic Web
1,084 Citations2001Alexander Maedche, Steffen Staab
The authors present an ontology learning framework that extends typical ontology engineering environments by using semiautomatic ontology construction tools and encompasses ontology import, extraction, pruning, refinement and evaluation.
Research Design Explained
905 Citations1987Mark L. Mitchell, Janina M. Jolley
This book discusses research methodology, ethics and Validity, and a Beginning Researcher's Guide to Statistics, which helps new researchers evaluate research results and plan and conduct Ethical Research.
Journal of Web SemanticsSemantic annotation for knowledge management: Requirements and a survey of the state of the art
525 Citations2005Victoria Uren, Philipp Cimiano +5 more
This analysis shows that, while there is still some way to go before semantic annotation tools will be able to address fully all the knowledge management needs, research in the area is active and making good progress.
Lecture notes in computer scienceThe Philosophy of Information Retrieval Evaluation
385 Citations2002Ellen M. Voorhees
The fundamental assumptions and appropriate uses of the Cranfield paradigm, especially as they apply in the context of the evaluation conferences, are reviewed.
Survey of semantic annotation platforms
319 Citations2005Lawrence H. Reeve, Hyoil Han
This paper examines current Semantic Web annotation platforms that provide annotation and related services, and reviews their architecture, approaches and performance.
Knowledge Base Population: Successful Approaches and Challenges
311 Citations2011Heng Ji, Ralph Grishman
The techniques which can serve as a basis for a good KBP system are provided, the remaining challenges by comparison with traditional Information Extraction (IE) and Question Answering (QA) tasks are laid out, and some suggestions to address these challenges are provided.
Extracting company names from text
252 Citations2002Lisa F. Rau
A detailed description is given of an implemented algorithm that extracts company names automatically from financial news by combining heuristics, exception lists and extensive corpus analysis.
BioinformaticsProtein Structures and Information Extraction from BiologicalTexts: The PASTA System
159 Citations2002Robert Gaizauskas, George Demetriou +2 more
Elsevier eBooksInformation Extraction, Automatic
148 Citations2006Hamish Cunningham
The article covers the origins of IE and the factors relevant to its deployment in applications contexts, presents scenarios in which the technology has been applied, and breaks down the task into five subtasks and defines them.
Information Extraction Supported Question Answering
139 Citations1999Rohini K. Srihari, Wei Li
This study shows that information extraction system, Textract, can provide solid support for QA, and low-level IE like Named Entity tagging is often a necessary component in handling most types of questions.
An Unsupervised Method for General Named Entity Recognition and Automated Concept Discovery
133 Citations2004Enrique Alfonseca, Suresh Manandhar
This work describes here a procedure to automatically extend an ontology with domainspecific knowledge that is completely unsupervised, so it can be applied to different languages and domains.
Computer Standards & InterfacesSemantic annotations for web services discovery and composition
129 Citations2008Hassina Nacer, Djamil Aı̈ssani +1 more
This paper presents a model of Semantic Annotations for Web Services Discovery and its Composition that uses an inter-connected network of semantic Web services describing in OWL-S, using the similarity measure between concepts based on ontology, built before any submitted request.
Lecture notes in computer scienceOverview of the INEX 2009 Entity Ranking Track
123 Citations2010Gianluca Demartini, Tereza Iofciu +1 more
The XER tasks and the evaluation procedure used at the XER track in 2009, where a new version of Wikipedia was used as underlying collection are described; and the approaches adopted by the participants are summarized.
Overview of the TREC 2011 Entity Track.
103 Citations2011Krisztian Balog, Arjen P. de Vries +1 more
The goal of the entity track is to perform entity-oriented search tasks on the World Wide Web to answer many user information needs that would be better answered by specific entities instead of just any type of documents.
Teaching SociologyEvaluating Information: A Guide for Users of Social Science Research
102 Citations1999Jacqueline Carrigan, Jeffrey Katzer +2 more
Proper Name Extraction from Non-Journalistic Texts
84 Citations2001Thierry Poibeau, Leila Kosseim
The influence of the corpus on the automatic identification of proper names in texts is discussed and an approach to adapt a proper name extraction system developed for newspapers to the analysis of e-mail is described.
uO Research (University of Ottawa)Semi-supervised Named Entity Recognition: Learning to recognize 100 entity types with little supervision
81 Citations2007David R. Nadeau
It is shown that limited supervision can build complete NER systems, and the development of an acronym detection algorithm is introduced, thus solving a rare but very difficult problem in alias resolution.
Named entity discovery using comparable news articles
77 Citations2004Yusuke Shinyama, Satoshi Sekine
A way to discover Named Entities by using the distribution of words in news articles, which can be used to strengthen the lexical knowledge of a Named Entity tagger.
Evaluation of Named Entity Extraction Systems
48 Citations2009Mónica Marrero, Sonia Sánchez-Cuadrado +2 more
The sufficiency of the technical characteristics of the tools and their evaluation ratios, presents an objective perspective of the quality and the effectiveness of the recognition and classification techniques of each tool.
Automatic adaptation of proper noun dictionaries through cooperation of machine learning and probabilistic methods
47 Citations2000Georgios Petasis, Alessandro Cucchiarelli +4 more
This paper proposes a technique for the automatic updating of an Pn Dictionary through the cooperation of an inductive and a probabilistic classifier, and shows that, whenever an existing PN Dictionary allows the identification of 50% of the proper nouns within a corpus, this technique allows, without additional manual effort, the successful recognition of the remaining 50%.
ACM SIGIR ForumAgainst recall
28 Citations2009Justin Zobel, Alistair Moffat +1 more
It is concluded that use of recall as a measure of the effectiveness of ranked querying is indefensible, due to the lack of a plausible user model in which recall is a measured measure of satisfaction.
Computer Standards & InterfacesProviding standard-oriented data models and interfaces to eGovernment services
24 Citations2008Luís Álvarez Sabucedo, Luis Anido +2 more
This paper tackles the provision of a front-end solution to access services in Public Administrations by proposing a semantic-based approach centered in citizens and a complete software interface within a holistic model of service.
Information Retrieval Meta-Evaluation: Challenges And Opportunities In The Music Domain.
23 Citations2011Julián Urbano
A survey of past meta-evaluation work in the context of Text Information Retrieval argues that the music community still needs to address various issues concerning the evaluation of music systems and the IR cycle, pointing out directions for further research and proposals in this line.
INEX 2009 workshop pre-proceedings
19 Citations2009S. Geva, Jaap Kamps +1 more
QT can potentially provide the theoretical basis of new genre of information processing models more aligned with human cognition, and quantum inference effects currently being used to fit models of human decision making may be applied to model interference between different dimensions of relevance.
Entity extraction is a boring solved problem
17 Citations2007Marc Vilain, Jennifer Su +1 more
Empirical results that contradict the prevailing opinion that entity extraction is a boring solved problem are presented and an error analysis is given that suggests research challenges for entity extraction that are neither boring nor solved.
arXiv (Cornell University)Information Retrieval Systems Adapted to the Biomedical Domain
4 Citations2012Mónica Marrero, Sonia Sánchez-Cuadrado +3 more