SemEval-2010 Task 1: Coreference Resolution in Multiple Languages
Published 15 July 2010
Marta Vilar Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martí, Mariona Taulé, Véronique Hoste
Citations150
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper presents the SemEval-2010 task on Coreference Resolution in Multiple Languages, which was to evaluate and compare automatic coreference resolution systems for six different languages in four evaluation settings and using four different metrics.
Abstract
This paper presents the SemEval-2010 task on Coreference Resolution in Multiple Languages. The goal was to evaluate and compare automatic coreference resolution systems for six different languages (Catalan, Dutch, English, German, Italian, and Spanish) in four evaluation settings and using four different metrics. Such a rich scenario had the potential to provide insight into key issues concerning coreference resolution: (i) the portability of systems across languages, (ii) the relevance of different levels of linguistic information, and (iii) the behavior of scoring metrics. 1
Keywords
Computer Science
Text, speech and language technologyImprovements in Part-of-Speech Tagging with an Application to German
821 Citations1999Helmut Schmid
This paper presents a meta-modelling system that automates the very labor-intensive and therefore time-heavy and expensive process of manually tagging part-of-speech content in a variety of languages.
A model-theoretic coreference scoring scheme
682 Citations1995Marc Vilain, John D. Burger +3 more
This note describes a scoring scheme for the coreference task in MUC6 that improves on the original approach by grounding the scoring scheme in terms of a model; producing more intuitive recall and precision scores; and not requiring explicit computation of the transitive closure of coreference.
On coreference resolution performance metrics
535 Citations2005Xiaoqiang Luo
The paper proposes a Constrained Entity-Alignment F-Measure (CEAF) for evaluating coreference resolution and shows that the best alignment is a maximum bipartite matching problem which can be solved by the Kuhn-Munkres algorithm.
Natural Language EngineeringBLANC: Implementing the Rand index for coreference evaluation
200 Citations2010Marc Recasens, Eduard Hovy
The BiLateral Assessment of Noun-Phrase Coreference rewards both coreference and non-coreference links by averaging the F-scores of the two types, and does not ignore singletons – the main problem with the MUC score – and does not inflate the score in their presence.
Estimation of conditional probabilities with decision trees and an application to fine-grained POS tagging
181 Citations2008Helmut Schmid, Florian Laws
A HMM part-of-speech tagging method which is particularly suited for POS tagsets with a large number of fine-grained tags based on splitting of the POS tags into attribute vectors and decomposition of the contextual POS probabilities of the HMM into a product of attribute probabilities.
Information Processing & ManagementTwo uses of anaphora resolution in summarization
161 Citations2007Josef Steinberger, Massimo Poesio +2 more
A new method for using anaphoric information in Latent Semantic Analysis (LSA) is proposed, and its application to develop an LSA-based summarizer which achieves a significantly better performance than a system not using anphoric information.
International Conference on Semantic Computing (ICSC 2007)FSM-based model for spatio-temporal event recognition for HCS
119 Citations2007Apurba Pradhan, Eduard Hovy +3 more
A finite state machine (FSM)-based model for specification and identification of spatio-temporal events at the single-cell level is presented and results for a time-lapse apoptosis screen are presented.
arXiv (Cornell University)Memory-Based Shallow Parsing
118 Citations1999Walter Daelemans, Sabine Buchholz +1 more
International Joint Conference on Artificial IntelligenceUsing decision trees for conference resolution
97 Citations1995Joseph F. McCarthy, Wendy G. Lehnert
Language Resources and EvaluationAnCora-CO: Coreferentially annotated corpora for Spanish and Catalan
85 Citations2009Marta Vilar Recasens, M. Antònia Martí
The enrichment of the AnCora corpora of Spanish and Catalan with coreference links between pronouns, full noun phrases, and discourse segments makes it possible to train and test learning-based algorithms for automatic coreference resolution, as well as to carry out bottom-up linguistic descriptions of coreference relations as they occur in real data.
MADOC (University of Mannheim)BART: A Multilingual Anaphora Resolution System
42 Citations2010Samuel Broscheit, Massimo Poesio +6 more
BART is a highly modular toolkit for coreference resolution that supports state-of-the-art statistical approaches and enables efficient feature engineering and relies on a maximum entropy-based classifier for pairs of mentions.
A unified representation for morphological, syntactic, semantic, and referential annotations
39 Citations2005Erhard Hinrichs, Sandra Kübler +1 more
This paper reports on the SYN-RA (SYNtax-based Reference Annotation) project, an on-going project of annotating German newspaper texts with referential relations that has developed an inventory of anaphoric and coreference relations for German in the context of a unified, XML-based annotation scheme for combining morphological, syntactic, semantic, andAnaphoric information.
Using Coreference for Question Answering
39 Citations1999Thomas Morton, Thomas S. Morton
Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus
36 Citations2010Kepa Joseba Rodríguez, Francesca Delogu +3 more
The Live Memories corpus is an Italian corpus annotated for anaphoric relations that contains texts from the Italian Wikipedia about the region Trentino/Süd Tirol and from blog sites with users' comments.
A dependency-driven parser for German dependency and constituency representations
31 Citations2008Johan Hall, Joakim Nivre
A dependency-driven parser that parses both dependency structures and constituent structures that is able to recover both constituent labels and grammatical functions with an F-Score over 75% for TuBa-D/Z and over 65% for TIGER.
Meeting of the Association for Computational LinguisticsSUCRE: A Modular System for Coreference Resolution
29 Citations2010Hamidreza Kobdani, Hinrich Schütze
SUCRE is a new software tool that is able to separately do noun, pronoun and full coreference resolution and its feature engineering based on a relational database model and a regular feature definition language.
Meeting of the Association for Computational LinguisticsUBIU: A Language-Independent System for Coreference Resolution
25 Citations2010Desislava Zhekova, Sandra Kübler
UBIU is presented, a language independent system for detecting full coreference chains, composed of named entities, pronouns, and full noun phrases which makes use of memory based learning and a feature model following Rahman and Ng (2009).
Using coreference for question answering
24 Citations1999Thomas Morton
An evaluation of a system which retrieves answers to queries based on coreference relationships between entities and events in the query and documents demonstrates that the amount of information that the user must process, to find an answer to their query, is reduced by an order of magnitude.
Anaphora Resolution Exercise: An overview
23 Citations2008Constantin Orǎsan, Dan Cristea +2 more
The aim of the Anaphora Resolution Exercise (ARE) was not about finding why certain methods are better, but to prepare the ground for a fully-fledged edition.
CINECA IRIS Institutial research information system (University of Pisa)TANL-1: Coreference Resolution by Parse Analysis and Similarity Clustering
18 Citations2010Giuseppe Attardi, Maria Simi +1 more
The authors' submission to the Semeval 2010 task on coreference resolution in multiple languages is based on parse analysis and similarity clustering, using a binary classifier based on Maximum Entropy to decide whether or not there is a relationship between each pair of mentions extracted from a textual document.
RelaxCor: A Global Relaxation Labeling Approach to Coreference Resolution
17 Citations2010Emili Sapena, Lluís Padró +1 more
Corry: A System for Coreference Resolution
15 Citations2010Olga Uryupina
Corry is a system for coreference resolution in English that supports both local and global models of coreference, and has shown the best performance level among all the systems in their track for the corresponding metric.
KNACK-2002: a Richly Annotated Corpus of Dutch Written Text
12 Citations2006Véronique Hoste, Guy De Pauw
The annotated KNACK-2002 corpus of Dutch written text is introduced, providing researchers with a useful gold standard data set for different NLP tasks in the domains of morphology, (morpho)syntax, semantics and discourse.