Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task

This volume contains a description of the CoNLL-2011 Shared Task and the participating systems, which was based on the English portion of OntoNotes 4.0 corpus and aimed to identify anaphoric mentions and perform coreference resolution to create clusters of mentions representing the same entity or event in the text.

Abstract

This volume contains a description of the CoNLL-2011 Shared Task and the participating systems. This year, the shared task was based on the English portion of OntoNotes 4.0 corpus. The goal was to identify anaphoric mentions - both entities and events - and perform coreference resolution to create clusters of mentions representing the same entity or event in the text. The OntoNotes data spans five genres and multiple layers of annotation in addition to coreference, including parses, semantic roles, word sense, and named entities, making it a rich and diverse corpus. One of the challenges for the shared task participants (though they were limited by the time constraints of the task) and also for continuing research going forward is to find effective ways to bring these multiple layers of information to bear on the coreference task to improve upon the current state of the art. As is traditional with CoNLL, we had two tracks - an open and a closed track. Since world knowledge is an important factor in coreference resolution, even in the closed task participants were allowed to use some limited, outside sources, including WordNet and a pre-computed table predicting number and gender information for noun phrases. For the open task, as usual, participants were allowed to use any other source of information, such as Wikipedia, gazetteers, etc., that did not violate the evaluation criteria designed to protect the test set. A total of 23 participants submitted system outputs and 21 of them submitted system description papers. We hope that this data set will provide a useful benchmark and spur further research in this important sub-field of language processing.