Attribute selection for referring expression generation
Published 1 January 2008Open access
Albert Gatt, Anja Belz
Citations25
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper analyses the systems that participated in the first Shared Task Challenge in NLG in terms of their algorithmic properties, comparing new techniques to classic ones, based on results from a new human task-performance experiment and from the intrinsic measures that were used in the Challenge.
Abstract
Referring expression generation has recently been the subject of the first Shared Task Challenge in NLG. In this paper, we analyse the systems that participated in the Challenge in terms of their algorithmic properties, comparing new techniques to classic ones, based on results from a new human task-performance experiment and from the intrinsic measures that were used in the Challenge. We also consider the relationship between different evaluation methods, showing that extrinsic task-performance experiments and intrinsic evaluation methods yield results that are not significantly correlated. We argue that this highlights the importance of including extrinsic evaluation methods in comparative NLG evaluations.
Keywords
Computer Science
Behavior Research Methods, Instruments, & ComputersDMDX: A Windows display program with millisecond accuracy
2,750 Citations2003Kenneth I. Forster, Jonathan C. Forster
DMDX is a Windows-based program designed primarily for language-processing experiments that uses the features of Pentium class CPUs and the library routines provided in DirectX to provide accurate timing and synchronization of visual and audio output.
Cognitive ScienceComputational Interpretations of the Gricean Maxims in the Generation of Referring Expressions
731 Citations1995Robert Dale, Ehud Reiter
A recommended algorithm is described, along with a specification of the resources a host system must provide in order to make use of the algorithm, and an implementation used in the natural language generation component of the IDAS system.
Journal of Memory and LanguageDo speakers and listeners observe the Gricean Maxim of Quantity?
333 Citations2006Paul E. Engelhardt, Kate Bailey +1 more
Artificial IntelligenceLessons from a failure: Generating tailored smoking cessation letters
229 Citations2003Ehud Reiter, Roma Robertson +1 more
Although it is rare for AI papers to present negative results, it is believed that useful lessons can be learned from STOP and the AI community as a whole could benefit from considering the issue of how, when, and why negative results should be reported.
Cooking up referring expressions
220 Citations1989Robert Dale
This paper describes the referring expression generation mechanisms used in EPICURE, a computer program which produces natural language descriptions of cookery recipes with an underlying ontology which permits the representation of non-singular entities and a PATR-like unification grammar to produce surface linguistic strings.
Artificial IntelligencePlanning english referring expressions
167 Citations1985Douglas E. Appelt
A theory of language generation based on planning makes it possible for one to account for noun phrases that refer, that inform the hearer of additional information, and that are coordinated with the speaker's physical actions to clarify his communicative intent.
Columbia Academic Commons (Columbia University)Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation
137 Citations2006Rebecca J. Passonneau
MASI, distance metric for comparing sets, is discussed and its use in quantifying the reliability of a specific dataset is illustrated, and it is argued that a paradigmatic reliability study should relate measures of inter-annotator agreement to independent assessments, such as significance tests of the annotated variables with respect to other phenomena.
Computational LinguisticsGenerating Referring Expressions: Boolean Extensions of the Incremental Algorithm
119 Citations2002Kees van Deemter
A logical perspective to the generation of referring expressions is brought, addressing the incompleteness of existing algorithms in this area, and generalizations and extensions of the Incremental Algorithm of Dale and Reiter (1995) are proposed.
Evaluating algorithms for the generation of referring expressions using a balanced corpus
105 Citations2007Albert Gatt, Ielka van der Sluis +1 more
A corpus-based evaluation methodology, applied to a number of classic algorithms, that focuses on balance and semantic transparency to enable comparison of human and algorithmic output.
Incremental generation of spatial referring expressions in situated dialog
93 Citations2006John D. Kelleher, Geert-Jan M. Kruijff
This paper addresses the issue of combinatorial explosion inherent in the construction of relational context models by contextually defining the set of objects in the context that may function as a landmark and sequencing the order in which spatial relations are considered using a cognitively motivated hierarchy of relations.
The European Journal of Cognitive PsychologyTracking the time course of multidimensional stimulus discrimination: Analyses of viewing patterns and processing times during “same”-“different“ decisions
90 Citations2002Eva Belke, Antje S. Meyer
Generating minimal definite descriptions
88 Citations2001Claire Gardent
An alternative, constraint-based algorithm is presented that builds on existing related algorithms in that it produces minimal descriptions for sets of individuals using positive, negative and disjunctive properties and is integrated with surface realisation.
Generating referring expressions involving relations
88 Citations1991Robert Dale, Nicholas J. Haddock
Dale's algorithm for determining the content of a referring expression is reviewed, which only permits the use of one-place predicates, revised and extended to deal with n-ary predicates.
Computational LinguisticsGenerating Referring Expressions that Involve Gradable Properties
78 Citations2006Kees van Deemter
A simple semantic analysis of vague descriptions that reflects the context-dependent meaning of the adjectives in them is proposed and shown how this type of analysis can inform algorithms for the generation of vague describe from numerical data.
Algorithms for generating referring expressions
64 Citations2006Jette Viethen, Robert Dale
This paper compares the output of three existing algorithms against a data set consisting of human-generated referring expressions, and identifies a number of significant differences between what people do and what these algorithms do.
The computational complexity of avoiding conversational implicatures
57 Citations1990Ehud Reiter
OAR@UM (University of Malta)The Attribute Selection for GRE Challenge: Overview and Evaluation Results
50 Citations2007Anja Belz, Albert Gatt
) Challenge was the first shared-task evaluation challenge of Natural Language Generation and all submitted systems were tested automat-ically for minimality, uniqueness and ‘hu-manlikeness’.
International Joint Conference on Artificial IntelligenceA computational model of referring
49 Citations1987Douglas E. Appelt, Amichai Kronfeld
It is concluded that FSH, T, and temperature play a role in the development of high TER by Sc monolayers (formation of tight junctions) in vitro, suggesting the synergistic effect of the two hormones on TER development.
Intrinsic vs. extrinsic evaluation measures for referring expression generation
48 Citations2008Anja Belz, Albert Gatt
It is found that there are no significant correlations between intrinsic and extrinsic evaluation measures for this task, and the kind of intrinsic evaluation metrics characteristic of current comparative HLT evaluation are applied.
Lecture notes in computer scienceOn Referring to Sets of Objects Naturally
33 Citations2004Helmut Horacek
This work applies and enhances a best-first searching procedure, which enables it to produce a larger variety of expressions of limited complexity, and incorporates restrictions which express constraints on preferred surface forms.
University of Groningen research database (University of Groningen / Centre for Information Technology)Evaluating Algorithms for the Generation of Referring Expressions: Going beyond Toy Domains.
20 Citations2007Ielka van der Sluis, Albert Gatt +1 more
Results indicate that state of the art algorithms perform very differ- ently when applied to a complex domain, and should contribute to a growing debate on the evaluation of nlg systems, arguing in favour of carefully constructed balanced and semanti- cally transparent corpora.
Edinburgh Research Explorer (University of Edinburgh)Multi-lingual Evaluation of a Natural Language Generation System
19 Citations2004Αθανάσιος Καρασίμος, Amy Isard
A user evaluation of the text output from the M-PIRO (Multilingual Personalised Information Objects) system, which dynamically generates descriptions of exhibits for a virtual museum, shows that subjects performed significantly better in a factual recall test when the descriptions included more sophisticated text structuring modules.
OAR@UM (University of Malta)Incremental Generation of Plural Descriptions: Similarity and Partitioning
10 Citations2007Albert Gatt, Kees van Deemter
A corpus-based study of plural descriptions, and a psycholinguisticallymotivated algorithm for plural reference generation that is based on partitioning and incorporates corpusderived heuristics is described.