Generating Expressions that Refer to Visible Objects
Aberdeen University Research Archive (Aberdeen University)Published 1 June 2013Open access
Margaret Mitchell, Kees van Deemter, Ehud Reiter
Citations69
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This research highlights the importance of knowing the carrier and removal status of canine coronavirus, as a source of infection for other animals, not necessarily belonging to the same breeds.
Abstract
We introduce a novel algorithm for generat-ing referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates abso-lute properties like color from relative proper-ties like size to stochastically generate a di-verse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm out-performs the well-known Incremental Algo-rithm (Dale and Reiter, 1995) and the Graph-Based Algorithm (Krahmer et al., 2003; Vi-ethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm’s non-determinism into account. 1
Keywords
Computer Science
ACM SIGKDD Explorations NewsletterThe WEKA data mining software
17,849 Citations2009Mark Hall, Eibe Frank +4 more
This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.
Naval Research Logistics QuarterlyThe Hungarian method for the assignment problem
12,574 Citations1955Harold W. Kuhn
Cognitive PsychologyA feature-integration theory of attention
12,207 Citations1980Anne Treisman, Garry A. Gelade
A new hypothesis about the role of focused attention is proposed, which offers a new set of criteria for distinguishing separable from integral features and a new rationale for predicting which tasks will show attention limits and which will not.
Speech Acts: An Essay in the Philosophy of Language
6,641 Citations1969John R. Searle
Nature reviews. NeuroscienceComputational modelling of visual attention
4,755 Citations2001Laurent Itti, Christof Koch
Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment, providing a framework for a computational and neurobiological understanding of visual attention.
Journal of the Society for Industrial and Applied MathematicsAlgorithms for the Assignment and Transportation Problems
3,984 Citations1957James Munkres
CERN Document Server (European Organization for Nuclear Research)Natural Language Processing with Python
3,449 Citations2009Steven Bird, Ewan Klein +1 more
2009 IEEE Conference on Computer Vision and Pattern RecognitionDescribing objects by their attributes
1,928 Citations2009Ali Farhadi, Ian Endres +2 more
This paper proposes to shift the goal of recognition from naming to describing, and introduces a novel feature selection method for learning attributes that generalize well across categories.
Computer Vision Graphics and Image ProcessingPreattentive processing in vision
930 Citations1985Anne Treisman
Visual analysis appears to be functionally divided between an early preattentive level of processing at which simple features are coded spatially in parallel and a later stage at which focused attention is required to conjoin the separate features into coherent objects.
Cognitive ScienceComputational Interpretations of the Gricean Maxims in the Generation of Referring Expressions
731 Citations1995Robert Dale, Ehud Reiter
A recommended algorithm is described, along with a specification of the resources a host system must provide in order to make use of the algorithm, and an implementation used in the natural language generation component of the IDAS system.
CognitionWhen do speakers take into account common ground?
709 Citations1996William S. Horton, Boaz Keysar
The results suggest that speakers do not engage in audience design in the initial planning of utterances; instead, they monitor those plans for violations of common ground.
Advances in psychologyAudience Design in Meaning and Reference
494 Citations1982Herbert H. Clark, Gregory L. Murphy
Nature NeuroscienceThe representation of perceived angular size in human primary visual cortex
485 Citations2006Scott O. Murray, Hüseyin Boyacı +1 more
FMRI shows that the retinal size of an object and the depth information in a scene are combined early in the human visual system, and a distant object that appears to occupy a larger portion of the visual field activates a larger area in V1.
Journal of Verbal Learning and Verbal BehaviorCommon ground at the understanding of demonstrative reference
440 Citations1983Herbert H. Clark, Robert Schreuder +1 more
Learning Visual Attributes
397 Citations2007Vittorio Ferrari, Andrew Zisserman
It is shown that attributes can be learnt starting from a text query to Google image search, and can then be used to recognize the attribute and determine its spatial extent in novel real-world images.
Nature NeuroscienceThe surface area of human V1 predicts the subjective experience of object size
380 Citations2010D. Samuel Schwarzkopf, Chen Song +1 more
It is found that the surface area of V1 predicts variability in conscious experience and is strongly and negatively correlated with the magnitude of two common visual illusions.
Journal of Memory and LanguageDo speakers and listeners observe the Gricean Maxim of Quantity?
333 Citations2006Paul E. Engelhardt, Kate Bailey +1 more
Computational LinguisticsComputational Generation of Referring Expressions: A Survey
333 Citations2011Emiel Krahmer, Kees van Deemter
The REG problem is introduced and early work in this area is described, discussing what basic assumptions lie behind it, and showing how its remit has widened in recent years.
Cooking up referring expressions
220 Citations1989Robert Dale
This paper describes the referring expression generation mechanisms used in EPICURE, a computer program which produces natural language descriptions of cookery recipes with an underlying ontology which permits the representation of non-singular entities and a PATR-like unification grammar to produce surface linguistic strings.
Current BiologyAttention-Dependent Representation of a Size Illusion in Human V1
218 Citations2008Fang Fang, Hüseyin Boyacı +2 more
Using functional magnetic resonance imaging, it was found that the spatial distribution of V1 activity induced by the far ring was also shifted toward a more eccentric representation of the visual field, whereas that inducing by the close ring was shifted toward the foveal representation, consistent with their perceptual appearances.
Psychological ScienceSpeakers' Overestimation of Their Effectiveness
209 Citations2002Boaz Keysar, Anne S. Henly
Computational LinguisticsGraph-Based Generation of Referring Expressions
205 Citations2003Emiel Krahmer, Sebastiaan van Erk +1 more
A scene is proposed to formalize a scene as a labeled directed graph and describe content selection as a subgraph construction problem that allows for an integration of rule-based generation techniques with more recent stochastic approaches.
Journal of Memory and LanguageWatching the eyes when talking about size: An investigation of message formulation and utterance planning☆
204 Citations2006Sarah Brown‐Schmidt, Michael K. Tanenhaus
Computational LinguisticsAn Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems
197 Citations2009Ehud Reiter, Anja Belz
The results of two studies of how well some metrics which are popular in other areas of NLP correlate with human judgments in the domain of computer-generated weather forecasts suggest that, at least in this domain, metrics may provide a useful measure of language quality, although the evidence for this is not as strong as one would ideally like to see.
Data Archiving and Networked Services (DANS)Efficient context-sensitive generation of referring expressions
112 Citations2002Emiel Krahmer, Mariët Theune
A Modification of the Algorithm Based on Salience: Determining the Context Set and Outline of the Modified Algorithm.
The use of spatial relations in referring expression generation
106 Citations2008Jette Viethen, Robert Dale
It is determined that, even in simple scenes where the use of relations is not strictly required in order to identify an entity, relations are in fact often used.
A fast algorithm for the generation of referring expressions
103 Citations1992Ehud Reiter, Robert Dale
A specification of the resources a host system must provide in order to make use of the algorithm, and an implementation used in the IDAS system are described.
The TUNA-REG Challenge 2009
95 Citations2009Albert Gatt, Anja Belz +1 more
The GREC Task at REG '08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia, and the evaluation methods were described and described.
Child DevelopmentThe Development of Communication: Changes with Age and Modeling
93 Citations1976Grover J. Whitehurst
Computational LinguisticsApplying Computational Models of Spatial Prepositions to Visually Situated Dialog
83 Citations2008John D. Kelleher, Fintan Costello
A generic architecture for a visually situated dialog system is described and the interactions between the spatial cognition module, which provides the interface to the models of prepositional semantics, and the other components in the architecture are highlighted.
Cognitive ScienceGeneration of Referring Expressions: Assessing the Incremental Algorithm
81 Citations2011Kees van Deemter, Albert Gatt +2 more
It turns out that the success of the IA depends substantially on the ''preference order'' (PO) employed by the IA, particularly in complex domains, and there are no compelling reasons for preferring the IA over some of its main competitors on these grounds.
Building a semantically transparent corpus for the generation of referring expressions
79 Citations2006Kees van Deemter, Ielka van der Sluis +1 more
Journal of Experimental Child PsychologyThe elaboration of the noun phrase in children's description of objects
72 Citations1975William Ford, David R. Olson
Topics in Cognitive ScienceToward a Computational Psycholinguistics of Reference Production
60 Citations2012Kees van Deemter, Albert Gatt +2 more
It is argued that computational and psycholinguistic approaches to reference production can benefit from closer interaction, and that this is likely to result in the construction of algorithms that differ markedly from the ones currently known in the computational literature.
Language Cognition and NeuroscienceSpeaking from experience: audience design as expert performance
58 Citations2012Timothy M. Gann, Dale J. Barr
Computational LinguisticsHuman Variation and Lexical Choice
56 Citations2002Ehud Reiter, Somayajulu Sripada
Evidence that different people probably associate slightly different meanings with words in a language community is summarized and its implications for natural language generation, especially for lexical choice, that is, choosing appropriate words for a generated text are discussed.
Intrinsic vs. extrinsic evaluation measures for referring expression generation
48 Citations2008Anja Belz, Albert Gatt
It is found that there are no significant correlations between intrinsic and extrinsic evaluation measures for this task, and the kind of intrinsic evaluation metrics characteristic of current comparative HLT evaluation are applied.
Referring expressions as formulas of description logic
44 Citations2008Carlos Areces, Alexander Koller +1 more
The problem of generating referring expressions (GRE) is reinterpreted as the problem of computing a formula in a description logic that is only satisfied by the referent, which offers a new unifying perspective under which existing GRE algorithms can be compared.
Artificial IntelligenceDynamically structuring, updating and interrelating representations of visual and linguistic discourse context
41 Citations2005John D. Kelleher, Fintan Costello +1 more
It is argued that computer systems attempting to model the evolving context of a visually situated discourse should integrate models of visual and linguistic salience within their natural language processing (NLP) framework and a real-time, natural language virtual reality (NLVR) system is developed that implements an NLP framework based on both visual and language salience.
Speaker-Dependent Variation in Content Selection for Referring Expression Generation
36 Citations2010Jette Viethen, Robert Dale
Journal of Psycholinguistic ResearchThe development of referential communication skills: Some situations in which speakers give redundant messages
25 Citations1985Susan Sonnenschein
These two studies investigated developmental and contextual changes in the production of redundant messages a type often used by speakers, and suggested that first-graders fail to give redundant messages because they do not realize that redundancy, in certain cases, will help their listeners.
Attribute selection for referring expression generation
25 Citations2008Albert Gatt, Anja Belz
This paper analyses the systems that participated in the first Shared Task Challenge in NLG in terms of their algorithmic properties, comparing new techniques to classic ones, based on results from a new human task-performance experiment and from the intrinsic measures that were used in the Challenge.
Controlling redundancy in referring expressions
16 Citations2008Jette Viethen, Robert Dale +3 more
eScholarship (California Digital Library)The Impact of Colour Difference and Colour Codability on Reference Production
16 Citations2012Jette Viethen, Martijn Goudbeek +1 more
The question of whether people’s preference for using the colour attribute diminishes or remains the same when the colour values in a visual scene are more similar to each other, and when no different basic colour terms exist for them is investigated.
Data Archiving and Networked Services (DANS)Learning Preferences for Referring Expression Generation: Effects of Domain, Language and Algorithm
12 Citations2012Ruud Koolen, Krahmer Emiel +1 more
This paper compares two REG algorithms in terms of their performance: the classic Incremental Algorithm and the more recent Graph algorithm, both of which rely on a notion of preferred attributes that can be learned from human descriptions.
Two Approaches for Generating Size Modifiers
9 Citations2011Margaret Mitchell, Kees van Deemter +1 more
This paper isolates a set of features that may be used in a hand-coded algorithm or a machine learning approach to generate one of six basic size types, and finds a statistically significant difference between the precision and recall of the two systems.
Does Size Matter -- How Much Data is Required to Train a REG Algorithm?
7 Citations2011Mariët Theune, Ruud Koolen +2 more
The results show that depending on the complexity of the domain, training on 10 to 20 items may already lead to a good performance in an algorithm for attribute selection, a subtask of Referring Expressions Generation.
Tilburg University Research PortalEffects of scene variation on referential overspecification
6 Citations2011Ruud Koolen, Martijn Goudbeek +1 more
It is suggested that scene variation causes speakers to make use of quick heuristics when selecting the content of their referring expressions, and the implications of these findings for computational models that automatically generate referring expressions are discussed.
Accessibility and Attention in Situated Dialogue: Roles and Regulations
3 Citations2009Ellen Gurman Bard, Robin L. Hill +2 more