Generating Image Descriptions Using Dependency Relational Patterns
Published 11 July 2010
Ahmet Aker, Robert Gaizauskas
Citations85
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results show that summaries biased by dependency pattern models lead to significantly higher ROUGE scores than both n-gram language models reported in previous work and also Wikipedia baseline summaries.
Abstract
This paper presents a novel approach to automatic captioning of geo-tagged images by summarizing multiple web-documents that contain information re-lated to an image’s location. The summa-rizer is biased by dependency pattern mod-els towards sentences which contain fea-tures typically provided for different scene types such as those of churches, bridges, etc. Our results show that summaries bi-ased by dependency pattern models lead to significantly higher ROUGE scores than both n-gram language models reported in previous work and also Wikipedia base-line summaries. Summaries generated us-ing dependency patterns also lead to more readable summaries than those generated without dependency patterns. 1
Keywords
Computer Science
ROUGE: A Package for Automatic Evaluation of Summaries
8,286 Citations2004Chin-Yew Lin
Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.
Lecture notes in computer scienceObject Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
1,608 Citations2002Pınar Duygulu, Kobus Barnard +2 more
This work shows how to cluster words that individually are difficult to predict into clusters that can be predicted well, and cannot predict the distinction between train and locomotive using the current set of features, but can predict the underlying concept.
A shortest path dependency kernel for relation extraction
971 Citations2005Răzvan Bunescu, Raymond J. Mooney
Experiments on extracting top-level relations from the ACE (Automated Content Extraction) newspaper corpus show that the new shortest path dependency kernel outperforms a recent approach based on dependency tree kernels.
Dependency tree kernels for relation extraction
825 Citations2004Aron Culotta, Jeffrey Sorensen
This work extends previous work on tree kernels to estimate the similarity between the dependency trees of sentences, and uses this kernel within a Support Vector Machine to detect and classify relations between entities in the Automatic Content Extraction (ACE) corpus of news articles.
A general language model for information retrieval
545 Citations1999Fei Song, W. Bruce Croft
A new language model for information retrieval is presented, which is based on a range of data smoothing techniques, including the Good-Turning estimate, curve-fitting functions, and model combinations, and can be easily extended to incorporate probabilities of phrases such as word pairs and word triples.
Learning the semantics of words and pictures
526 Citations2002Kobus Barnard, David Forsyth
The model is very promising for information retrieval tasks such as database browsing and searching for images based on text and/or image features, and can be used for novel applications such as associating words with pictures, and unsupervised learning for object recognition.
IEEE MultimediaName-It: naming and detecting faces in news videos
252 Citations1999Shin’ichi Satoh, Yutaka Nakamura +1 more
Name-It, a system that associates faces and names in news videos, takes a multimodal video analysis approach: face sequence extraction and similarity evaluation from videos, name extraction from transcripts, and video-caption recognition.
Automatic acquisition of domain knowledge for Information Extraction
228 Citations2000Roman Yangarber, Ralph Grishman +2 more
This paper presents an alternative approach, based on an automatic discovery procedure, EXDISCO, which identifies a set of relevant documents and aSet of event patterns from un-annotaled text, starting from a small set of "seed patterns."
Journal of DocumentationA taxonomy of relationships between images and text
188 Citations2003Emily Marsh, Marilyn Domas White
The paper uses qualitative content analysis to illustrate use of the taxonomy to analyze four image‐text pairs in government publications and discusses the implications of the research for information retrieval and document design.
A semantic approach to IE pattern induction
163 Citations2005Mark Stevenson, Mark Greenwood
This paper presents a novel algorithm for the acquisition of Information Extraction patterns that makes the assumption that useful patterns will have similar meanings to those already identified as relevant.
Automatic image captioning
114 Citations2005Jia-Yu Pan, Hyung-Jeong Yang +2 more
This work examines the problem of automatic image captioning and proposes methods to discover correlations between image features and keywords, so that it can automatically find good keywords for a new image.
Text Analysis for Automatic Image Annotation
74 Citations2007Koen Deschacht, Marie‐Francine Moens
A novel approach to automatically annotate images using associated text that detects and classify all entities in the text and combines these measures to compute the probability that an entity is present in the image.
Edinburgh Research ExplorerAutomatic Image Annotation Using Auxiliary Text Information
69 Citations2008Yansong Feng, Mirella Lapata
This paper creates a database of pictures that are naturally embedded into news articles and proposes to use their captions as a proxy for annotation keywords, showing that an image annotation model can be developed on this dataset alone without the overhead of manual annotation.
Automatic pattern acquisition for Japanese information extraction
49 Citations2001Kiyoshi Sudo, Satoshi Sekine +1 more
The procedure to acquire Tree-Based Patterns in Japanese from un-annotated text is outlined and the common paths in the parse tree of relevant sentences are taken as extracted patterns.
Summarization System Integrated with Named Entity Tagging and IE Pattern Discovery.
45 Citations2002Chikashi Nobata, Satoshi Sekine +2 more
This work has introduced information extraction technique such as named entity tagging and pattern discovery to a summarization system based on sentence extraction technique and achieved one of the best performance in subjective evaluation of summarization results.
Research on Language and ComputationDependency Pattern Models for Information Extraction
26 Citations2009Mark Stevenson, Mark Greenwood
This paper compares a variety of pattern models, including ones which have been previously reported and variations of them, and finds that the best performance was observed from two models which use the majority of relevant portions of the dependency tree without including irrelevant sections.
RIAO ConferenceAutomatic word assignment to images based on image division and vector quantization
24 Citations2000Yasuhide Mori, Hironobu Takahashi +1 more
This method is based on statistical learning from image databases with words and produces results which show that each sub-image can be correlated to a set of words, each of which is selected from words assigned to original images.
White Rose Research Online (University of Leeds, The University of Sheffield, University of York)Describing the where – improving image annotation and search through geography
23 Citations2008Ross S. Purves, Alistair Edwardes +1 more
This paper describes how geography might help to index the where facet of the Pansofsky-Shatford matrix and argues that such methods will become central as increasing numbers of images become georeferenced.
Model Summaries for Location-related Images
16 Citations2010Ahmet Aker, Robert Gaizauskas
A corpus of human generated model captions in English and German is described and it is found that a high correlation in ROUGE scores between post-edited and non-post-edited model summaries which indicates that the expensive process of post-editing is not necessary.
Summary Generation for Toponym-referenced Images using Object Type Language Models
15 Citations2009Ahmet Aker, Robert Gaizauskas
A novel approach to automatic captioning of toponym-referenced images by summarizing multiple web-documents that contain information related to an image’s location that can generate both query-based and language model-biased multidocument summaries.
Automatic image captioning from the web for GPS photographs
11 Citations2010Xin Fan, Ahmet Aker +4 more
This paper proposes a solution to automatically generate captions (including place name, keywords and summary) from the web content based on image location information via synergetic techniques from Geographic Information System, Web IR and multi-document summarisation.