Automatic Image Annotation Using Auxiliary Text Information
Edinburgh Research ExplorerPublished 1 June 2008Open access
Yansong Feng, Mirella Lapata
Citations69
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper creates a database of pictures that are naturally embedded into news articles and proposes to use their captions as a proxy for annotation keywords, showing that an image annotation model can be developed on this dataset alone without the overhead of manual annotation.
Abstract
The availability of databases of images labeled with keywords is necessary for developing and evaluating image annotation models. Dataset collection is however a costly and time consuming task. In this paper we exploit the vast resource of images available on the web. We create a database of pictures that are naturally embedded into news articles and propose to use their captions as a proxy for annotation
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
IEEE Transactions on Pattern Analysis and Machine IntelligenceNormalized cuts and image segmentation
15,696 Citations2000Jianbo Shi, Jitendra Malik
IEEE Transactions on Pattern Analysis and Machine IntelligenceContent-based image retrieval at the end of the early years
6,023 Citations2000A.W.M. Smeulders, Marcel Worring +3 more
The working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap are discussed, as well as aspects of system engineering: databases, system architecture, and evaluation.
ACM SIGIR ForumA Language Modeling Approach to Information Retrieval
2,532 Citations2017Jay Ponte, W. Bruce Croft
It will be shown that probabilistic methods can be used to predict topic changes in the context of the task of new event detection and provide further proof of concept for the use of language models for retrieval tasks.
Lecture notes in computer scienceObject Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
1,608 Citations2002Pınar Duygulu, Kobus Barnard +2 more
This work shows how to cluster words that individually are difficult to predict into clusters that can be predicted well, and cannot predict the distinction between train and locomotive using the current set of features, but can predict the underlying concept.
LDA-based document models for ad-hoc retrieval
1,086 Citations2006Xing Wei, W. Bruce Croft
This paper proposes an LDA-based document model within the language modeling framework, and evaluates it on several TREC collections, and shows that improvements over retrieval using cluster-based models can be obtained with reasonable efficiency.
Modeling annotated data
1,070 Citations2003David M. Blei, Michael I. Jordan
Three hierarchical probabilistic mixture models which aim to describe annotated data with multiple types, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type.
Multiple Bernoulli relevance models for image and video annotation
813 Citations2004Siwei Feng, R. Manmatha +1 more
This work shows how it can do both automatic image annotation and retrieval (using one word queries) from images and videos using a multiple Bernoulli relevance model, which significantly outperforms previously reported results on the task of image and video annotation.
IEEE Transactions on Image ProcessingImage classification for content-based indexing
758 Citations2001Aditya Vailaya, Mário A. T. Figueiredo +2 more
The goal is to combine multiple two-class classifiers into a single hierarchical classifier, and it is demonstrated that a small vector quantizer can be used to model the class-conditional densities of the features, required by the Bayesian methodology.
ScholarWorks@UMassAmherst (University of Massachusetts Amherst)A Model for Learning the Semantics of Pictures
681 Citations2003Victor Lavrenko, R. Manmatha +1 more
An approach to learning the semantics of images which allows us to automatically annotate an image with keywords and to retrieve images based on text queries using a formalism that models the generation of annotated images.
Content-based image retrieval
432 Citations2005Ritendra Datta, Jia Li +1 more
Some of the key contributions in the current decade related to image retrieval and automated image annotation are discussed, spanning 120 references, and a study on the trends in volume and impact of publications in the field with respect to venues/journals and sub-topics is concluded.
Lecture notes in computer scienceUsing Maximum Entropy for Automatic Image Annotation
137 Citations2004Jiwoon Jeon, R. Manmatha
The experimental results show that Maximum Entropy outperforms one of the classical translation models that has been applied to this task and the Cross Media Relevance Model.
IEEE Transactions on Circuits and Systems for Video TechnologyA Study of Quality Issues for Image Auto-Annotation With the Corel Dataset
47 Citations2007Jiayu Tang, Paul Lewis
This paper compares the performance of several annotation algorithms using the Corel set and finds that simple near neighbor propagation techniques perform fairly well and a support vector machine (SVM)-based annotation method achieves even better results, almost as good as the best found in the literature.