Combining textual and visual cues for content-based image retrieval on the World Wide Web
Published 27 November 2002
Marco La Cascia, Sunjay Sethi, Stan Sclaroff
Citations252
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A system is proposed that combines textual and visual statistics in a single index vector for content- based search of a WWW image database and allows improved performance in conducting content-based search.
Abstract
A system is proposed that combines textual and visual statistics in a single index vector for content-based search of a WWW image database. Textual statistics are captured in vector form using latent semantic indexing (LSI) based on text in the containing HTML document. Visual statistics are captured in vector form using color and orientation histograms. By using an integrated approach, it becomes possible to take advantage of possible statistical couplings between the content of the document (latent semantic content) and the contents of images (visual statistics). The combined approach allows improved performance in conducting content-based search. Search performance experiments are reported for a database containing 100,000 images collected from the WWW.
Keywords
Computer Science
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
International Journal of Computer VisionColor indexing
5,592 Citations1991Michael J. Swain, Dana H. Ballard
It is demonstrated that color histograms of multicolored objects provide a robust, efficient cue for indexing into a large database of models and that they can differentiate among a large number of objects.
ComputerQuery by image and video content: the QBIC system
3,058 Citations1995Myron Flickner, Harpreet Sawhney +10 more
The QBIC system is described and its query capabilities are demonstrated, which allows queries on large image and video databases based on example images, user-constructed sketches and drawings, selected color and texture patterns, camera and object motion, and other graphical information.
IEEE Transactions on Pattern Analysis and Machine IntelligenceThe design and use of steerable filters
2,934 Citations1991William T. Freeman, Edward H. Adelson
The authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters, allowing one to adaptively steer a filter to any orientation, and to determine analytically the filter output as a function of orientation.
IEEE Transactions on Systems Man and CyberneticsTextural Features Corresponding to Visual Perception
2,130 Citations1978Hideyuki Tamura, Shunji Mori +1 more
The discrepancies between human vision and computerized techniques that are encountered in this study indicate fundamental problems in digital analysis of textures and could be overcome by analyzing their causes and using more sophisticated techniques.
International Journal of Computer VisionVisual learning and recognition of 3-d objects from appearance
1,891 Citations1995Hiroshi Murase, Shree K. Nayar
A near real-time recognition system with 20 complex objects in the database has been developed and a compact representation of object appearance is proposed that is parametrized by pose and illumination.
View-based and modular eigenspaces for face recognition
1,813 Citations1994Pentland, Moghaddam +1 more
A modular eigenspace description technique is used which incorporates salient features such as the eyes, nose and mouth, in an eigenfeature layer, which yields higher recognition rates as well as a more robust framework for face recognition.
International Journal of Computer VisionPhotobook: Content-based manipulation of image databases
1,564 Citations1996Alex Pentland, Rosalind W. Picard +1 more
The Photobook system is described, which is a set of interactive tools for browsing and searching images and image sequences that make direct use of the image content rather than relying on text annotations to provide a sophisticated browsing and search capability.
Elsevier eBooksQuery by Image and Video Content: The QBIC System
1,562 Citations2002Myron Flickner, Harpreet Sawhney +10 more
SIAM ReviewUsing Linear Algebra for Intelligent Information Retrieval
1,484 Citations1995Michael W. Berry, Susan Dumais +1 more
A lexical match between words in users’ requests and those in or assigned to documents in a database helps retrieve textual materials from scientific databases.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Photobook: tools for content-based manipulation of image databases</title>
883 Citations1994Alexander P. Pentland, Rosalind W. Picard +1 more
The Photobook system is described, which is a set of interactive tools for browsing and searching images and image sequences that differ from those used in standard image databases in that they make direct use of the image content rather than relying on annotations.
IEEE Transactions on Pattern Analysis and Machine IntelligenceEfficient color histogram indexing for quadratic form distance functions
834 Citations1995John W. Hafner, Harpreet Sawhney +3 more
An improved shipping container having novel locking features in the end panel and corner flaps and improved bulge resistance at the end panels from sideward bulge of the product.
Comparing images using color coherence vectors
831 Citations1996Greg Pass, Ramin Zabih +1 more
It is shown that CCV’s can give superior results to color histogram-based methods for comparing images that incorporates spatial information, and to whom correspondence should be addressed tograms for image retrieval.
ComputerChabot: retrieval from a relational database of images
652 Citations1995Vanessa Ogle, Michael Stonebraker
This work presents an approach that integrates a relational database retrieval system with a color analysis technique, and shows how a coarse granularity is used for content analysis improves the ability to retrieve images efficiently.
Behavior Research Methods, Instruments, & ComputersImproving the retrieval of information from external sources
576 Citations1991Susan Dumais
A statistical method is described called latent semantic indexing, which models the implicit higher order structure in the association of words and objects and improves retrieval performance by up to 30%.
IEEE Transactions on ComputersApplication of the Karhunen-Loève Expansion to Feature Selection and Ordering
537 Citations1970Keinosuke Fukunaga, Warren Koontz
A method is developed herein to use the Karhunen-Loeve expansion to extract features relevant to classification of a sample taken from one of two pattern classes.
IEEE Transactions on Pattern Analysis and Machine IntelligencePeriodicity, directionality, and randomness: Wold features for image modeling and retrieval
522 Citations1996F. Liu, Rosalind W. Picard
The Wold model appears to offer a perceptually more satisfying measure of pattern similarity while exceeding the performance of these other methods by traditional pattern recognition criteria.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Tools and techniques for color image retrieval</title>
468 Citations1996John R. Smith, Shih‐Fu Chang
This work proposes a technique by which the color content of images and videos is automatically extracted to form a class of meta-data that is easily indexed and evaluates the retrieval effectiveness of the color set back-projection method and compares its performance to other color image retrieval methods.
Pattern RecognitionSHAPE-BASED RETRIEVAL: A CASE STUDY WITH TRADEMARK IMAGE DATABASES
381 Citations1998Anil K. Jain, Aditya Vailaya
Considering the substantial amount of time and effort needed for a manual retrieval from a large image database, an automatic shape-based retrieval technique can significantly simplify the retrieval task.
IEEE MultimediaVisually searching the Web for content
350 Citations1997J. R. Smith, Shih-Fu Chang
WebSeek provides a complete system that collects visual information from the Web by automated agents, then catalogs and indexes it for fast searching and retrieval.
WebSeer: An Image Search Engine for the World Wide Web
303 Citations1996Charles Frankel, Michael J. Swain +1 more
WebSeer uses image content in addition to associated text to index images, presenting the user with a selection that potentially fits her needs on the Web.
ImageRover: a content-based image browser for the World Wide Web
301 Citations1997Stan Sclaroff, Leonid Taycher +1 more
ImageRover is a search-by-image-content navigation tool for the World Wide Web that utilizes a distributed fleet of WWW robots to gather images expediently and employs a novel relevance feedback algorithm that selects the distance metrics that are appropriate for a particular query.
ComputerSimilar-shape retrieval in shape data management
299 Citations1995R. Mehrotra, James E. Gary
This work discusses the central issues in similar-shape retrieval and explains how these issues are resolved in a shape retrieval scheme called FIBSSR (Feature Index-Based Similar-Shape Retrieval).
Interactive Learning Using a “Society of Models”
266 Citations1997Tom Minka, Rosalind W. Picard
Multimedia SystemsVision texture for annotation
253 Citations1995Rosalind W. Picard, Thomas P. Minka
This paper demonstrates a new application of computer vision to digital libraries — the use of texture forotation, the description of content, and is equipped with the ability to choose the one that “best explains” the regions selected by the user for annotating.
PicHunter: Bayesian relevance feedback for image retrieval
215 Citations1996Ingemar J. Cox, Matthew L. Miller +2 more
PicHunter is an image retrieval system that implements a novel approach to relevance feedback, such that the entire history of user selections contributes to the system's estimate of the user's goal image.
Supporting similarity queries in MARS
210 Citations1997Michaël Ortega, Rui Yong +3 more
The ranking and retrieval algorithms developed in MARS based on the Boolean retrievaI model are discussed and the results of the experiments that demonstrate the effectiveness of the developed model for image retrieval are described.
Pattern RecognitionInteractive learning with a “society of models”
210 Citations1997Thomas P. Minka, Rosalind W. Picard
ComputerAutomatic indexing and content-based retrieval of captioned images
196 Citations1995Rohini K. Srihari
The research group has developed an automatic indexing system for captioned pictures of people; the indexing information and other textual information is subsequently used in a content-based image retrieval system.
IEEE Transactions on Signal ProcessingA unified texture model based on a 2-D Wold-like decomposition
192 Citations1993Joseph M. Francos, Amihai Meiri +1 more
A unified texture model that is applicable to a wide variety of texture types found in natural images is presented and results show that the deterministic components should be parameterized separately from the purely indeterministic component.
Texture orientation for sorting photos "at a glance"
189 Citations2002M. Gorkani, Rosalind W. Picard
The authors demonstrate a simple test of global multiscale orientation for quickly searching a database of vacation photos for likely "city/suburb" shots and find the orientation features achieve agreement with human classification in 91 out of 98 of the scenes.
Lecture notes in computer scienceFinding pictures of objects in large collections of images
164 Citations1996David Forsyth, Jitendra Malik +6 more
The approach to object recognition is described, which is structured around a sequence of increasingly specialized grouping activities that assemble coherent regions of image that can be shown to satisfy increasingly stringent constraints.
A relevance feedback architecture for content-based multimedia information retrieval systems
122 Citations1997Yong Rui, Thomas S. Huang +2 more
This paper introduces a human-computer interaction based approach to MIR in which the user guides the system during retrieval using relevance feedback and shows that the retrieval performance improves significantly by incorporating humans in the retrieval process.
Efficient query refinement for image retrieval
95 Citations2002Chahab Nastar, Matthias Mitschke +1 more
This paper uses multiple image features, and presents in details a new relevance feedback technique that integrates the positive and negative examples provided by the user that is more performant than the standard relevance feedback approach.
JACOB: just a content-based query system for video databases
88 Citations2002Marco La Cascia, Edoardo Ardizzone
The authors describe JACOB, a prototypal system allowing content-based browsing and querying in video databases that automatically splits a video into a sequence of shots, extracts a few representative frames from each shot and computes r-frame descriptors based on features like color and texture.
Multimedia Tools and ApplicationsAutomatic Video Database Indexing and Retrieval
82 Citations1997Edoardo Ardizzone, Marco La Cascia
A system allowing content-based annotation and querying in video databases, which automatically splits a video into a sequence of shots, extracts a few representative frames from each shot and computes r-frame descriptors based on color, texture and motion.
A comparative study of several color models for color image invariant retrieval
49 Citations1996Th. Gevers, A.W.M. Smeulders
Learning feature relevance and similarity metrics in image databases
40 Citations2002Bir Bhanu, Jing Peng +1 more
The authors present a novel image retrieval system that continuously learns the weights of features and selects an appropriate similarity metric based on the user's feedback given as positive or negative image examples.
World Wide Web Image Search Engines
22 Citations1995Stan Sclaroff
A world wide web image search engine that crawls the web collecting information about the images it finds, computes the appropriate image decompositions and indices, and stores this extracted information for searches based on image content is proposed.
Distinguishing the picture captions on the World Wide Web (poster)
3 Citations1997Neil C. Rowe, Brian Frew
This work in MARIE-3 showed that seven quickly-found parameters of images are sufficient to distinguish photographs with 70% recall and 70% precision on a test set of random Web pages.