Detecting objects in large image collections and videos by efficient subimage retrieval
Published 1 September 2009
Christoph H. Lampert
Citations64
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An extensive evaluation on several datasets shows that ESR is not only very fast, but it also achieves excellent detection accuracies thereby improving over previous systems for object-based image retrieval.
Abstract
We study the task of detecting the occurrence of objects in large image collections or in videos, a problem that combines aspects of content based image retrieval and object localization. While most previous approaches are either limited to special kinds of queries, or do not scale to large image sets, we propose a new method, efficient subimage retrieval (ESR), that is at the same time very flexible and very efficient. Relying on a two-layered branch-and-bound setup, ESR performs object-based image retrieval in sets of 100,000 or more images within seconds. An extensive evaluation on several datasets shows that ESR is not only very fast, but it also achieves excellent detection accuracies thereby improving over previous systems for object-based image retrieval.
Keywords
Computer Science
IEEE Transactions on Pattern Analysis and Machine IntelligenceContent-based image retrieval at the end of the early years
6,023 Citations2000A.W.M. Smeulders, Marcel Worring +3 more
The working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap are discussed, as well as aspects of system engineering: databases, system architecture, and evaluation.
International Journal of Computer VisionColor indexing
5,592 Citations1991Michael J. Swain, Dana H. Ballard
It is demonstrated that color histograms of multicolored objects provide a robust, efficient cue for indexing into a large database of models and that they can differentiate among a large number of objects.
Object retrieval with large vocabularies and fast spatial matching
2,959 Citations2007James Philbin, Ondřej Chum +3 more
To improve query performance, this work adds an efficient spatial verification stage to re-rank the results returned from the bag-of-words model and shows that this consistently improves search quality, though by less of a margin when the visual vocabulary is large.
ACM Transactions on Multimedia Computing Communications and ApplicationsContent-based multimedia information retrieval
1,524 Citations2006Michael S. Lew, Nicu Sebe +2 more
This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques.
IEEE Transactions on Pattern Analysis and Machine IntelligenceBlobworld: image segmentation using expectation-maximization and its application to image querying
1,407 Citations2002Chad Carson, Serge Belongie +2 more
Results indicating that querying for images using Blobworld produces higher precision than does querying using color and texture histograms of the entire image in cases where the image contains distinctive objects are presented.
Beyond sliding windows: Object localization by efficient subwindow search
686 Citations2008Christoph H. Lampert, Matthew B. Blaschko +1 more
A simple yet powerful branch-and-bound scheme that allows efficient maximization of a large class of classifier functions over all possible subimages and converges to a globally optimal solution typically in sublinear time is proposed.
Information RetrievalFeatures for image retrieval: an experimental comparison
587 Citations2007Thomas Deselaers, Daniel Keysers +1 more
An experimental comparison of a large number of different image descriptors for content-based image retrieval is presented and the often used, but very simple, color histogram performs well in the comparison and thus can be recommended as a simple baseline for many applications.
IEEE Transactions on Pattern Analysis and Machine IntelligenceObject matching using deformable templates
518 Citations1996Anil K. Jain, Yu Zhong +1 more
A Bayesian scheme, which is based on prior knowledge and the edge information in the input image, is employed to find a match between the deformed template and objects in the image and computational efficiency is achieved via a coarse-to-fine implementation of the matching algorithm.
IEEE Transactions on Pattern Analysis and Machine IntelligenceEfficient Visual Search of Videos Cast as Text Retrieval
496 Citations2009Josef Šivic, Andrew Zisserman
An approach to object retrieval which searches for and localizes all the occurrences of an object in a video, given a query image of the object, and investigates retrieval performance with respect to different quantizations of region descriptors and compares the performance of several ranking measures.
Efficient near-duplicate detection and sub-image retrieval
344 Citations2004Yongyang Ke
Evaluation of color descriptors for object and scene recognition
231 Citations2008Koen E. A. van de Sande, Theo Gevers +1 more
Pattern Recognition LettersContent-based query of image databases: inspirations from text retrieval
190 Citations2000David Squire, Wolfgang Müller +2 more
IEEE Transactions on Pattern Analysis and Machine IntelligenceLocalized Content-Based Image Retrieval
169 Citations2008R. Rahmani, Sally A. Goldman +3 more
A localized CBIR system that uses labeled images in conjunction with a multiple-instance learning algorithm to first identify the desired object and weight the features accordingly, and then to rank images in the database using a similarity measure that is based upon only the relevant portions of the image.
Lecture notes in computer scienceFinding pictures of objects in large collections of images
164 Citations1996David Forsyth, Jitendra Malik +6 more
The approach to object recognition is described, which is structured around a sequence of increasingly specialized grouping activities that assemble coherent regions of image that can be shown to satisfy increasingly stringent constraints.
2009 IEEE Conference on Computer Vision and Pattern RecognitionPose search: Retrieving people using their pose
154 Citations2009Vittorio Ferrari, Manuel J. Marín‐Jiménez +1 more
This work proposes and investigates several pose descriptors which are person, clothing, background and lighting independent, and improves the performance over existing methods for localizing upper body layout on unconstrained video.
Pattern RecognitionFocused color intersection with efficient searching for object extraction
69 Citations1997Vivin Vinod, Hiroshi Murase
Experiments conducted demonstrate that multiple known objects in complex scenes can be extracted by this process and the method is stable against scale changes, two-dimensional rotation, moderate changes in shape and partial occlusion.
Object-based queries using color points of interest
61 Citations2002Valérie Gouet, Nozha Boujemaa
A local image descriptor based on color points of interest is proposed, i.e. points extracted and characterized from a color signal at one go, which is involved in an image indexing and retrieval approach which is dedicated to object or sub-image retrieval.
Face image retrieval using HMMs
38 Citations2003Aleix M. Martı́nez
A new face recognition system that can be used to index (and thus retrieve) images and videos of a database of faces, based on identifying frontal faces, that allows reasonable variability in facial expressions, illumination conditions, and occlusions caused by eye-wear or items of clothing such as scarves.
IEEE Transactions on Circuits and Systems for Video TechnologyA fast multiresolution feature matching algorithm for exhaustive search in large image databases
29 Citations2001Byung Cheol Song, Myung Jun Kim +1 more
A fast search algorithm based on a multiresolution data structure that computes the lower bound of distance at each level and compares it with the latest minimum distance, starting from the low-resolution level to dramatically reduce the total computational complexity.
Object fingerprints for content analysis with applications to street landmark localization
13 Citations2008Wen Wu, Jie Yang
A street landmark localization system that combines salient region detection, segmentation, and object fingerprint extraction techniques for the purpose and shows superior performance to state-of-the-art recognition algorithms.
Lecture notes in computer scienceImage Retrieval Using a Hierarchy of Clusters
7 Citations2001Daniela Stan, Ishwar K. Sethi
The proposed procedure has the advantage of handling high dimensional data, and dealing with non-Euclidean similarity metrics in order to explore the nature of the image feature vectors.