Context by region ancestry
Published 1 September 2009
Joseph J. Lim, Pablo Arbeláez, Chunhui Gu, Jitendra Malik
Citations55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper considers the leaves of a hierarchical segmentation tree as elementary units, and constructs region trees by using a high-performance segmentation method, showing that region ancestry efficiently encodes information about discriminative parts, objects and scenes.
Abstract
In this paper, we introduce a new approach for modeling visual context. For this purpose, we consider the leaves of a hierarchical segmentation tree as elementary units. Each leaf is described by features of its ancestral set, the regions on the path linking the leaf to the root. We construct region trees by using a high-performance segmentation method. We then learn the importance of different descriptors (e.g. color, texture, shape) of the ancestors for classification. We report competitive results on the MSRC segmentation dataset and the MIT scene dataset, showing that region ancestry efficiently encodes information about discriminative parts, objects and scenes.
Keywords
Computer ScienceEngineering
LIBLINEAR: A Library for Large Linear Classification
6,625 Citations2008Rong-En Fan, Kai‐Wei Chang +3 more
International Journal of Computer VisionModeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope
6,410 Citations2001Aude Oliva, Antonio Torralba
The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.
A Bayesian Hierarchical Model for Learning Natural Scene Categories
3,613 Citations2005Li Fei-Fei, Pietro Perona
This work proposes a novel approach to learn and recognize natural scene categories by representing the image of a scene by a collection of local regions, denoted as codewords obtained by unsupervised learning.
International Journal of Computer VisionRepresenting and Recognizing the Visual Appearance of Materials using Three-dimensional Textons
1,494 Citations2001Thomas Leung, Jitendra Malik
A unified model to construct a vocabulary of prototype tiny surface patches with associated local geometric and photometric properties, represented as a set of linear Gaussian derivative filter outputs, under different lighting and viewing conditions is provided.
Lecture notes in computer scienceTextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation
1,153 Citations2006Jamie Shotton, John Winn +2 more
A new approach to learning a discriminative model of object classes, incorporating appearance, shape and context information efficiently, is proposed, which is used for automatic visual recognition and semantic segmentation of photographs.
Cognitive PsychologyScene perception: Detecting and judging objects undergoing relational violations
1,072 Citations1982Irving Biederman, Robert J. Mezzanotte +1 more
Results provide converging evidence that semantic relations can be accessed from the results of a single fixation and were available sufficiently early during the time course of scene perception to affect the perception of the objects in the scene.
Semantic texton forests for image categorization and segmentation
1,012 Citations2008Jamie Shotton, Matthew Johnson +1 more
The proposed semantic texton forests are ensembles of decision trees that act directly on image pixels, and therefore do not need the expensive computation of filter-bank responses or local descriptors, and give at least a five-fold increase in execution speed.
Annual Review of PsychologyHIGH-LEVEL SCENE PERCEPTION
969 Citations1999John M. Henderson, Andrew Hollingworth
Three areas of high-level scene perception research are reviewed, focusing on the role of eye movements in scene perception and the influence of ongoing cognitive processing on the position and duration of fixations in a scene.
International Journal of Computer VisionRobust Higher Order Potentials for Enforcing Label Consistency
893 Citations2009Pushmeet Kohli, Ľubor Ladický +1 more
This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner based on higher order conditional random fields and uses potentials defined on sets of pixels generated using unsupervised segmentation algorithms.
Multiscale conditional random fields for image labeling
843 Citations2004Xuming He, Richard S. Zemel +1 more
An approach to include contextual features for labeling images, in which each pixel is assigned to one of a finite set of labels, are incorporated into a probabilistic framework, which combines the outputs of several components.
Geometric context from a single image
680 Citations2005Derek Hoiem, Alexei A. Efros +1 more
This work shows that it can estimate the coarse geometric properties of a scene by learning appearance-based models of geometric classes, even in cluttered natural scenes, and provides a multiple-hypothesis framework for robustly estimating scene structure from a single image and obtaining confidences for each geometric label.
Memory & CognitionThe effects of contextual scenes on the identification of objects
654 Citations1975tephen E. Palmer
This experiment demonstrates the influence of the prior presentation of visual scenes on the identification of briefly presented drawings of real-world objects using Morton’s (1970) “logogen” model.
Objects in Context
618 Citations2007Andrew Rabinovich, Andrea Vedaldi +3 more
This work proposes to incorporate semantic object context as a post-processing step into any off-the-shelf object categorization model using a conditional random field (CRF) framework, which maximizes object label agreement according to contextual relevance.
2009 IEEE Conference on Computer Vision and Pattern RecognitionFrom contours to regions: An empirical evaluation
441 Citations2009Pablo Arbeláez, Michael Maire +2 more
This work provides extensive experimental evaluation to demonstrate that, when coupled to a high-performance contour detector, the OWT-UCM algorithm produces state-of-the-art image segmentations.
CaltechAUTHORS (California Institute of Technology)A Bayesian Hierarchical Model for Learning Natural Scene Categories
412 Citations2005Li, Fei-Fei, Perona, Pietro
International Journal of Computer VisionMulti-Class Segmentation with Relative Location Prior
412 Citations2008Stephen Jay Gould, Jim Rodgers +3 more
This work proposes a method for capturing global information from inter-class spatial relationships and encoding it as a local feature and shows that the incorporation of relative location information allows it to significantly outperform the current state-of-the-art.
Using contours to detect and localize junctions in natural images
405 Citations2008Michael Maire, Pablo Arbeláez +2 more
A new high-performance contour detector using a combination of local and global cues that provides the best performance to date on the Berkeley Segmentation Dataset (BSDS) benchmark and shows that improvements in the contour model lead to better junctions.
Lecture notes in computer scienceLearning Spatial Context: Using Stuff to Find Things
379 Citations2008Geremy Heitz, Daphne Koller
This paper clusters image regions based on their ability to serve as context for the detection of objects and shows that the things and stuff (TAS) context model produces meaningful clusters that are readily interpretable, and helps improve detection ability over state-of-the-art detectors.
Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes
320 Citations2003Kevin Murphy, Antonio Torralba +1 more
This work presents a conditional random field for jointly solving the tasks of object detection and scene classification, and proposes to use the scene context as an extra source of (global) information, to help resolve local ambiguities.
Auto-context and its application to high-level vision tasks
247 Citations2008Zhuowen Tu
An auto-context algorithm that learns an integrated low-level and context model, and is very general and easy to implement, and has the potential to be used for a wide variety of problems of multi-variate labeling.
A hierarchical field framework for unified context-based classification
225 Citations2005Sanjiv Kumar, Martial Hebert
A two-layer hierarchical formulation to exploit different levels of contextual information in images for robust classification and is general enough to be applied to different domains ranging from pixelwise image labeling to contextual object detection.
Robust higher order potentials for enforcing label consistency
202 Citations2008Pushmeet Kohli, Ľubor Ladický +1 more
The MIT Press eBooksImage Retrieval and Classification Using Local Distance Functions
195 Citations2007Andrea Frome, Yoram Singer +1 more
This paper introduces and experiment with a framework for learning local perceptual distance functions for visual recognition as a combination of elementary distances between patch-based visual features, and applies this framework to the tasks of image retrieval and classification of novel images.
Region Classification with Markov Field Aspect Models
189 Citations2007Jakob Verbeek, Bill Triggs
Combining spatial and aspect models significantly improves the region-level classification accuracy, and models trained with image-level labels outperform PLSA trained with pixel-level ones.
IEEE Transactions on Pattern Analysis and Machine IntelligenceContext-based vision: recognizing objects using information from both 2D and 3D imagery
165 Citations1991Thomas M. Strat, Martin A. Fischler
In this work, a system is explicitly designed to identify and use context as an integral part of recognition that eliminates the traditional dependence on stored geometric models and universal image partitioning algorithms.
Lecture notes in computer scienceObject Recognition by Integrating Multiple Image Segmentations
129 Citations2008Caroline Pantofaru, Cordelia Schmid +1 more
By integrating the image partition hypotheses in an intuitive combined top-down and bottom-up recognition approach, this work improves object and feature support and explores possible extensions of the method and whether they provide improved performance.
Depth from Familiar Objects: A Hierarchical Model for 3D Scenes
68 Citations2006Erik B. Sudderth, Antonio Torralba +2 more
An integrated, probabilistic model for the appearance and three-dimensional geometry of cluttered scenes and a robust likelihood model accounts for outliers in matched stereo features, allowing effective learning of 3D object structure from partial 2D segmentations.
From appearance to context-based recognition: Dense labeling in small images
65 Citations2008Devi Parikh, C. Lawrence Zitnick +1 more
The role of context for dense scene labeling in small images with low resolution images with impoverished appearance information is explored and the algorithm achieves state-of-the-art performance on MSRC and Corel datasets.
Learning subcategory relevances for category recognition
58 Citations2008Siniša Todorović, Narendra Ahuja
This work proposes an approach to learning that demonstrates the advantages of exploiting subcategory taxonomy for recognition, and suggests that a feature space spanned by part properties, instead of direct object properties, allows for linear separation of image classes.
Learning class-specific affinities for image labelling
41 Citations2008Dhruv Batra, Rahul Sukthankar +1 more
This paper presents an algorithm to learn class-specific similarity functions that achieves state-of-the-art performance on standard datasets and provides a principled probabilistic interpretation for learning all of the parameters that define these affinities.
eScholarship (California Digital Library)Recursive Segmentation and Recognition Templates for 2D Parsing
22 Citations2011Leo Zhu, Yuanhao Chen +3 more
Discriminative modeling by Boosting on Multilevel Aggregates
13 Citations2008Jason J. Corso