3D ShapeNets for 2.5D Object Recognition and Next-Best-View Prediction
arXiv (Cornell University)Published 22 June 2014Open access
Zhirong Wu, Shuran Song, Aditya Khosla, Xiaoou Tang, Jianxiong Xiao
Citations93
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work proposes to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network, and shows that this 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.
Abstract
3D shape is a crucial but heavily underutilized cue in object recognition, mostly due to the lack of a good generic shape representation. With the recent boost of inexpensive 2.5D depth sensors (e.g. Microsoft Kinect), it is even more urgent to have a useful 3D shape model in an object recognition pipeline. Furthermore, when the recognition has low confidence, it is important to have a fail-safe mode for object recognition systems to intelligently choose the best view to obtain extra observation from another viewpoint, in order to reduce the uncertainty as much as possible. To this end, we propose to represent a geometric 3D shape as a probabil-ity distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network. Our model naturally supports object recognition from 2.5D depth map and also view planning for object recognition. We construct a large-scale 3D computer graphics dataset to train our model, and conduct extensive experiments to study this new representation. 1
Keywords
Computer ScienceEarth and Planetary SciencesEngineering
Neural ComputationA Fast Learning Algorithm for Deep Belief Nets
16,404 Citations2006Geoffrey E. Hinton, Simon Osindero +1 more
A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
IEEE Transactions on Pattern Analysis and Machine IntelligenceObject Detection with Discriminatively Trained Part-Based Models
9,976 Citations2009Pedro F. Felzenszwalb, Ross Girshick +2 more
An object detection system based on mixtures of multiscale deformable part models that is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges is described.
Lecture notes in computer scienceIndoor Segmentation and Support Inference from RGBD Images
5,616 Citations2012Nathan Silberman, Derek Hoiem +2 more
The goal is to parse typical, often messy, indoor scenes into floor, walls, supporting surfaces, and object regions, and to recover support relationships, to better understand how 3D cues can best inform a structured 3D interpretation.
Psychological ReviewRecognition-by-components: A theory of human image understanding.
5,507 Citations1987Irving Biederman
Recognition-by-components (RBC) provides a principled account of the heretofore undecided relation between the classic principles of perceptual organization and pattern recognition.
Neural ComputationTraining Products of Experts by Minimizing Contrastive Divergence
5,017 Citations2002Geoffrey E. Hinton
A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.
SUN database: Large-scale scene recognition from abbey to zoo
3,202 Citations2010Jianxiong Xiao, James Hays +3 more
This paper proposes the extensive Scene UNderstanding (SUN) database that contains 899 categories and 130,519 images and uses 397 well-sampled categories to evaluate numerous state-of-the-art algorithms for scene recognition and establish new bounds of performance.
Lecture notes in computer scienceA Practical Guide to Training Restricted Boltzmann Machines
2,802 Citations2012Geoffrey E. Hinton
This guide is an attempt to share expertise at training restricted Boltzmann machines with other machine learning researchers.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Method for registration of 3-D shapes</title>
2,314 Citations1992Paul J. Besl, Neil David McKay
The described method is also useful for deciding fundamental issues such as the congruence (shape equivalence) of different geometric representations as well as for estimating the motion between point sets where the correspondences are not known.
Computer Graphics ForumOn Visual Similarity Based 3D Model Retrieval
1,425 Citations2003Ding‐Yun Chen, Xiao‐Pei Tian +2 more
A visual similarity‐based 3D model retrieval system that is robust against similarity transformation, noise, model degeneracy, and provides 42%, 94% and 25% better performance than three other competing approaches.
The princeton shape benchmark
1,422 Citations2004Philip Shilane, P. Min +2 more
It is concluded that no single descriptor is best for all classifications, and thus the main contribution of this paper is to provide a framework to determine the conditions under which each descriptor performs best.
Rotation invariant spherical harmonic representation of 3D shape descriptors
1,232 Citations2003Michael Kazhdan, Thomas Funkhouser +1 more
Sparse deep belief net model for visual area V2
881 Citations2007Honglak Lee, Chaitanya Ekanadham +1 more
An unsupervised learning model is presented that faithfully mimics certain properties of visual area V2 and the encoding of these more complex "corner" features matches well with the results from the Ito & Komatsu's study of biological V2 responses, suggesting that this sparse variant of deep belief networks holds promise for modeling more higher-order features.
International Journal of Computer Vision3D Object Modeling and Recognition Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints
428 Citations2006Fred Rothganger, Svetlana Lazebnik +2 more
A novel representation for three-dimensional objects in terms of local affine-invariant descriptors of their images and the spatial relationships between the corresponding surface patches is introduced, allowing the acquisition of true 3D affine and Euclidean models from multiple unregistered images, as well as their recognition in photographs taken from arbitrary viewpoints.
Communications of the ACMUnsupervised learning of hierarchical representations with convolutional deep belief networks
387 Citations2011Honglak Lee, Roger Grosse +2 more
The convolutional deep belief network is presented, a hierarchical generative model that scales to realistic image sizes and is translation-invariant and supports efficient bottom-up and top-down probabilistic inference.
Using fast weights to improve persistent contrastive divergence
281 Citations2009Tijmen Tieleman, Geoffrey E. Hinton
It is shown that the weight updates force the Markov chain to mix fast, and using this insight, an even faster mixing chain is developed that uses an auxiliary set of "fast weights" to implement a temporary overlay on the energy landscape.
IEEE Transactions on Pattern Analysis and Machine IntelligenceInformation theoretic sensor data selection for active object recognition and state estimation
255 Citations2002Joachim Denzler, Christopher M. Brown
A formalism for optimal sensor parameter selection for iterative state estimation in static systems using Shannon's information theory to select information-gathering actions that maximize mutual information, thus optimizing the information that the data conveys about the true state of the system.
International Journal of Computer VisionThe Shape Boltzmann Machine: A Strong Model of Object Shape
128 Citations2013S. M. Ali Eslami, Nicolas Heess +2 more
The SBM characterizes a strong model of shape, in that samples from the model look realistic and it can generalize to generate samples that differ from training examples, and it finds that the SBM learns distributions that are qualitatively and quantitatively better than existing models for this task.
Predicting Failures of Vision Systems
127 Citations2014Peng Zhang, Jiuling Wang +3 more
This work shows that a surprisingly straightforward and general approach, that is ALERT, can predict the likely accuracy (or failure) of a variety of computer vision systems - semantic segmentation, vanishing point and camera parameter estimation, and image memorability prediction - on individual input images.
A textured object recognition pipeline for color and depth image data
117 Citations2012Jie Tang, Stephen D. Miller +2 more
An object recognition system which leverages the additional sensing and calibration information available in a robotics setting together with large amounts of training data to build high fidelity object models for a dataset of textured household objects is presented.
Hypothesis testing framework for active object detection
29 Citations2013Nikolay Atanasov, Bharath Sankaran +4 more
This paper proposes an active approach to object detection by controlling the point of view of a mobile depth camera by solving an active M-ary hypothesis testing problem using a point-based approximate POMDP algorithm.
Active view selection for object and pose recognition
17 Citations2009Zhaoyin Jia, Yao‐Jen Chang +1 more
An algorithm for multi-view object and pose recognition that makes use of a boosting method to find the best sequence across the multi-views and exploits the relation between the different view points using the Adaboost algorithm.