Learning Transformational Invariants from Natural Movies
Published 8 December 2008
Charles F. Cadieu, Bruno A. Olshausen
Citations62
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A hierarchical, probabilistic model that learns to extract complex motion from movies of the natural environment that encoded transformational invariants, which are selective for the speed and direction of a moving pattern, but are invariant to its spatial structure.
Abstract
We describe a hierarchical, probabilistic model that learns to extract complex mo-tion from movies of the natural environment. The model consists of two hidden layers: the first layer produces a sparse representation of the image that is ex-pressed in terms of local amplitude and phase variables. The second layer learns the higher-order structure among the time-varying phase variables. After train-ing on natural movies, the top layer units discover the structure of phase-shifts within the first layer. We show that the top layer units encode transformational invariants: they are selective for the speed and direction of a moving pattern, but are invariant to its spatial structure (orientation/spatial-frequency). The diver-sity of units in both the intermediate and top layers of the model provides a set of testable predictions for representations that might be found in V1 and MT. In addition, the model demonstrates how feedback from higher levels can influence representations at lower levels as a by-product of inference in a graphical model. 1
Keywords
Computer ScienceMathematicsNeuroscience
Vision ResearchSparse coding with an overcomplete basis set: A strategy employed by V1?
3,720 Citations1997Bruno A. Olshausen, David J. Field
These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli.
Journal of the Optical Society of America ASpatiotemporal energy models for the perception of motion
3,515 Citations1985Edward H. Adelson, James R. Bergen
A class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency that permit a qualitative understanding of a variety of motion phenomena.
Vision ResearchThe “independent components” of natural scenes are edge filters
2,269 Citations1997Anthony J. Bell, T. J. Sejnowski
It is shown that a new unsupervised learning algorithm based on information maximization, a nonlinear "infomax" network, when applied to an ensemble of natural scenes produces sets of visual filters that are localized and oriented.
IEEE Transactions on Pattern Analysis and Machine IntelligenceRobust Object Recognition with Cortex-Like Mechanisms
1,682 Citations2007T. Serre, Lior Wolf +3 more
A hierarchical system that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation by alternating between a template matching and a maximum pooling operation is described.
Journal of the Optical Society of America AHierarchical Bayesian inference in the visual cortex
1,517 Citations2003Tai Sing Lee, David B. Mumford
This work proposes a new theoretical setting based on the mathematical framework of hierarchical Bayesian inference for reasoning about the visual system, and suggests that the algorithms of particle filtering and Bayesian-belief propagation might model these interactive cortical computations.
Neural ComputationSlow Feature Analysis: Unsupervised Learning of Invariances
1,377 Citations2002Laurenz Wiskott, Terrence J. Sejnowski
Slow feature analysis (SFA) is a new method for learning invariant or slowly varying features from a vectorial input signal that is guaranteed to find the optimal solution within a family of functions directly and can learn to extract a large number of decor-related features, which are ordered by their degree of invariance.
Learning methods for generic object recognition with invariance to pose and lighting
1,332 Citations2004Yann LeCun, Fu Jie Huang +1 more
A real-time version of the system was implemented that can detect and classify objects in natural scenes at around 10 frames per second and proved impractical, while convolutional nets yielded 16/7% error.
International Journal of Computer VisionComputation of component image velocity from local phase information
1,074 Citations1990David J. Fleet, Allan D. Jepson
The resulting technique is predominantly linear, efficient, and suitable for parallel processing, and is local in space-time, robust with respect to noise, and permits multiple estimates within a single neighborhood.
Vision ResearchA model of neuronal responses in visual area MT
953 Citations1998Eero P. Simoncelli, David J. Heeger
A computational model of MT physiology, in which local image velocities are represented via the distribution of MT neuronal responses, which is performed in two stages, corresponding to neurons in cortical areas V1 and MT.
Neural ComputationLearning Invariance from Transformation Sequences
683 Citations1991Péter Földiák
An application of the algorithm is presented in which the network learns invariance to shift in retinal position, which may be involved in the development of the characteristic shift invariance property of complex cells in the primary visual cortex and also in theDevelopment of more complicated invariance properties of neurons in higher visual areas.
Neural ComputationEmergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces
546 Citations2000Aapo Hyvärinen, Patrik O. Hoyer
It is shown that the same principle of independence maximization can explain the emergence of phase- and shift-invariant features, similar to those found in complex cells, by maximizing the independence between norms of projections on linear subspaces.
Progress in NeurobiologyINVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM
543 Citations1997Guy Wallis, Edmund T. Rolls
Evidence is described, showing that some neurons in the macaque temporal cortical visual areas have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning.
Journal of NeuroscienceA selection model for motion processing in area MT of primates
171 Citations1995SJ Nowlan, TJ Sejnowski
A computational model for motion processing in area MT is presented that is based on the observed response properties of cortical neurons and is consistent with the visual perception of partially occluded and transparent moving stimuli.
Neural ComputationA Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals
160 Citations2005Yan Karklin, Michael S. Lewicki
A hierarchical Bayesian model is presented that is able to capture higher-order nonlinear structure and represent nonstationary data distributions and Adapting the model to image or audio data yields a nonlinear, distributed code for higher- order statistical regularities that reflect more abstract, invariant properties of the signal.
European Journal of NeuroscienceLearning the invariance properties of complex cells from their responses to natural stimuli
96 Citations2002Wolfgang Einhäuser, Christoph Kayser +2 more
A network model capable of learning from natural stimuli and developing receptive field properties which match those of cortical simple and complex cells is suggested and lends a physiologically based explanation of the development of complex cell invariance response properties.
Neural ComputationBilinear Sparse Coding for Invariant Vision
94 Citations2004David B. Grimes, Rajesh P. N. Rao
This work describes an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model and shows that from an arbitrary set of natural images, the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformation.
Journal of the Optical Society of America ABubbles: a unifying framework for low-level statistical properties of natural image sequences
92 Citations2003Aapo Hyvärinen, Jarmo Hurri +1 more
This work proposes a unifying framework for statistical properties of biological visual systems, based on the concept of spatiotemporal activity "bubbles", which can be estimated by maximization of the sparsenesses of linear filter outputs.
Neural ComputationEmergence of Position-Independent Detectors of Sense of Rotation and Dilation with Hebbian Learning: An Analysis
62 Citations1993Kechen Zhang, Martin I. Sereno +1 more
This work shows analytically that a Hebb rule trained with arbitrary rotation, dilation/contraction, and translation velocity fields yields units with weight fields that are a rotation plus a dilation or contraction field, and whose responses to a rotating or dilating/contracting disk are exactly position independent.
Journal of the Optical Society of America AThe atoms of vision: Cartesian or polar?
51 Citations1999Christoph Zetzsche, Gerhard Krieger +1 more
It is shown that a more efficient representation can be obtained by a nonlinear encoding that yields a feature space with polar organization, and some striking similarities are pointed out between the polar representation in visual cortex and basic image-coding strategies pursued in shape-gain vector quantization schemes.
Neural ComputationInvariant Global Motion Recognition in the Dorsal Visual System: A Unifying Theory
33 Citations2006Edmund T. Rolls, Simon M. Stringer
This work proposes that the dorsal visual system uses a hierarchical feedforward network architecture with training of the connections with a short-term memory trace associative synaptic modification rule to capture what is invariant at each stage, and shows that the proposal is computationally feasible.