Learning Invariance from Transformation Sequences

Neural ComputationPublished 1 June 1991

Péter Földiák

DOI10.1162/neco.1991.3.2.194

Source URL

Citations683

SJR quartileQ1

SJR score0.83

SNIP1.45

TL;DR

An application of the algorithm is presented in which the network learns invariance to shift in retinal position, which may be involved in the development of the characteristic shift invariance property of complex cells in the primary visual cortex and also in theDevelopment of more complicated invariance properties of neurons in higher visual areas.

Abstract

The visual system can reliably identify objects even when the retinal image is transformed considerably by commonly occurring changes in the environment. A local learning rule is proposed, which allows a network to learn to generalize across such transformations. During the learning phase, the network is exposed to temporal sequences of patterns undergoing the transformation. An application of the algorithm is presented in which the network learns invariance to shift in retinal position. Such a principle may be involved in the development of the characteristic shift invariance property of complex cells in the primary visual cortex, and also in the development of more complicated invariance properties of neurons in higher visual areas.

Keywords

Computer ScienceNeuroscience

Neural ComputationBackpropagation Applied to Handwritten Zip Code Recognition

11,878 Citations1989Yann LeCun, Bernhard E. Boser +5 more

This paper demonstrates how constraints from the task domain can be integrated into a backpropagation network through the architecture of the network, successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service.

Psychological ReviewToward a modern theory of adaptive networks: Expectation and prediction.

1,457 Citations1981Richard S. Sutton, Andrew G. Barto

The adaptive element presented learns to increase its response rate in anticipation of increased stimulation, producing a conditioned response before the occurrence of the unconditioned stimulus, and is in strong agreement with the behavioral data regarding the effects of stimulus context.

Experimental Brain ResearchVisual neurones responsive to faces in the monkey temporal cortex

1,394 Citations1982David I. Perrett, Edmund T. Rolls +1 more

Findings indicate that explanations in terms of arousal, emotional or motor reactions, simple visual feature sensitivity or receptive fields are insufficient to account for the selective responses to faces and face features observed in this population of STS neurones.

Cognitive ScienceFeature Discovery by Competitive Learning*

1,073 Citations1985David E. Rumelhart, David Zipser

This paper shows how a set of feature detectors which capture important aspects of the set of stimulus input patterns are discovered and how these feature detectors form the basis of a multilayer system that serves to learn categorizations of stimulus sets which are not linearly separable.

View PDF

Cognitive ScienceFeature discovery by competitive learning

890 Citations1985David E. Rumelhart, David Zipser

Biological CyberneticsForming sparse representations by local anti-Hebbian learning

553 Citations1990Péter Földiák

It is shown here that a layer of simple Hebbian units connected by modifiable anti-Hebbian feed-back connections can learn to code a set of patterns in such a way that statistical dependency between the elements of the representation is reduced, while information is preserved.

Lecture notes in computer scienceLearning translation invariant recognition in a massively parallel networks

173 Citations1987Geoffrey E. Hinton

This paper describes a recently developed procedure that can learn to perform a recognition task and uses canonical internal representations of the patterns to identify familiar patterns in novel positions.

Journal of NeurophysiologyA complex-cell receptive-field model

144 Citations1985Hedva Spitzer, Shaul Hochstein

This model may be used as a working hypothesis for the analysis of these aspects of the various cortical receptive-field types as well as for the tests for the responses it predicts for the application of counterphase-grating stimulation.

Journal of the Operational Research SocietyThe Hedonistic Neuron: A Theory of Memory, Learning and Intelligence

138 Citations1983John S. Edwards

Neural ComputationRemoving Time Variation with the Anti-Hebbian Differential Synapse

62 Citations1991Graeme Mitchison

A local synaptic learning rule is described that can be used to remove the effects of certain types of systematic temporal variation in the inputs to a unit and will generate center-surround receptive fields that remove temporally varying linear gradients from the inputs.