A learning rule for extracting spatio-temporal invariances

It is demonstrated that a model neuron which adapts to make its output vary smoothly over time can learn to extract invariances implicit in its input, using a linear combination of Hebbian and anti-Hebbian synaptic changes.

Abstract

The inputs to photoreceptors tend to change rapidly over time, whereas physical parameters (e.g. surface depth) underlying these changes vary more slowly. Accordingly, if a neuron codes for a physical parameter then its output should also change slowly, despite its rapidly fluctuating inputs. We demonstrate that a model neuron which adapts to make its output vary smoothly over time can learn to extract invariances implicit in its input. This learning consists of a linear combination of Hebbian and anti-Hebbian synaptic changes, operating simultaneously upon the same connection weights but at different time scales. This is shown to be sufficient for the unsupervised learning of simple spatio-temporal invariances.