Online reservoir adaptation by intrinsic plasticity for backpropagation–decorrelation and echo state learning

It is shown experimentally that a biologically motivated learning rule based on neural intrinsic plasticity can drive the neurons' output activities to approximate exponential distributions and implement sparse codes in the reservoir.

Abstract

We propose to use a biologically motivated learning rule based on neural intrinsic plasticity to optimize reservoirs of analog neurons. This rule is based on an information maximization principle, it is local in time and space and thus computationally efficient. We show experimentally that it can drive the neurons' output activities to approximate exponential distributions. Thereby it implements sparse codes in the reservoir. Because of its incremental nature, the intrinsic plasticity learning is well suited for joint application with the online backpropagation-decorrelation or the least mean squares reservoir learning, whose performance can be strongly improved. We further show that classical echo state regression can also benefit from reservoirs, which are pre-trained on the given input signal with the implicit plasticity rule.