Short term memory in echo state networks

Abstract

The report investigates the short-term memory capacity of echo state recurrent neural networks. A quantitative measure MC of short-term memory capacity is introduced. The main result is that MC 5 N for networks with linear Output units and i.i.d. input, where N is network size. Conditions under which these maximal memory capacities are realized are described. Several theoretical and practical examples demonstrate how the short-term memory capacities of echo state networks can be exploited for dynamic Pattern recognition and stochastic sequence modeling tasks.