Evolving Memory Cell Structures for Sequence Learning
Lecture notes in computer sciencePublished 1 January 2009Open access
Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber
Citations109
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper optimize LSTM's computational structure using a multi-objective evolutionary algorithm, which reflects the structure's usefulness for learning various formal languages.
Abstract
Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
IEEE Transactions on Evolutionary ComputationA fast and elitist multiobjective genetic algorithm: NSGA-II
47,759 Citations2002Kalyanmoy Deb, Amrit Pratap +2 more
This paper suggests a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties, and modify the definition of dominance in order to solve constrained multi-objective problems efficiently.
Neural ComputationLearning to Forget: Continual Prediction with LSTM
5,419 Citations2000Felix A. Gers, Jürgen Schmidhuber +1 more
This work identifies a weakness of LSTM networks processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's internal state could be reset, and proposes a novel, adaptive forget gate that enables an LSTm cell to learn to reset itself at appropriate times, thus releasing internal resources.
Proceedings of the IEEEBackpropagation through time: what it does and how to do it
4,904 Citations1990Paul J. Werbos
This paper first reviews basic backpropagation, a simple method which is now being widely used in areas like pattern recognition and fault diagnosis, and describes further extensions of this method, to deal with systems other than neural networks, systems involving simultaneous equations or true recurrent networks, and other practical issues which arise with this method.
Evolutionary ComputationEvolving Neural Networks through Augmenting Topologies
3,245 Citations2002Kenneth O. Stanley, Risto Miikkulainen
A method is presented, NeuroEvolution of Augmenting Topologies (NEAT), which outperforms the best fixed-topology method on a challenging benchmark reinforcement learning task and shows how it is possible for evolution to both optimize and complexify solutions simultaneously.
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
1,261 Citations2009
This chapter contains sections titled: Introduction Exponential Error Decay Dilemma: Avoiding Aradient Decay Prevents Long-Term Latching Remedies and Conclusion.
IEEE Transactions on Neural NetworksLSTM recurrent networks learn simple context-free and context-sensitive languages
726 Citations2001Felix A. Gers, E. Schmidhuber
Long short-term memory (LSTM) variants are also the first RNNs to learn a simple context-sensitive language, namely a(n)b( n)c(n).
Adaptive BehaviorIncremental Evolution of Complex General Behavior
425 Citations1997Faustino Gomez, Risto Miikkulainen
This article proposes an approach wherein complex general behavior is learned incrementally, by starting with simpler behavior and gradually making the task more challenging and general, which evolves more effective and more general behavior.
Cambridge University Press eBooksA Biologically Based Computational Model of Working Memory
325 Citations1999Randall C. O’Reilly, Todd S. Braver +1 more
This model defines working memory as controlled processing involving active maintenance and/or rapid learning, where controlled processing is an emergent property of the dynamic interactions of multiple brain systems, but the prefrontal cortex and hippocampus are especially influential owing to their specialized processing abilities.
Neural ComputationTraining Recurrent Networks by Evolino
254 Citations2007Jürgen Schmidhuber, Daan Wierstra +2 more
Bern Open Repository and Information System (University of Bern)A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks
182 Citations2007Marcus Liwicki, Alex Graves +2 more
A new connectionist approach to on-line handwriting recognition and address in particular the problem of recognizing handwritten whiteboard notes using a recently introduced objective function, known as Connectionist Temporal Classification (CTC), that directly trains the network to label unsegmented sequence data.
Lecture notes in computer scienceSolving Deep Memory POMDPs with Recurrent Policy Gradients
146 Citations2007Daan Wierstra, A. Foerster +2 more
Recurrent Policy Gradients, a modelfree reinforcement learning (RL) method creating limited-memory stochastic policies for partially observable Markov decision problems (POMDPs) that require long-term memories of past observations is presented.
Lecture notes in computer scienceIncremental One-Class Learning with Bounded Computational Complexity
111 Citations2007Rowland R. Sillito, Robert B. Fisher
An incremental one-class learning algorithm is proposed for the purpose of outlier detection that is shown to outperform a current state-of-the-art incremental one-class learning algorithm (Incremental SVDD [5]) on a variety of datasets, while requiring only an upper limit on model complexity to be specified.
Adaptive BehaviorEmpirical Studies in Action Selection with Reinforcement Learning
48 Citations2007Shimon Whiteson, Matthew E. Taylor +1 more
A novel way of making evolutionary and TD methods work together is explored by using evolution to automatically discover good representations for TD function approximators, and results are presented demonstrating that this novel approach can outperform both TD and evolutionary methods alone.
Recurrent Neural Networks Can Learn to Implement Symbol-Sensitive Counting
28 Citations1997Paul Rodriguez, Janet Wiles
This work shows that a RNN can learn a harder CFL, a simple palindrome, by organizing its resources into a symbol-sensitive counting solution, and provides a dynamical systems analysis which demonstrates how the network can not only count, but also copy and store counting information.
Reinforcement learning in partially observable mobile robot domains using unsupervised event extraction
19 Citations2003Boudewijn Bakker, Fredrik Linåker +1 more
This paper describes how learning tasks in partially observable mobile robot domains can be solved by combining reinforcement learning with an unsupervised learning "event extraction" mechanism, called ARAVQ, which transforms the robot's continuous, noisy, high-dimensional sensory input stream into a compact sequence of high-level events.