Efficient evolution of asymmetric recurrent neural networks using a PDGP-inspired two-dimensional representation
Lecture notes in computer sciencePublished 1 January 1998
João Carlos Figueira Pujol, Riccardo Poli
Citations24
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new method to build recurrent neural networks based on evolutionary computation is presented, which combines a linear chromosome with a two-dimensional representation inspired by Parallel Distributed Genetic Programming to evolve the architecture and the weights simultaneously.
Abstract
Recurrent neural networks are particularly useful for processing time sequences and simulating dynamical systems. However, methods for building recurrent architectures have been hindered by the fact that available training algorithms are considerably more complex than those for feedforward networks. In this paper, we present a new method to build recurrent neural networks based on evolutionary computation, which combines a linear chromosome with a two-dimensional representation inspired by Parallel Distributed Genetic Programming (a form of genetic programming for the evolution of graph-like programs) to evolve the architecture and the weights simultaneously. Our method can evolve general asymmetric recurrent architectures as well as specialized recurrent architectures. This paper describes the method and reports on results of its application.
Keywords
Computer Science
Neural Networks: A Comprehensive Foundation
29,816 Citations1998Simon Haykin
Thorough, well-organized, and completely up to date, this book examines all the important aspects of this emerging technology, including the learning process, back-propagation learning, radial-basis function networks, self-organizing systems, modular networks, temporal processing and neurodynamics, and VLSI implementation of neural networks.
Genetic Algorithms in Search, Optimization and Machine Learning
17,773 Citations1988David E. Goldberg, David Robson
Medical Entomology and ZoologyGenetic Programming: On the Programming of Computers by Means of Natural Selection
13,257 Citations1992John R. Koza
This book discusses the evolution of architecture, primitive functions, terminals, sufficiency, and closure, and the role of representation and the lens effect in genetic programming.
Cognitive ScienceFinding Structure in Time
10,841 Citations1990Jeffrey L. Elman
A proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory and suggests a method for representing lexical categories and the type/token distinction is developed.
IEEE Transactions on Neural NetworksRecurrent neural networks and robust time series prediction
1,301 Citations1994Jerome T. Connor, R. Douglas Martin +1 more
A robust learning algorithm is proposed and applied to recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component and are shown to give better predictions than neural networks trained on unfiltered time series.
IEEE Signal Processing MagazineProgress in supervised neural networks
1,246 Citations1993D. Hush, B.G. Horne
Theoretical results concerning the capabilities and limitations of various neural network models are summarized, and some of their extensions are discussed.
IEEE Transactions on Neural NetworksAn evolutionary algorithm that constructs recurrent neural networks
1,022 Citations1994Peter J. Angeline, Gregory M. Saunders +1 more
It is argued that genetic algorithms are inappropriate for network acquisition and an evolutionary program is described, called GNARL, that simultaneously acquires both the structure and weights for recurrent networks.
Physical Review LettersGeneralization of back-propagation to recurrent neural networks
951 Citations1987Fernando J. Pineda
An adaptive neural network with asymmetric connections is introduced that bears a resemblance to the master/slave network of Lapedes and Farber but it is architecturally simpler.
IEEE Transactions on Neural NetworksLearning long-term dependencies in NARX recurrent neural networks
773 Citations1996Tsung-Nan Lin, B.G. Horne +2 more
It is shown that the long-term dependencies problem is lessened for a class of architectures called nonlinear autoregressive models with exogenous (NARX) recurrent neural networks, which have powerful representational capabilities.
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)Computational capabilities of recurrent NARX neural networks
456 Citations1997Hava T. Siegelmann, B.G. Horne +1 more
It is constructively proved that the NARX networks with a finite number of parameters are computationally as strong as fully connected recurrent networks and thus Turing machines, raising the issue of what amount of feedback or recurrence is necessary for any network to be Turing equivalent and what restrictions on feedback limit computational power.
IEEE Transactions on Neural NetworksGenetic evolution of the topology and weight distribution of neural networks
438 Citations1994Vittorio Maniezzo
This paper proposes a system based on a parallel genetic algorithm with enhanced encoding and operational abilities that has been applied to two widely different problem areas: Boolean function learning and robot control.
Machine LearningGenetic reinforcement learning for neurocontrol problems
201 Citations1994Darrell Whitley, Stephen Dominic +2 more
On a simulated inverted-pendulum control problem, “genetic reinforcement learning” produces competitive results with AHC, another well-known reinforcement learning paradigm for neural networks that employs the temporal difference method.
IEEE Transactions on Neural NetworksEvolving recurrent perceptrons for time-series modeling
130 Citations1994John R. McDonnell, Donald Waagen
It is speculated that the stochastic training method implemented in this study for training recurrent perceptrons can be used to train perceptron networks that have radically recurrent architectures.
IEEE Transactions on Neural NetworksPruning recurrent neural networks for improved generalization performance
129 Citations1994C. Lee Giles, Christian W. Omlin
Why Ants are Hard
129 Citations1998William B. Langdon, Riccardo Poli
Analysis of the program search space in terms of fixed length schema suggests it is highly deceptive and that for the simplest solutions large building blocks must be assembled before they have above average fitness.
Neural NetworksGeneral asymmetric neural networks and structure design by genetic algorithms
122 Citations1992Stefan Bornholdt, Dirk Graudenz
A learning algorithm for neural networks based on genetic algorithms is proposed and a simplified model for a brain with sensory and motor neurons is studied, whose structure is solely determined by an evolutionary process.
Machine LearningGenetic Reinforcement Learning for Neurocontrol Problems
115 Citations1993Darrell Whitley, Stephen Dominic +2 more
On a simulated inverted-pendulum control problem, “genetic reinforcement learning” produces competitive results with AHC, another well-known reinforcement learning paradigm for neural networks that employs the temporal difference method.
Artificial Neural Nets and Genetic AlgorithmsDiscovery of Symbolic, Neuro-Symbolic and Neural Networks with Parallel Distributed Genetic Programming
44 Citations1998Riccardo Poli
The representation for programs and the genetic operators on which PDGP is based are described and Experimental results on the XOR problem axe are reported.
Genetic Programming of Minimal Neural Nets Using Occam's Razor
38 Citations1993Byoung‐Tak Zhang, Heinz Mühlenbein
A genetic programming method is investigated for optimizing both the architecture and the connection weights of multilayer feedforward neural networks and a newness function is proposed that makes an optimal trade-o between the error tting ability and the parsimony of the network.
Evolving Artificial Neural Networks through Evolutionary Programming.
36 Citations1996Xin Yao, Yong Liu
A tool for micro-soldering the connecting tags of an integrated circuit chip to corresponding terminals on a substrate by Joule heating includes a high conductivity bit having a planar bottom face adapted to contact the tags and press them against the terminals while applying sufficient heat to the tags to solder them to the terminals.
Evolving recurrent neural networks with non-binary encoding
24 Citations2002Martin Mandischer
It is shown that evolutionary algorithms can be used for the construction of networks for real-world tasks and a data structure based genotypic network representation, as well as genetic operators, are introduced.
IEEE International Conference on Neural NetworksA comparison of recurrent neural network learning algorithms
22 Citations2002Alessandro Logar, Edward Corwin +1 more
Selected recurrent network training algorithms are described, and their performances are compared with respect to speed and accuracy for a given problem to allow more accurate comparison between training algorithms for networks with few nodes.
Some Steps Towards a Form of Parallel Distributed Genetic Programming
15 Citations1996Roberto Poli
PDGP (Parallel Distributed Genetic Programming), a new form of genetic pro- gramming which is suitable for the development of fine-grained parallel programs, is described, based on a graph-like representation for parallel programs and genetic operators which guarantee the syntactic correctness of the offspring.
Lecture notes in computer scienceGenetic synthesis of discrete-time recurrent neural network
11 Citations1993F. J. Marín, F. Sandoval
Empirical studies show that the proposed different genetic model for optimizing both network architecture and connection weights of Discrete-Time Recurrent Neural Networks in evolutionary process can efficiently generate the appropiate network size and topology for small applications.
Evolutionary generation and training of recurrent artificial neural networks
9 Citations2002José Sántos, Richard J. Duro
An evolutionary artificial neural network training and design methodology is presented, aimed at obtaining optimum or quasi-optimum synchronous recurrent neural networks capable of processing sequential inputs and developed GENIAL, a genetic algorithm development environment specifically designed for solving this type of problem.
Artificial Neural Nets and Genetic AlgorithmsEvolving Recurrent Neural Networks
4 Citations1993Kristian Lindgren, Anders Nilsson +2 more
An evolutionary algorithm which allows entities to increase and decrease in complexity during the evolutionary process is applied to recurrent neural networks, which provides a suitable set of test problems.
Evolving Neural Controllers Using a Dual Network Representation
1 Citations2007João Carlos Figueira, Riccardo Poli
A linear chromosome combined with a grid-based representation of the network and a new crossover operator allow the evolution of the architecture and the weights simultaneously and there is no need for a separate weight optimization procedure.