RECURRENT NEURAL NETWORKS ARE UNIVERSAL APPROXIMATORS
International Journal of Neural SystemsPublished 1 August 2007
Anton Maximilian Schäfer, Hans-Georg Zimmermann
Citations202
SJR quartileQ1
SJR score1.58
SNIP1.67
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
Recurrent Neural Networks (RNN) have been developed for a better understanding and analysis of open dynamical systems. Still the question often arises if RNN are able to map every open dynamical system, which would be desirable for a broad spectrum of applications. In this article we give a proof for the universal approximation ability of RNN in state space model form and even extend it to Error Correction and Normalized Recurrent Neural Networks.
Keywords
Computer SciencePhysics and Astronomy
Neural NetworksMultilayer feedforward networks are universal approximators
21,248 Citations1989Kurt Hornik, Maxwell B. Stinchcombe +1 more
It is rigorously established that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available.
The MIT Press eBooksParallel Distributed Processing
15,227 Citations1986David E. Rumelhart, James L. McClelland +1 more
Mathematics of Control Signals and SystemsApproximation by superpositions of a sigmoidal function
13,684 Citations1989George Cybenko
Neural NetworksOn the approximate realization of continuous mappings by neural networks
4,220 Citations1989Ken-ichi Funahashi
It is proved that any continuous mapping can be approximately realized by Rumelhart-Hinton-Williams' multilayer neural networks with at least one hidden layer whose output functions are sigmoid functions.
Neural ComputationReal-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
4,037 Citations2002Wolfgang Maass, Thomas Natschläger +1 more
A new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks, based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry.
IEEE Transactions on Neural NetworksUniversal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems
1,169 Citations1995Tianping Chen, Hong Chen
The main results are: every Tauber-Wiener function is qualified as an activation function in the hidden layer of a three-layered neural network and the possibility by neural computation to approximate the output as a whole of a dynamical system, thus identifying the system.
A Field Guide to Dynamical Recurrent Networks
339 Citations2009John F. Kolen, Stefan C. Kremer
This book presents the range of dynamical recurrent network (DRN) architectures that will be used in the book and transforms the text from a collection of papers into a coherent book.
Information and ComputationThe Dynamic Universality of Sigmoidal Neural Networks
119 Citations1996Joe Kilian, Hava T. Siegelmann
The techniques can be applied to a much more general class of “sigmoidal-like” activation functions, suggesting that Turing universality is a relatively common property of recurrent neural network models.
Lecture notes in computer scienceHow to Train Neural Networks
50 Citations1998Ralph Neuneier, Hans Georg Zimmermann
The Observer - Observation Dilemma is solved by forcing the network to construct smooth approximation functions, and some pruning algorithms to optimize the network architecture are proposed.
Studies in computational financeModelling and Forecasting Financial Data
46 Citations2002Abdol S. Soofi, Liangyue Cao
Neural ComputationUniversal Approximation of Multiple Nonlinear Operators by Neural Networks
35 Citations2002Andrew D. Back, Tianping Chen
A theory is proposed that provides a possible mechanism by which some classes of neural networks with fixed weights to model multiple nonlinear dynamical systems can occur.
Studies in computational financeModeling Dynamical Systems by Error Correction Neural Networks
30 Citations2002Hans-Georg Zimmermann, Ralph Neuneier +1 more
A new time delay recurrent neural network called ECNN is introduced, which includes the last model error as an additional input, which can interpret the models misspecification as an external shock which can be used to guide the model dynamics afterwards.
The MIT Press eBooksModeling Large Dynamical Systems with Dynamical Consistent Neural Networks
22 Citations2006Hans-Georg Zimmermann, Ralph Grothmann +2 more
This chapter contains sections titled Handling Uncertainty, Function and Structure in Recurrent Neural Networks, and Conclusion, which summarizes the findings of the previous sections.