Training radial basis neural networks with the extended Kalman filter
NeurocomputingPublished 1 October 2002
Dan Simon
Citations146
SJR quartileQ1
SJR score1.47
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that the use of the extended Kalman filter results in better learning than conventional RBF networks and faster learning than gradient descent.
Abstract
Radial basis function (RBF) neural networks provide attractive possibilities for solving signal processing and pattern classification problems. Several algorithms have been proposed for choosing the RBF prototypes and training the network. The selection of the RBF prototypes and the network weights can be viewed as a system identification problem. As such, this paper proposes the use of the extended Kalman filter for the learning procedure. After the user chooses how many prototypes to include in the network, the Kalman filter simultaneously solves for the prototype vectors and the weight matrix. A decoupled extended Kalman filter is then proposed in order to decrease the computational effort of the training algorithm. Simulation results are presented on reformulated radial basis neural networks as applied to the Iris classification problem. It is shown that the use of the Kalman filter results in better learning than conventional RBF networks and faster learning than gradient descent.
Keywords
Computer Science
ScienceOptimization by Simulated Annealing
44,600 Citations1983Scott Kirkpatrick, C. D. Gelatt +1 more
A detailed analogy with annealing in solids provides a framework for optimization of the properties of very large and complex systems.
CERN Document Server (European Organization for Nuclear Research)Applied Optimal Estimation
6,388 Citations1974A. Gelb
This is the first book on the optimal estimation that places its major emphasis on practical applications, treating the subject more from an engineering than a mathematical orientation, and the theory and practice of optimal estimation is presented.
Neural ComputationFast Learning in Networks of Locally-Tuned Processing Units
4,219 Citations1989John Moody, Christian J. Darken
This work proposes a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988).
Complex SystemsRadial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks
3,439 Citations1988David S. Broomhead, David Lowe
The relationship between 'learning' in adaptive layered networks and the fitting of data with high dimensional surfaces is discussed, leading naturally to a picture of 'generalization in terms of interpolation between known data points and suggests a rational approach to the theory of such networks.
IEEE Transactions on Neural NetworksOrthogonal least squares learning algorithm for radial basis function networks
3,361 Citations1991Sheng Chen, C.F.N. Cowan +1 more
The authors propose an alternative learning procedure based on the orthogonal least-squares method, which provides a simple and efficient means for fitting radial basis function networks.
CERN Document Server (European Organization for Nuclear Research)Learning Automata: An Introduction
1,559 Citations1989Kumpati S. Narendra, Mandayam A. L. Thathachar
From the combination of knowledge and actions, someone can improve their skill and ability and this learning automata an introduction tells you that any book will give certain knowledge to take all benefits.
IEEE Transactions on Neural NetworksNeurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks
525 Citations1994G.V. Puskorius, L.A. Feldkamp
These simulations suggest that recurrent controller networks trained by Kalman filter methods can combine the traditional features of state-space controllers and observers in a homogeneous architecture for nonlinear dynamical systems, while simultaneously exhibiting less sensitivity than do purely feedforward controller networks to changes in plant parameters and measurement noise.
IEEE Transactions on Neural NetworksReformulated radial basis neural networks trained by gradient descent
281 Citations1999N.B. Karayiannis
Experiments involving a variety of reformulated RBF networks generated by linear and exponential generator functions indicate that gradient descent learning is simple, easily implementable, and produces RBf networks that perform considerably better than conventional RBF models trained by existing algorithms.
IEEE Transactions on Neural NetworksCombined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks
220 Citations1999Sheng Chen, Yulin Wu +1 more
A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm at the upper level.
Neural NetworksOptimal filtering algorithms for fast learning in feedforward neural networks
168 Citations1992Samir Shah, F. Palmieri +1 more
This paper applies optimal filtering techniques to train feedforward networks in the standard supervised learning framework, and presents three algorithms which are computationally more expensive than standard back propagation, but local at the neuron level.
IEEE Transactions on Fuzzy SystemsWill the real iris data please stand up?
122 Citations1999James C. Bezdek, J.M. Keller +3 more
This correspondence points out several published errors in replicates of the well-known iris data, which was collected by Anderson but first published by Fisher (1936).
IEEE Transactions on Neural NetworksOn the Kalman filtering method in neural network training and pruning
82 Citations1999John Sum, Chi-Sing Leung +2 more
In this paper, some cues on the setting of the initial condition will be presented with a simple example illustrated and an elegant equation linking the error sensitivity measure (the saliency) and the result obtained via extended Kalman filter is devised.
IEEE Transactions on Neural NetworksOn-line training of recurrent neural networks with continuous topology adaptation
62 Citations1996Dragan Obradović
If the target system dynamics are changing over time, it is shown that a suitable forgetting factor can be used to "unlearn" the no longer-relevant dynamics and the resulting parameter increase can be handled "smoothly" without interfering with the already acquired information.
IEEE Transactions on Neural NetworksDiscrete-time backpropagation for training synaptic delay-based artificial neural networks
60 Citations1999Richard J. Duro, José Reyes
The aim of this paper is to endow a well-known structure for processing time-dependent information, synaptic delay-based ANN's, with a reliable and easy to implement algorithm suitable for training temporal decision processes, by extending the backpropagation algorithm to discrete-time feedforward networks that include adaptable internal time delays in the synapses.
IEEE Transactions on Neural NetworksA fast U-D factorization-based learning algorithm with applications to nonlinear system modeling and identification
48 Citations1999Youmin Zhang, X.R. Li
A fast learning algorithm for training multilayer feedforward neural networks by using a fading memory extended Kalman filter and a U-D factorization-based FMEKF is proposed to further improve the learning rate and accuracy of the FNN.
A fully Kalman-trained radial basis function network for nonlinear speech modeling
39 Citations2002M. Birgmeier
The network is applied to the task of learning the dynamics of speech signals obtained from sustained vowels, and subsequently used to re-synthesize these vowels autonomously.
Neural NetworksEfficient learning procedures for optimal interpolative nets
33 Citations1993Sam-Kit Sin, R.J.P. deFigueiredo
A recursive learning algorithm called RLS-OI is introduced, which extends the original OI solution to a recursive least squares solution and selects a small but meaningful subset of exemplars known as prototypes and recursively updates the synaptic weights of the neural network as each new prototype is found.
IEEE Transactions on Neural NetworksDistributed fault tolerance in optimal interpolative nets
21 Citations2001D. Simon
Results of trained OI Nets on the Iris classification problem show that fault tolerance can be increased with the algorithm presented, resulting in low weight salience and distributed computation.
NeurocomputingThe design of a real-time neurocomputer based on RBF networks
6 Citations1998V.David Sánchez A, Shon Sloat +3 more
The subsystem is integrated into an avionics system for surveillance and targeting functions and its use for signal peak detection, and the integrated training and testing environment used to train the RBF networks are presented.