Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks
Published 22 March 2004
V.G. Gudise, Ganesh K. Venayagamoorthy
Citations426
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
Particle swarm optimization (PSO) motivated by the social behavior of organisms, is a step up to existing evolutionary algorithms for optimization of continuous nonlinear functions. Backpropagation (BP) is generally used for neural network training. Choosing a proper algorithm for training a neural network is very important. In this paper, a comparative study is made on the computational requirements of the PSO and BP as training algorithms for neural networks. Results are presented for a feedforward neural network learning a nonlinear function and these results show that the feedforward neural network weights converge faster with the PSO than with the BP algorithm.
Keywords
Computer Science
Particle swarm optimization
47,523 Citations2002James Kennedy, R.C. Eberhart
NatureLearning representations by back-propagating errors
30,885 Citations1986David E. Rumelhart, Geoffrey E. Hinton +1 more
Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
A modified particle swarm optimizer
10,157 Citations2002Yibing Shi, R.C. Eberhart
A new parameter, called inertia weight, is introduced into the original particle swarm optimizer, which resembles a school of flying birds since it adjusts its flying according to its own flying experience and its companions' flying experience.
IEEE Transactions on Neural NetworksTraining feedforward networks with the Marquardt algorithm
7,625 Citations1994Martin Hagan, Mohammad Bagher Menhaj
The Marquardt algorithm for nonlinear least squares is presented and is incorporated into the backpropagation algorithm for training feedforward neural networks and is found to be much more efficient than either of the other techniques when the network contains no more than a few hundred weights.
Proceedings of the IEEEBackpropagation through time: what it does and how to do it
4,904 Citations1990Paul J. Werbos
This paper first reviews basic backpropagation, a simple method which is now being widely used in areas like pattern recognition and fault diagnosis, and describes further extensions of this method, to deal with systems other than neural networks, systems involving simultaneous equations or true recurrent networks, and other practical issues which arise with this method.
Particle swarm optimization: developments, applications and resources
4,298 Citations2001Eberhart, Yuhui Shi
Developments in the particle swarm algorithm since its origin in 1995 are reviewed and brief discussions of constriction factors, inertia weights, and tracking dynamic systems are included.
Empirical study of particle swarm optimization
3,917 Citations2003Yibing Shi, R.C. Eberhart
The experimental results show that the PSO is a promising optimization method and a new approach is suggested to improve PSO's performance near the optima, such as using an adaptive inertia weight.
Lecture notes in computer scienceParameter selection in particle swarm optimization
3,505 Citations1998Yuhui Shi, Russell C. Eberhart
This paper first analyzes the impact that inertia weight and maximum velocity have on the performance of the particle swarm optimizer, and then provides guidelines for selecting these two parameters.
Elsevier eBooksLearning Internal Representations by Error Propagation
3,013 Citations1988David E. Rumelhart, Geoffrey E. Hinton +1 more
This chapter contains sections titled: The Problem, The Generalized Delta Rule, Simulation Results, Some Further Generalizations, Conclusion.
Neural ComputationFirst- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method
1,193 Citations1992Roberto Battiti
First- and second-order optimization methods for learning in feedforward neural networks are reviewed to illustrate the main characteristics of the different methods and their mutual relations.
Medical Entomology and ZoologyPrinciples of Neurocomputing for Science and Engineering
584 Citations2000Fredric M. Ham, Ivica Kostanic
Principles of Neurocomputing for Science and Engineering, unlike other neural networks texts, is written specifically for scientists and engineers who want to apply neural networks to solve complex problems.
Cooperative learning in neural networks using particle swarm optimizers
323 Citations2000F. van den Bergh, Andries P. Engelbrecht
This paper presents a method to employ particle swarms optimizers in a cooperative configuration by splitting the input vector into several sub-vectors, each which is optimized cooperatively in its own swarm.
Particle swarm optimisation for evolving artificial neural network
197 Citations2002Chunkai Zhang, Shao Hui-he +1 more
A new evolutionary system for evolving artificial feedforward neural networks, which is based on the particle swarm optimisation (PSO) algorithm, which shows that ANNs evolved by PSONN have good accuracy and generalisation ability.
Using the particle swarm optimization technique to train a recurrent neural model
118 Citations2002J. Salerno
A number of results using an evolutionary learning technique entitled particle swarm optimization on a number of neural model architectures using the XOR problem are presented and this technique is applied to a real problem-parsing natural language phrases.
A particle swarm optimization for reactive power and voltage control in electric power systems considering voltage security assessment
68 Citations2003Hotaka Yoshida, Yoshikazu Fukuyama +2 more
The proposed method expands the original PSO to handle a MINLP and determines an on-line VVC strategy with continuous and discrete control variables such as automatic voltage regulator operating values of generators, tap positions of on-load tap changer of transformers, and the number of pieces of reactive power compensation equipment.
Reducing the computational demands of continually online trained artificial neural networks for system identification and control of fast processes
53 Citations2002B. Burton, Ronald G. Harley
Many of the generic factors which influence the computational demands of continually online trained backpropagation artificial neural networks (ANNs) used to identify and control fast processes are described.
Application of combined neural networks in nonlinear function approximation
18 Citations2002Meng Jinli, Zhiyi Sun
The validity of the method presented is proved by two practical examples and the method can speed up the convergence and improve the generalization ability of neural networks.
A new evolved artificial neural network and its application
17 Citations2002Chunkai Zhang, Li Yu +1 more
An evolved artificial neural network that is evolved by the particle swarm optimisation (PSO) algorithm is used in modelling a product quality estimator for a fractionator of the hydrocracking unit in the oil refining industry and has good accuracy and generalisation ability.