Improving the Rprop Learning Algorithm
Published 1 January 2000
Christian Igel, Michael Hüsken
Citations350
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Modifications of the Rprop algorithm are introduced that improve its learning speed and the resulting speedup is experimentally shown for a set of neural network learning tasks as well as for artificial error surfaces.
Abstract
The Rprop algorithm proposed by Riedmiller and Braun is one of the best performing first-order learning methods for neural networks. We introduce modifications of the algorithm that improve its learning speed. The resulting speedup is experimentally shown for a set of neural network learning tasks as well as for artificial error surfaces.
Keywords
Computer SciencePhysics and Astronomy
IEEE International Conference on Neural NetworksA direct adaptive method for faster backpropagation learning: the RPROP algorithm
3,927 Citations2002Martin Riedmiller, Heinrich Braun
A learning algorithm for multilayer feedforward networks, RPROP (resilient propagation), is proposed that performs a local adaptation of the weight-updates according to the behavior of the error function to overcome the inherent disadvantages of pure gradient-descent.
Neural NetworksIncreased rates of convergence through learning rate adaptation
1,795 Citations1988Robert A. Jacobs
A study of Steepest Descent and an analysis of why it can be slow to converge and four heuristics for achieving faster rates of convergence are proposed.
Biological CyberneticsAccelerating the convergence of the back-propagation method
1,101 Citations1988Thomas P. Vogl, J. K. Mangis +3 more
Considering the selection of weights in neural nets as a problem in classical nonlinear optimization theory, the rationale for algorithms seeking only those weights that produce the globally minimum error is reviewed and rejected.
Computer Standards & InterfacesAdvanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms
594 Citations1994Martin Riedmiller
The concept of supervised learning in multi-layer perceptrons based on the technique of gradient descent is introduced and the behavior of several learning procedures on some popular benchmark problems is reported, thereby illuminating convergence, robustness, and scaling properties of the respective algorithms.
Neural NetworksSuperSAB: Fast adaptive back propagation with good scaling properties
363 Citations1990Tom Tollenaere
It is shown that SuperSAB may converge orders of magnitude faster than the original back propagation algorithm, and is only slightly instable, while the algorithm is very insensitive to the choice of parameter values, and has excellent scaling properties.
Repository KITopen (Karlsruhe Institute of Technology)PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms
234 Citations1994Lutz Prechelt
The purpose of the problem and rule collection is to give researchers easy access to data for the evaluation of their algorithms and networks and to make direct comparison of the published results feasible.
On the Adaptation of Arbitrary Normal Mutation Distributions in Evolution Strategies: The Generating Set Adaptation
169 Citations1995Nikolaus Hansen, Andreas Ostermeier +1 more
A new adaptation scheme for adapting arbitrary normal mutation distributions in evolution strategies is introduced which can adapt correct scaling and correlations between object parameters and reliably adapts mutation distributions corresponding to hyperellipsoids with high axis ratio.
Lecture notes in computer scienceAcceleration techniques for the backpropagation algorithm
149 Citations1990Fernando M. Silva, Luı́s B. Almeida
Backpropagation converges slowly, even for medium sized network problems, because of the usually large dimension of the weight space and from the particular shape of the error surface in each iteration point.
Elsevier eBooksSpeeding up Backpropagation
147 Citations1990Fernando M. Silva, Luı́s B. Almeida
This paper presents an improvement to the backpropagation algorithm, based on the use of an independent, adaptive learning rate parameter for each weight, and gives experimental results showing the improvement in learning speed.
Comparison of optimized backpropagation algorithms.
78 Citations1993Wolfram Schiffmann, Merten Joost +1 more
An overview of many different speedup techniques for backpropagation and many of them were assessed by a very hard practical classification task, which consists of a big medical data set.
International Journal of Uncertainty Fuzziness and Knowledge-Based SystemsSpeeding Up Backpropagation Algorithms by Using Cross-Entropy Combined with Pattern Normalization
38 Citations1998Merten Joost, Wolfram Schiffmann
This paper demonstrates how the backpropagation algorithm (BP) and its variants can be accelerated significantly while the quality of the trained nets will increase.
arXiv (Cornell University)Neural Network Methods for Boundary Value Problems Defined in Arbitrarily Shaped Domains
6 Citations1998I.E. Lagaris, Aristidis Likas +1 more
Two networks are employed: a multilayer perceptron and a radial basis function network are used to account for the satisfaction of the boundary conditions of complex boundary geometry.