Improved convergence rate of back-propagation with dynamic adaption of the learning rate
Lecture notes in computer sciencePublished 12 January 2006
Ralf Salomon
Citations10
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown, first, how the introduction of test cycles brings out a great improvement in the convergence rate and, second, that costly experiments used to adjust learning-relevant parameters could be dispensed with.
Abstract
This article deals with back-propagation, a learning method for neural nets. It is shown, first, how the introduction of test cycles brings out a great improvement in the convergence rate and, second, that costly experiments used to adjust learning-relevant parameters could be dispensed with.
Keywords
Computer ScienceEngineering
Neural NetworksIncreased rates of convergence through learning rate adaptation
1,795 Citations1988Robert A. Jacobs
A study of Steepest Descent and an analysis of why it can be slow to converge and four heuristics for achieving faster rates of convergence are proposed.
Neural Information Processing SystemsEfficient Parallel Learning Algorithms for Neural Networks
133 Citations1988Alan Kramer, Alberto Sangiovanni‐Vincentelli
Parallelizable optimization techniques such as the Polak-Ribiere method are significantly more efficient than the Backpropagation algorithm and the noisy real-valued learning problem of hand-written character recognition.
IEEE International Conference on Neural NetworksImproving the learning rate of back-propagation with the gradient reuse algorithm
76 Citations1988Hush, Azucena Salas
It is shown that convergence speedup is a function of the reuse rate, and that the reused rate can be controlled by using a dynamic convergence parameter.