Inversion of multilayer nets
Published 1 January 1989
Linden, Kindermann
Citations203
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is concluded that recurrent nets and even time-delay nets might be invertible.
Abstract
The method of inversion for arbitrary continuous multilayer nets is developed. The inversion is done by computing iteratively an input vector which minimizes the least-mean-square errors to approximate a given output target. This inversion is not unique for given targets and depends on the starting point in input space. The inversion method turns out to be a valuable tool for the examination of multilayer nets (MLNs). Applications of the inversion method to constraint satisfaction, feature detection, and the testing of reliability and performance of MLNs are outlined. It is concluded that recurrent nets and even time-delay nets might be invertible.>
Keywords
Computer ScienceEngineering
NatureLearning representations by back-propagating errors
30,885 Citations1986David E. Rumelhart, Geoffrey E. Hinton +1 more
Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Machine LearningLearning to Predict by the Methods of Temporal Differences
3,894 Citations1988Richard S. Sutton
This article introduces a class of incremental learning procedures specialized for prediction-that is, for using past experience with an incompletely known system to predict its future behavior, and proves their convergence and optimality for special cases and relate them to supervised-learning methods.
Machine LearningLearning to predict by the methods of temporal differences
2,758 Citations1988Richard S. Sutton
Physical Review LettersGeneralization of back-propagation to recurrent neural networks
951 Citations1987Fernando J. Pineda
An adaptive neural network with asymmetric connections is introduced that bears a resemblance to the master/slave network of Lapedes and Farber but it is architecturally simpler.
Scholarworks (University of Massachusetts Amherst)Temporal credit assignment in reinforcement learning
778 Citations1984Richard S. Sutton
Neural Information Processing SystemsSkeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
539 Citations1988Michael C. Mozer, Paul Smolensky
The basic idea is to iteratively train the network to a certain performance criterion, compute a measure of relevance that identifies which input or hidden units are most critical to performance, and automatically trim the least relevant units.
ScholarlyCommons (University of Pennsylvania)Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization
239 Citations1988Raymond L. Watrous
It is shown that in plateau regions of relatively constant gradient, the momentum term acts to increase the step size by a factor of 1/1-μ, where μ is the momentumTerm, and in valley regions with steep sides,The momentum constant acts to focus the search direction toward the local minimum by averaging oscillations in the gradient.
IEEE International Conference on Neural NetworksBackpropagation: past and future
223 Citations1988Werbos
The author proposes development of a general theory of intelligence in which backpropagation and comparisons to the brain play a central role, and points to a series of intermediate steps and applications leading up to the construction of such generalized systems.
Neural ComputationDeterministic Boltzmann Learning Performs Steepest Descent in Weight-Space
209 Citations1989Geoffrey E. Hinton
By using the appropriate interpretation for the way in which a DBM represents the probability of an output vector given an input vector, it is shown that the DBM performs steepest descent in the same function as the original SBM, except at rare discontinuities.
Neural NetworksExplorations of the mean field theory learning algorithm
122 Citations1989Carsten Peterson, Eric Hartman
The mean field theory learning algorithm is benchmarked against the back-propagation learning algorithm (BP) on two different feature recognition problems: two-dimensional mirror symmetry and multidimensional statistical pattern classification.
Informatik-FachberichteDetection of Minimal Microfeatures by Internal Feedback
5 Citations1989Jörg Kindermann, Alexander Linden
The notion of minimal microfeatures is defined and a new method of internal feedback for multilayer networks is introduced, which allows the detection of sets of minimalmicrofeatures, i.