Learning and Generalization with Minimerror, A Temperature-Dependent Learning Algorithm
Neural ComputationPublished 1 November 1995
Bruno Raffin, Mirta B. Gordon
Citations10
SJR quartileQ1
SJR score0.83
SNIP1.45
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The numerical performances of Minimerror, a recently introduced learning algorithm for the perceptron that has analytically been shown to be optimal both on learning linearly and nonlinearly separable functions, are studied.
Abstract
We study the numerical performances of Minimerror, a recently introduced learning algorithm for the perceptron that has analytically been shown to be optimal both on learning linearly and nonlinearly separable functions. We present its implementation on learning linearly separable boolean functions. Numerical results are in excellent agreement with the theoretical predictions.
Keywords
Computer ScienceEngineering
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)The Cascade-Correlation learning architecture
2,401 Citations2018Scott E. Fahlman, Christian Lebière
The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network.
NeurocomputingSingle-layer learning revisited: a stepwise procedure for building and training a neural network
825 Citations1990S. Knerr, L. Personnaz +1 more
A stepwise procedure for building and training a neural network intended to perform classification tasks, based on single layer learning rules, is presented, which breaks up the classification task into subtasks of increasing complexity in order to make learning easier.
Journal of Physics A Mathematical and GeneralOptimal storage properties of neural network models
558 Citations1988E. Gardner, Bernard Derrida
The authors calculate the number, p= alpha N of random N-bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter K.
IEEE Transactions on Neural NetworksPerceptron-based learning algorithms
551 Citations1990Stephen I. Gallant
The heart of these algorithms is the pocket algorithm, a modification of perceptron learning that makes perceptronLearning well-behaved with nonseparable training data, even if the data are noisy and contradictory.
Neural ComputationThe Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks
516 Citations1990Marcus Frean
Simulations suggest that this method for building and training multilayer perceptrons composed of linear threshold units is efficient in terms of the numbers of units constructed, and the networks it builds can generalize over patterns not in the training set.
Physical Review AStatistical mechanics of learning from examples
513 Citations1992H. Sebastian Seung, Haim Sompolinsky +1 more
It is shown that for smooth networks, i.e., those with continuously varying weights and smooth transfer functions, the generalization curve asymptotically obeys an inverse power law, while for nonsmooth networks other behaviors can appear, depending on the nature of the nonlinearities as well as the realizability of the rule.
Reviews of Modern PhysicsThe statistical mechanics of learning a rule
473 Citations1993Timothy L. H. Watkin, Albrecht Rau +1 more
Journal of Physics A Mathematical and GeneralLearning in feedforward layered networks: the tiling algorithm
431 Citations1989Marc Mézard, Jean‐Pierre Nadal
A new algorithm which builds a feedforward layered network in order to learn any Boolean function of N Boolean units, which is an algorithm for growth of the network, which adds layers, and units inside a layer, at will until convergence.
Journal of Physics A Mathematical and GeneralLearning algorithms with optimal stability in neural networks
425 Citations1987Werner Krauth, Marc Mézard
The authors motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix (Jij) and numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns.
Europhysics Letters (EPL)Maximum Storage Capacity in Neural Networks
350 Citations1987E. Gardner
The upper storage capacity of a neural network for patterns of fixed magnetization m is calculated and the optimal capacity increases with the correlation m2 between the patterns.
Europhysics Letters (EPL)The AdaTron: An Adaptive Perceptron Algorithm
209 Citations1989Joachim K. Anlauf, Michael Biehl
A new learning algorithm for neural networks of spin glass type is proposed that is found to relax exponentially towards the perceptron of optimal stability using the concept of adaptive learning.
Physical Review LettersGeneralization performance of Bayes optimal classification algorithm for learning a perceptron
152 Citations1991Manfred Opper, David Haussler
It is shown that approximations to the generalization error of the Bayes optimal algorithm can be achieved by learning algorithms that use a two-layer neutral net to learn a perceptron.
Journal of Physics A Mathematical and GeneralOn the ability of the optimal perceptron to generalise
120 Citations1990Manfred Opper, Wolfgang Kinzel +2 more
A linearly separable Boolean function is derived from a set of examples by a perceptron with optimal stability by calculating the probability to reconstruct a pattern which is not learnt using the replica method.
Network Computation in Neural SystemsNeural trees: a new tool for classification
111 Citations1990J. A. Sirat, Jean‐Pierre Nadal
A new classifier based on neural network techniques, inspired from a growth algorithm, recently introduced for feedforward neural networks, which is easily and efficiently extended to classification in a multiclass probl...
Neural ComputationA "Thermal" Perceptron Learning Rule
102 Citations1992Marcus Frean
The thermal perceptron is a simple extension to Rosenblatt's perceptron learning rule for training individual linear threshold units that finds stable weights for nonseparable problems as well as separable ones.
Europhysics Letters (EPL)The Hebb Rule for Learning Linearly Separable Boolean Functions: Learning and Generalization
77 Citations1989F. Vallet
The Hebb solution for the perceptron realization of an arbitrary linearly separable Boolean function defined on the hypercube of dimension N is investigated and the learning and generalization rates in the N → ∞ limit are calculated.
Journal of Physics A Mathematical and GeneralThe roles of stability and symmetry in the dynamics of neural networks
70 Citations1988Werner Krauth, J -P Nadal +1 more
It is proposed that two parameters, characterising the symmetry of the neural net's connections and the stabilities of the patterns, are responsible for most of the dynamical effects of the retrieval phase of spin-glass-like neural networks.
Physical Review LettersInference of a rule by a neural network with thermal noise
69 Citations1990G. Györgyi
A transition to optimal learning at finite temperature is found and the generalization error can be decreased by adding thermal noise to the synaptic coupling parameters.
Journal of Physics A Mathematical and GeneralLearning and retrieval in attractor neural networks above saturation
63 Citations1991Meir Griniasty, Hanoch Gutfreund
The authors investigate neural networks in the range of parameters when the ground-state energy is positive to conclude that the retrieval behaviour of attractor neural networks can be improved by learning above saturation.
Europhysics Letters (EPL)A Growth Algorithm for Neural Network Decision Trees
58 Citations1990Mostefa Golea, Mario Marchand
This paper considers the problem of constructing a tree of perceptrons able to execute a given but arbitrary Boolean function defined on Ni input bits and applies a sequential and parallel learning procedure to add hidden units until the task in hand is performed.
Journal of Physics A Mathematical and GeneralRetrieval phase diagrams for attractor neural networks with optimal interactions
44 Citations1990D J Amit, M. R. Evans +2 more
The authors consider the retrieval properties of attractor neural networks whose synaptic matrices have been constructed to maximise the number of patterns which can be stored in a perceptron satisfying certain constraints and finds that at high synaptic noise levels the diluted Hopfield net stores more effectively than the network trained for optimal perceptron storage.
Europhysics Letters (EPL)The Offset Algorithm: Building and Learning Method for Multilayer Neural Networks
30 Citations1992Dominique Martinez, D. Estève
A general method for building and training a multilayer neural network functioning as a parity machine is proposed, composed of two basic steps: a growth step in which two hidden layers are built and a pruning process in which any redundant units are removed.
Journal de Physique IA fast method for calculating the perceptron with maximal stability
28 Citations1993P. Ruján
An active set approach to the dual problem, finding the minimal connector between two disjoint convex hulls using the Perceptron with maximal stability.
Neural ComputationLearning the Unlearnable
18 Citations1991Dan Nabutovsky, Eytan Domany
It is proved that when no solution exists, the algorithm detects this in a finite time, which is polynomial in typical cases and exponential in the worst case, when the set of patterns is nonstrictly linearly separable.
Europhysics Letters (EPL)Learning with a Temperature-Dependent Algorithm
14 Citations1995Mirta B. Gordon, D. R Grempel
It is shown that learning at finite temperature increases the stabilities of learned patterns, endowing the perceptron with robustness, at the price of accepting a small fraction of errors in the learning set.
Journal de Physique ILearning algorithms for perceptrons from statistical physics
7 Citations1993Mirta B. Gordon, P. Péretto +1 more
In the limits of zero or infinite temperatures some of the rules behave like already known algorithms, but new strategies for learning are obtained at finite temperatures, which minim12e the number of errors on the training set.
Journal of Physics A Mathematical and GeneralThe generalization probability of a perceptron using the Delta -rule
1 Citations1992S Seewer, P. Ruján
The numerical results indicate that the Delta -rule gives results which are superior to Hebb's rule, and comparable with the maximal stability perceptron rule (minover), both for randomly generated linear separable example sets and for optimal queries.