Associative memory with high information content
Physical review. A, General physicsPublished 1 March 1989
Joachim M. Buhmann, R. Divko, Klaus Schulten
Citations168
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
An extension of Hopfield's model with adaptive threshold and inhibitory interactions yields a network capable of nearly optimal storage of patterns of low activity. A replica symmetric solution of the mean-field equations is presented for the noise-free case. The following properties are demonstrated: For a low level of activity a the storage capacity increases as -(a lna${)}^{\mathrm{\ensuremath{-}}1}$; up to 0.38 bits per synapse can be stored; spurious states can be suppressed; the network is not opinionated, i.e., it can categorize inputs as not similar enough to patterns stored.
Keywords
Computer ScienceNeuroscienceEngineering
Proceedings of the National Academy of SciencesNeural networks and physical systems with emergent collective computational abilities.
19,349 Citations1982J. J. Hopfield
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Proceedings of the National Academy of SciencesNeurons with graded response have collective computational properties like those of two-state neurons.
6,955 Citations1984J. J. Hopfield
A model for a large network of "neurons" with a graded response (or sigmoid input-output relation) is studied and collective properties in very close correspondence with the earlier stochastic model based on McCulloch - Pitts neurons are studied.
Physical review. A, General physicsSpin-glass models of neural networks
1,193 Citations1985Daniel J. Amit, Hanoch Gutfreund +1 more
Two dynamical models, proposed by Hopfield and Little to account for the collective behavior of neural networks, are analyzed and it is shown that the long-time behavior of the two models is identical, for all temperatures below a transition temperature ${T}_{c}$.
Physical Review LettersStoring Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks
1,047 Citations1985Daniel J. Amit, Hanoch Gutfreund +1 more
The Hopfield model for a neural network is studied in the limit when the number of stored patterns increases with the size of the network, as $p=\ensuremath{\alpha}N$, and it is shown that, despite its spin-glass features, the model exhibits associative memory.
Journal of Physics A Mathematical and GeneralThe space of interactions in neural network models
1,046 Citations1988E. Gardner
A local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist and is shown to be locally stable.
NatureNon-Holographic Associative Memory
1,014 Citations1969David Willshaw, O. Buneman +1 more
The features of a hologram that commend it as a model of associative memory can be improved on by other devices.
IEEE Transactions on Information TheoryThe capacity of the Hopfield associative memory
897 Citations1987Robert J. McEliece, Edward C. Posner +2 more
Techniques from coding theory are applied to study rigorously the capacity of the Hopfield associative memory, in particular to capacity under quantization of the outer-product connection matrix.
Biological CyberneticsOn associative memory
445 Citations1980G Palm
The information storing capacity of certain associative and auto-associative memories is calculated and the usefulness of associative memories, as opposed to conventional listing memories, is discussed — especially in connection with brain modelling.
Journal of Physics A Mathematical and GeneralLearning algorithms with optimal stability in neural networks
425 Citations1987Werner Krauth, Marc Mézard
The authors motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix (Jij) and numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns.
Physical review. A, General physicsInformation storage in neural networks with low levels of activity
344 Citations1987Daniel J. Amit, Hanoch Gutfreund +1 more
The Hopfield model of a neural network is extended to allow for the storage and retrieval of biased patterns, and a global constraint is implemented, which restricts the configuration space to states whose mean activity is in the neighborhood of (1/2)(1+a).
Physical review. A, General physicsNeural networks with nonlinear synapses and a static noise
237 Citations1986Haim Sompolinsky
The theory of neural networks is extended to include a static noise as well as nonlinear updating of synapses by learning, which may modify the energy surface and lead to interesting new computational capabilities in an unsaturated network.
Europhysics Letters (EPL)Noise-Driven Temporal Association in Neural Networks
96 Citations1987Joachim M. Buhmann, Klaus Schulten
A network of spinlike neurons with asymmetric exchange interactions and stochastic spike response which can learn and recall time sequences of biased patterns is proposed.
Journal of Physics A Mathematical and GeneralDynamics and statistical mechanics of the Hopfield model
65 Citations1987Ann Bruce, E. Gardner +1 more
Large scale Monte Carlo simulations using a distributed array processor provide some support for the existence of a phase transition close to the predicted value of alpha c, implying a discontinuous change in the fraction of bits recalled correctly.
IEEE Transactions on Systems Man and CyberneticsElectronic Implementation of Associative Memory Based on Neural Network Models
34 Citations1987A. Moopenn, John Lambe +1 more
The nature of false memory errors, their effect on the information storage capacity of binary connection matrix memories, and a novel technique to eliminate such errors with the help of asymmetrical extra connections are discussed.
Europhysics Letters (EPL)Dynamical Learning Process for Recognition of Correlated Patterns in Symmetric Spin Glass Models
32 Citations1987Gerhard Pöppel, U. Krey
In the framework of spin glass models with symmetric interactions a local dynamical learning process is studied, by which the energy landscape is modified in such a way that even strongly correlated noisy patterns can be recognized.