Characteristics of sparsely encoded associative memory
Neural NetworksPublished 1 January 1989
Шун-ичи Амари
Citations215
SJR quartileQ1
SJR score1.49
SNIP2.02
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is proved that the sparsely encoded associative memory has a large basin of attraction around each memorized pattern, when and only when an activity control mechanism is attached to it.
Abstract
Characteristics of an autocorrelation or crosscorrelation associative memory largely depend on how items are encoded in pattern vectors to be stored. When most of the components of encoded patterns to be stored are 0 and only a small ratio of the components are 1, the encoding scheme is said to be sparse. The memory capacity and information capacity of a sparsely encoded associative memory are analyzed in detail, and are proved to be in proportion of n2logn2, n being the number of neurons, which is very large compared to the ordinary non-sparse encoding scheme of about 0.15n. Moreover, it is proved that the sparsely encoded associative memory has a large basin of attraction around each memorized pattern, when and only when an activity control mechanism is attached to it.
Keywords
Computer ScienceNeuroscienceEngineering
Proceedings of the National Academy of SciencesNeural networks and physical systems with emergent collective computational abilities.
19,349 Citations1982J. J. Hopfield
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Physical review. A, General physicsSpin-glass models of neural networks
1,193 Citations1985Daniel J. Amit, Hanoch Gutfreund +1 more
Two dynamical models, proposed by Hopfield and Little to account for the collective behavior of neural networks, are analyzed and it is shown that the long-time behavior of the two models is identical, for all temperatures below a transition temperature ${T}_{c}$.
Journal of Physics A Mathematical and GeneralThe space of interactions in neural network models
1,046 Citations1988E. Gardner
A local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist and is shown to be locally stable.
IEEE Transactions on Information TheoryThe capacity of the Hopfield associative memory
897 Citations1987Robert J. McEliece, Edward C. Posner +2 more
Techniques from coding theory are applied to study rigorously the capacity of the Hopfield associative memory, in particular to capacity under quantization of the outer-product connection matrix.
NatureNeuronal correlate of pictorial short-term memory in the primate temporal cortexYasushi Miyashita
888 Citations1988Yasushi Miyashita, Han Soo Chang
A group of shape-selective neurons in an anterior ventral part of the temporal cortex of monkeys that exhibited sustained activity during the delay period of a visual short-term memory task are reported.
IEEE Transactions on ComputersCorrelation Matrix Memories
878 Citations1972Teuvo Kohonen
A new model for associative memory, based on a correlation matrix, is suggested, in which any part of the memorized information can be used as a key and the memories are selective with respect to accumulated data.
Biological CyberneticsNeural theory of association and concept-formation
502 Citations1977Шун-ичи Амари
Primitive neural models of association and concept-formation are presented, which will elucidate the distributed and multiply superposed manner of retaining knowledge in the brain.
IEEE Transactions on ComputersLearning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements
460 Citations1972Шун-ичи Амари
The stability of state transition in an autonomous logical net of threshold elements is studied by the use of characteristics of thresholds elements.
Biological CyberneticsOn associative memory
445 Citations1980G Palm
The information storing capacity of certain associative and auto-associative memories is calculated and the usefulness of associative memories, as opposed to conventional listing memories, is discussed — especially in connection with brain modelling.
Neural NetworksStatistical neurodynamics of associative memory
422 Citations1988Шун-ичи Амари, Kenjiro Maginu
A new statistical neurodynamical method is proposed for analyzing the non-equilibrium dynamical behaviors of an autocorrelation associative memory model and explains the strange behaviors due to strange shapes of the basins of attractors.
Elsevier eBooksCompetition and Cooperation in Neural Nets
379 Citations1977Шун-ичи Амари, Michael A. Arbib
The aim of this paper is to develop a mathematical study of a number of neural models as a theory of neural fields, with special attention to the analysis of equilibrium states of the system and their stability, and to the interaction of different stimuli separated in space and time.
IEEE Transactions on Systems Man and CyberneticsAssociatron-A Model of Associative Memory
366 Citations1972Kaoru Nakano
The Associatron is considered to be a simplified model of the neural network and can be constructed as a cellular structure, where each cell is connected to only its neighbor cells and all cells run in parallel.
Journal de Physique LettresScaling laws for the attractors of Hopfield networks
103 Citations1985G. Weisbuch, F. Fogelman-Soulie
Physical Review LettersExact solution of a layered neural network model
69 Citations1987Ronny Meir, Eytan Domany
Biological CyberneticsOn the storage capacity of an associative memory with randomly distributed storage elements
47 Citations1981G. Palm
In an associative memory with randomly distributed storage elements at least 0.05 bit per storage element can be stored, the number of elements in the memory is determined by the random number generator.
IEEE Transactions on Pattern Analysis and Machine IntelligenceReliability and Speed of Recall in an Associative Network
38 Citations1985Anders Lansner, Örjan Ekeberg
It is shown that a high storage capacity is possible, without sacrificing reliability in the recall process, an efficient algorithm for retrieval of the information stored, and the speed of recall employing various degrees of parallelism is discussed.
Springer series in synergeticsLocal Synaptic Rules with Maximal Information Storage Capacity
13 Citations1988G. Palm
A family of mathematical neural network models that investigate learning and synaptic plasticity that is a long way from the reality of magnetic materials, but still further remote from real brain dynamics.