Improving recall in associative memories by dynamic threshold
Neural NetworksPublished 1 January 1994
Tao Wang
Citations15
SJR quartileQ1
SJR score1.49
SNIP2.02
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A simple learning method and a dynamic threshold concept for associative memories (AMs) is presented and the dynamic threshold introduces a threshold in the recall phase to reduce the probability of converging to spurious states.
Abstract
In this paper, a simple learning method and a dynamic threshold concept for associative memories (AMs) is presented. The learning approach is designed to store all training patterns with basins of attraction as large as possible. After the learning process stops, the dynamic threshold introduces a threshold in the recall phase. It can reduce the probability of converging to spurious states. A large number of computer simulations are implemented to show the improved recalls.
Keywords
Computer ScienceNeuroscience
Proceedings of the National Academy of SciencesNeural networks and physical systems with emergent collective computational abilities.
19,349 Citations1982J. J. Hopfield
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Springer series in information sciencesSelf-Organization and Associative Memory
8,770 Citations1989Teuvo Kohonen
Springer series in information sciencesSelf-Organization and Associative Memory
3,820 Citations1988Teuvo Kohonen
IEEE Transactions on Electronic ComputersGeometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition
2,037 Citations1965Thomas M. Cover
It is shown that a family of surfaces having d degrees of freedom has a natural separating capacity of 2d pattern vectors, thus extending and unifying results of Winder and others on the pattern-separating capacity of hyperplanes.
Journal of Physics A Mathematical and GeneralThe space of interactions in neural network models
1,046 Citations1988E. Gardner
A local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist and is shown to be locally stable.
IEEE Transactions on Information TheoryThe capacity of the Hopfield associative memory
897 Citations1987Robert J. McEliece, Edward C. Posner +2 more
Techniques from coding theory are applied to study rigorously the capacity of the Hopfield associative memory, in particular to capacity under quantization of the outer-product connection matrix.
Biological CyberneticsNeural theory of association and concept-formation
502 Citations1977Шун-ичи Амари
Primitive neural models of association and concept-formation are presented, which will elucidate the distributed and multiply superposed manner of retaining knowledge in the brain.
IEEE Transactions on ComputersLearning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements
460 Citations1972Шун-ичи Амари
The stability of state transition in an autonomous logical net of threshold elements is studied by the use of characteristics of thresholds elements.
Journal of Physics A Mathematical and GeneralLearning algorithms with optimal stability in neural networks
425 Citations1987Werner Krauth, Marc Mézard
The authors motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix (Jij) and numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns.
Neural NetworksStatistical neurodynamics of associative memory
422 Citations1988Шун-ичи Амари, Kenjiro Maginu
A new statistical neurodynamical method is proposed for analyzing the non-equilibrium dynamical behaviors of an autocorrelation associative memory model and explains the strange behaviors due to strange shapes of the basins of attractors.
Neural networks: an introduction
409 Citations1995Berndt Müller, Joachim Reinhardt
The concepts of neural-network models and techniques of parallel distributed processing are comprehensively presented in a three-step approach and the reader is introduced to "neural" information processing, such as associative memory, perceptrons, feature-sensitive networks, learning strategies and practical applications.
Physical review. A, General physicsCollective computational properties of neural networks: New learning mechanisms
330 Citations1986L. Personnaz, Isabelle Guyon +1 more
Physical Review LettersLearning of correlated patterns in spin-glass networks by local learning rules
262 Citations1987S. Diederich, Manfred Opper
A partir d'une configuration arbitraire de liaisons synaptiques, on peut mettre en memoire jusqu'a N schemas par des modifications successives des efficacites synaptique dans un reseau de type verre de spins.
IEEE Control Systems MagazineAssociative memories via artificial neural networks
234 Citations1990A.N. Michel, Jay A. Farrell
Several design techniques that can be used for continuous-time and discrete-time neural networks to realize associative memories are presented and some stability concepts are outlined.
Proceedings of the IEEEMathematical foundations of neurocomputing
229 Citations1990Шун-ичи Амари
An advanced theory of learning and self-organization is proposed, covering backpropagation and its generalizations as well as the formation of topological maps and neural representations of information.
IEEE Transactions on Information TheoryA generalized convergence theorem for neural networks
133 Citations1988Jehoshua Bruck, Joseph W. Goodman
A neural network model is presented in which each neuron performs a threshold logic function that always converges to a stable state when operating in a serial mode and to a cycle of length at most 2 whenoperating in a fully parallel mode.
Journal of Physics A Mathematical and GeneralContent-addressability and learning in neural networks
114 Citations1988Bruce M. Forrest
Behaviour consistent with finite-size scaling, characteristic of a first-order phase transition, is shown to be exhibited by the basins of attraction of the stored patterns both in the case of the Hopfield model and for systems using a local iterative learning algorithm designed to optimise the basin of attraction.
Journal de physiqueDomains of attraction in neural networks
111 Citations1988Thomas B. Kepler, L. F. Abbott
Neural NetworksAssociative memory network composed of neurons with hysteretic property
57 Citations1990H. Yanai, Yasuji Sawada
For an auto-correlation-type associative memory network, the capacity becomes greater and basins of attractor of memorized patterns are larger when there is a hysteretic property than otherwise.
Journal of Physics A Mathematical and GeneralOptimal learning in neural network memories
55 Citations1989L. F. Abbott, Thomas B. Kepler
IEEE Transactions on Information TheoryOn the number of spurious memories in the Hopfield model (neural network)
48 Citations1990Jehoshua Bruck, Vwani Roychowdhury
The outer-product method for programming the Hopfield model is discussed, which can result in many spurious stable states-exponential in the number of vectors that are to be stored.
Optical EngineeringHigh Performance Recording Algorithm For Hopfield Model Associative Memories
29 Citations1989M.H. Hassoun, Amr Youssef
A new recording technique for Hopfield-type dynamic auto-associative memories based on the finite and exponentially convergent algorithm of Ho and Kashyap for the solution of a system of linear inequalities is proposed.
Biological CyberneticsMathematical aspects of outer-product asynchronous content-addressable memories
10 Citations1990E. K. Blum
Evidence is given by mathematical analysis and by example that the construction of CAM's as quasineural networks based on the twin principles of an outer-product weight matrix and a random asynchronous single-neuron dynamics encounters two obstacles to good performance which appear to be inherent.
International Journal of Pattern Recognition and Artificial IntelligenceCHARACTERIZING ONE-LAYER ASSOCIATIVE NEURAL NETWORKS WITH OPTIMAL NOISE-REDUCTION ABILITY
1 Citations1992Tao Wang, Xiaoliang Xing +1 more
An optimal learning algorithm for designing one-layer neural networks by means of global minimization is described, taking the properties of a well-defined neural network into account, and a cost function is derived to measure the goodness of the network quantitatively.