Holographic reduced representations
IEEE Transactions on Neural NetworksPublished 1 May 1995
Tony Plate
Citations660
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper describes a method for representing more complex compositional structure in distributed representations that uses circular convolution to associate items, which are represented by vectors.
Abstract
Associative memories are conventionally used to represent data with very simple structure: sets of pairs of vectors. This paper describes a method for representing more complex compositional structure in distributed representations. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, simple frame-like structures, and reduced representations can be represented in a fixed width vector. These representations are items in their own right and can be used in constructing compositional structures. The noisy reconstructions extracted from convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.
Keywords
Computer SciencePhysics and Astronomy
Proceedings of the National Academy of SciencesNeural networks and physical systems with emergent collective computational abilities.
19,349 Citations1982J. J. Hopfield
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
The MIT Press eBooksParallel Distributed Processing
15,227 Citations1986David E. Rumelhart, James L. McClelland +1 more
CognitionConnectionism and cognitive architecture: A critical analysis
4,011 Citations1988Jerry A. Fodor, Zenon W. Pylyshyn
Differences between Connectionist proposals for cognitive architecture and the sorts of models that have traditionally been assumed in cognitive science are explored and the possibility that Connectionism may provide an account of the neural structures in which Classical cognitive architecture is implemented is considered.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Distributed representations
1,043 Citations2014Geoffrey E. Hinton
Medical Entomology and ZoologySparse Distributed Memory
973 Citations1988Pentti Kanerva
Pentti Kanerva's Sparse Distributed Memory presents a mathematically elegant theory of human long term memory that resembles the cortex of the cerebellum, and provides an overall perspective on neural systems.
Artificial IntelligenceTensor product variable binding and the representation of symbolic structures in connectionist systems
959 Citations1990Paul Smolensky
The tensor product representation rests on a principled analysis of structure; it saturates gracefully as larger structures are represented; it permits recursive construction of complex representations from simpler ones; it extends naturally to continuous structures and continuous representational patterns.
Artificial IntelligenceRecursive distributed representations
866 Citations1990Jordan Pollack
This paper presents a connectionist architecture which automatically develops compact distributed representations for variable-sized recursive data structures, as well as efficient accessing mechanisms for them.
Fast Fourier Transform
375 Citations1975N. Ahmed, K.R. Rao
This chapter develops a fast algorithm, called the fast Fourier transform (FFT), which significantly reduces the number of arithmetic operations and memory required to compute the DFT (or its inverse).
IEEE Transactions on Signal ProcessingOn the complex backpropagation algorithm
316 Citations1992N. Benvenuto, Francesco Piazza
A recursive algorithm for updating the coefficients of a neural network structure for complex signals is presented and the method yields the complex form of the conventional backpropagation algorithm.
Artificial IntelligenceMapping part-whole hierarchies into connectionist networks
311 Citations1990Geoffrey E. Hinton
Three different ways of mapping part-whole hierarchies into connectionist networks are described, suggesting that neural networks have two quite different methods for performing inference.
Cognitive ScienceA Distributed Connectionist Production System
287 Citations1988David S. Touretzky, Geoffrey E. Hinton
The simulation of DCPS is intended as a detailed demonstration of the feasibility of certain ideas and should not be viewed as a full implementation of production systems.
Signals and linear systems
223 Citations1973Robert A. Gabel, Richard A. Roberts
This book discusses linear Systems, Discrete-Time Systems, and Continuous-Time systems, and an Introduction to the Design of Digital Filters.
Neural ComputationOptimal Plasticity from Matrix Memories: What Goes Up Must Come Down
193 Citations1990David Willshaw, Peter Dayan
Calculating and maximizing the signal-to-noise ratio, a measure of the potential fidelity of recall, in a class of associative matrix memories, addresses the issue of the computational efficiency of particular synaptic learning rules homosynaptic versus heterosyaptic and monotonic versus nonmonotonic changes in synaptic efficacy.
Choice Reviews OnlineHandbook of digital signal processing: engineering applications
164 Citations1988
This book discusses the design and implementation of Digital FIR Filters for Interpolating and Desampling, and the role of Fourier Transforms in this process.
Optical EngineeringOptical Associative Memory Using Fourier Transform Holograms
136 Citations1987Eung Gi Paek, Demetri Psaltis
An experimental demonstration of a holographic associative memory that utilizes an array of classic VanderLugt correlators to implement in parallel the inner product between an input and a set of stored reference images.
Artificial IntelligenceBoltzCONS: Dynamic symbol structures in a connectionist network
116 Citations1990David S. Touretzky
The point of the work is to show how neural networks can exhibit compositionality and distal access, two properties that distinguish symbol processing from lower-level cognitive functions such as pattern recognition.
Applied OpticsOptical implementations of associative networks with versatile adaptive learning capabilities
91 Citations1987Arthur D. Fisher, Wendy L. Lippincott +1 more
The practical issues involved in real optical architectures are analyzed, and actual laboratory optical implementations of associative modules based on Hebbian and Widrow-Hoff learning rules are discussed, including successful experimental demonstrations of their operation.
FigshareA distributed connectionist representation for concept structures
75 Citations2018David S. Touretzky, Shai Geva +1 more
This work describes a representation for frame-like concept structures in a neural network called DUCS and shows how Hinton's notion of a 'reduced description' can be used to make one concept fill a slot in another.
Journal of Mathematical PsychologyExpectation and variance of item resemblance distributions in a convolution-correlation model of distributed memory
35 Citations1988Elke U. Weber
The applicability of distributed memory models that use convolution and correlation as their encoding and retrieval operations are extended by providing analytic derivations of the expressions needed to apply such models to a wide range of experimental paradigms.
Biological CyberneticsSome algebraic relations between involutions, convolutions and correlations, with applications to holographic memories
35 Citations2008Peter H. Schönemann
Biological CyberneticsSome algebraic relations between involutions, convolutions, and correlations, with applications to holographic memories
32 Citations1987Peter H. Schönemann
This work has shown that the relation a#b=S(a * Sb), whereS is an involution which reflects the order in the integral domainZ on which the sequences are defined, can be used to represent a non-associative correlation algebra by an associative convolution algebra equipped with the involutionS.
Holographic Recurrent Networks
32 Citations1992Tony Plate
Holographic Recurrent Networks are recurrent networks which incorporate associative memory techniques for storing sequential structure and the performance of HRNs is found to be superior to that of ordinary recurrent networks on sequence generation tasks.
KybernetikOn holographic models of memory
24 Citations1973Tomaso Poggio
By means of the Volterra series an extension of the association properties of holography is proposed and the significance of non-linearities embedded within the formal structure of a holographic-like associative memory is pointed out.
Applied OpticsKey and recollection vector effects on heteroassociative memory performance
19 Citations1989David Casasent, Brian A. Telfer
This work derive new equations for the storage capacity and noise performance of HAMs, emphasize how they differ from those derived for AAMs, suggest new performance measures to be used, and show how different recollection vector encodings can improve HAM performance.
FigshareCoarse-Coded Symbol Memories and Their Properties
13 Citations1988Rosenfeld, Ronald, David S. Touretzky
The predicted capacity of one of the schemes is compared with the actual measur ements of the coar secoded working memory of DCP S, Touret zky and Hinto n's dist ributed connectionist product ion system, and a simple linear relat ionship between the resources allocated to the syste m and the capacity is found.
A parsing architecture based on distributed memory machines
5 Citations1984Jon M. Slack
The paper demonstrates how principles of Lexical Functional Grammar can be derived from the properties of the memory mechanisms of distributed memory machines.