Connectionism: past, present, and future
Artificial Intelligence ReviewPublished 1 January 1989
Jordan Pollack
Citations51
SJR quartileQ1
SJR score3.01
SNIP5.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The history of the field, often in relation to AI, is surveyed, its current successes and failures are discussed, and some predictions for where it might lead in the future are made.
Abstract
Research efforts to study computation and cognitive modeling on neurally-inspired mechanisms have come to be called Connectionism. Rather than being brand new, it is actually the rebirth of a research programme which thrived from the 40s through the 60s and then was severely retrenched in the 70s. Connectionism is often posed as a paradigmatic competitor to the Symbolic Processing tradition of Artificial Intelligence (Dreyfus & Dreyfus, 1988), and, indeed, the counterpoint in the timing of their intellectual and commercial fortunes may lead one to believe that research in cognition is merely a zero-sum game. This paper surveys the history of the field, often in relation to AI, discusses its current successes and failures, and makes some predictions for where it might lead in the future.
Keywords
Computer ScienceSocial Sciences
ScienceOptimization by Simulated Annealing
44,600 Citations1983Scott Kirkpatrick, C. D. Gelatt +1 more
A detailed analogy with annealing in solids provides a framework for optimization of the properties of very large and complex systems.
American Journal of Physics<i>The Fractal Geometry of Nature</i>
21,750 Citations1983Benoît B. Mandelbrot, John Wheeler
Proceedings of the National Academy of SciencesNeural networks and physical systems with emergent collective computational abilities.
19,349 Citations1982J. J. Hopfield
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.
Bulletin of Mathematical BiologyA logical calculus of the ideas immanent in nervous activity
17,984 Citations1943Warren S. McCulloch, Walter Pitts
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
The MIT Press eBooksParallel Distributed Processing
15,227 Citations1986David E. Rumelhart, James L. McClelland +1 more
Journal of the Royal Statistical Society Series A (General)The Fractal Geometry of Nature.
14,691 Citations1984Colin Sparrow, B. B. Mandelbrot
Cognitive ScienceFinding Structure in Time
10,841 Citations1990Jeffrey L. Elman
A proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory and suggests a method for representing lexical categories and the type/token distinction is developed.
Biological Cybernetics“Neural” computation of decisions in optimization problems
6,040 Citations1985J. J. Hopfield, David W. Tank
Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks.
The American Journal of PsychologyThe Organization of Behavior; A Neuropsychological Theory
4,856 Citations1950Fred Attneave, M. B. +1 more
Journal of the American Medical AssociationThe Organization of Behavior: A Neuropsychological Theory
4,240 Citations1950Therese Mahoney, Margaret Halliday +339 more
CognitionConnectionism and cognitive architecture: A critical analysis
4,011 Citations1988Jerry A. Fodor, Zenon W. Pylyshyn
Differences between Connectionist proposals for cognitive architecture and the sorts of models that have traditionally been assumed in cognitive science are explored and the possibility that Connectionism may provide an account of the neural structures in which Classical cognitive architecture is implemented is considered.
Long Range PlanningChaos: Making a new science
3,460 Citations1989
The chaos making a new science is universally compatible with any devices to read, and it is available in the digital library an online access to it is set as public so you can download it instantly.
Cognitive ScienceA Learning Algorithm for Boltzmann Machines*
3,324 Citations1985David H. Ackley, Geoffrey E. Hinton +1 more
Chaos: The Making of a New Science
2,725 Citations1988James Gleick
Behavioral and Brain SciencesOn the proper treatment of connectionism
2,081 Citations1988Paul Smolensky
A picture is painted in which traditional and connectionist theoretical constructs collaborate intimately to provide an understanding of cognition.
MIT Press eBooksInformation processing in dynamical systems: foundations of harmony theory
1,666 Citations1986Paul Smolensky
The work reported in this chapter rests on the conviction that a methodology that has a crucial role to play in the development of cognitive science is mathematical analysis.
Psychological ReviewAn interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model.
1,501 Citations1982David E. Rumelhart, James L. McClelland
The interactive activation model of context effects in letter perception is reviewed, elaborated, and tested, and several recent findings that seem to challenge the model are considered and a number of extensions are proposed.
Cognitive ScienceCompetitive Learning: From Interactive Activation to Adaptive Resonance
1,476 Citations1987Stephen Grossberg
Functional and mechanistic comparisons are made between several network models of cognitive processing: competitive learning, interactive activation, adaptive resonance, and back propagation.
Cognitive ScienceConnectionist Models and Their Properties
1,324 Citations1982Jerome A. Feldman, D.H. Ballard
A general connectionist model is introduced and how it might be used in cognitive science is considered, among the issues addressed are: stability and noise-sensitivity, distributed decision-making, time and sequence problems, and the representation of complex concepts.
Advances in psychologySerial Order: A Parallel Distributed Processing Approach
1,230 Citations1997Michael I. Jordan
A theory of learned sequential behavior is presented, with a focus on coarticulatory phenomena in speech, implemented as a recurrent parallel distributed processing network that is trained via a generalized error-correcting algorithm.
Neural NetworksAnalysis of hidden units in a layered network trained to classify sonar targets
1,111 Citations1988Rebecca Gorman, Terrence J. Sejnowski
A neural network learning procedure has been applied to the classification of sonar returns from two undersea targets, a metal cylinder and a similarly shaped rock, and network performance and classification strategy was comparable to that of trained human listeners.
Cognitive ScienceFeature Discovery by Competitive Learning*
1,073 Citations1985David E. Rumelhart, David Zipser
This paper shows how a set of feature detectors which capture important aspects of the set of stimulus input patterns are discovered and how these feature detectors form the basis of a multilayer system that serves to learn categorizations of stimulus sets which are not linearly separable.
Parallel models of associative memory
920 Citations1981Geoffrey E. Hinton, James A. Anderson
This chapter discusses G.E. Hinton's models of Information Processing in the Brain, Implementing Semantic Networks in Parallel Hardware, and R. Ratcliff's Parallel-Processing Mechanisms and Processing of Organized Information in Human Memory.
Psychological ReviewDistinctive features, categorical perception, and probability learning: Some applications of a neural model.
860 Citations1977James A. Anderson, Jack W. Silverstein +2 more
The model can predict overshooting, recency data, and probabilities occurring in systems with more than two events with reasonably good accuracy and applies the model to "categorical perception" and probability learning.
Learning distributed representations of concepts.
821 Citations1989Geoffrey E. Hinton
Communication and cyberneticsAssociative Memory
455 Citations1977Teuvo Kohonen
This associative memory a system theoretical approach is well known book in the world, of course many people will try to own it and this is it the book that you can receive directly after purchasing.
ScienceChaos, Strange Attractors, and Fractal Basin Boundaries in Nonlinear Dynamics
437 Citations1987Celso Grebogi, Edward Ott +1 more
Basic developments in the field of chaotic dynamics of dissipative systems are reviewed, Topics covered include strange attractors, how chaos comes about with variation of a system parameter, universality, fractal basin boundaries and their effect on predictability, and applications to physical systems.
University of North Texas Digital Library (University of North Texas)Nonlinear signal processing using neural networks: Prediction and system modelling
375 Citations1987Alan S. Lapedes
It is demonstrated that the backpropagation learning algorithm for neural networks may be used to predict points in a highly chaotic time series with orders of magnitude increase in accuracy over conventional methods including the Linear Predictive Method and the Gabor-Volterra-Weiner Polynomial Method.
PubMedLearning by statistical cooperation of self-interested neuron-like computing elements.
187 Citations1985Andy Barto
It is argued that some of the longstanding problems concerning adaptation and learning by networks might be solvable by this form of cooperativity, and computer simulation experiments are described that show how networks of self-interested components that are sufficiently robust can solve rather difficult learning problems.
International Joint Conference on Artificial IntelligenceSymbols among the neurons: details of a connectionist inference architecture
180 Citations1985David S. Touretzky, Geoffrey E. Minton
The research provides a detailed account of two important symbolic reasoning operations, pattern matching and variable binding, as emergent properties of collections of neuron-like elements in a distributed neural network.
Artificial IntelligencePhase transitions in artificial intelligence systems
156 Citations1987Bernardo A. Huberman, Tad Hogg
It is predicted that large-scale artificial intelligence systems and cognitive models will undergo sudden phase transitions from disjointed parts into coherent structures as their topological connectivity increases beyond a critical value.
Artificial IntelligenceThe ubiquity of discovery
108 Citations1977Douglas B. Lenat
The ultimate goals of this kind of research are to de-mystify the process by which new science and art are created, and to build tools (computer programs) which enhance man's mental capabilities.
Elsevier eBooksA Learning Algorithm for Boltzmann Machines**The research reported here was supported by grants from the System Development Foundation. We thank Peter Brown, Francis Crick, Mark Derthick, Scott Fahlman, Jerry Feldman, Stuart Geman, Gail Gong, John Hopfield, Jay McClelland, Barak Pearlmutter, Harry Printz, Dave Rumelhart, Tim Shallice, Paul Smolensky, Rick Szeliski, and Venkataraman Venkatasubramanian for helpful discussions.Reprint requests should be addressed to David Ackley, Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA 15213.
98 Citations1987David H. Ackley, Geoffrey E. Hinton +1 more
A general parallel search method is described, based on statistical mechanics, and it is shown how it leads to a general learning rule for modifying the connection strengths so as to incorporate knowledge about a task domain in an efficient way.
Indian Journal of Experimental BiologySeveral Studies on Natural Language ·and Back-Propagation
59 Citations1987Robert B. Allen
Two sets of brief studies show how networks may be developed for processing simple demonstratives and analogies and provide additional support for the applicability of neural algorithms to natural language processing.
AIP conference proceedingsContext-free parsing with connectionist networks
44 Citations1986Mark Fanty
The MIT Press eBooksCompetitive Learning: From Interactive Activation to Adaptive Resonance
44 Citations1988Stephen Grossberg
A stochastic approach to parsing
39 Citations1986Geoffrey Sampson
Grammatical parsing -resolving unanalysed linear sequences of words into meaningful grammatical structures -can be regarded as a perception problem logically analogous to those just cited, and simulated annealing holds great promise as a parsing technique.
Biological CyberneticsSynthesis of nonlinear control surfaces by a layered associative search network
36 Citations1982Andrew G. Barto, Charles W. Anderson +1 more
It is argued that this approach to nonlinearity can be extended to a large class of nonlinear control problems.
UR Research (University of Rochester)A connectionist approach to visual recognition
28 Citations1982Daniel Sabbah
A massively parallel, connectionist approach is brought to bear on the problem of visual recognition that eliminates the need for establishing a search order on exploring interpretations and exhibits many similarities with the structure and behavior of animal vision systems.
Sequencing in a connectionist model of language processing
9 Citations1988Michael Gasser, Michael G. Dyer
The thrust of this paper is that sequencing can also be modelled as a process of competition between candidates activated on the basis of input features, and this account allows output ordering to arise out of the interaction of syntactic with semantic and pragmatic factors.