Adaptive Mixtures of Local Experts
Neural ComputationPublished 1 February 1991
Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, Geoffrey E. Hinton
Citations4,988
SJR quartileQ1
SJR score0.83
SNIP1.45
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new supervised learning procedure for systems composed of many separate networks, each of which learns to handle a subset of the complete set of training cases, which is demonstrated to be able to be solved by a very simple expert network.
Abstract
We present a new supervised learning procedure for systems composed of many separate networks, each of which learns to handle a subset of the complete set of training cases. The new procedure can be viewed either as a modular version of a multilayer supervised network, or as an associative version of competitive learning. It therefore provides a new link between these two apparently different approaches. We demonstrate that the learning procedure divides up a vowel discrimination task into appropriate subtasks, each of which can be solved by a very simple expert network.
Keywords
Computer Science
Neural ComputationFast Learning in Networks of Locally-Tuned Processing Units
4,219 Citations1989John Moody, Christian J. Darken
This work proposes a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988).
The Journal of the Acoustical Society of AmericaControl Methods Used in a Study of the Vowels
2,783 Citations1952Gordon E. Peterson, Harold L. Barney
Control methods used in the evaluation of effects of language and dialectal backgrounds and vocal and auditory characteristics of the individuals concerned in a vowel study program at Bell Telephone Laboratories are discussed.
Queensland's institutional digital repository (The University of Queensland)Mixture models : inference and applications to clustering
2,074 Citations1988Geoffrey J. McLachlan, K. E. Basford
Neural ComputationLocal Learning Algorithms
544 Citations1992Léon Bottou, Vladimir Vapnik
A single analysis suggests that neither kNN or RBF, nor nonlocal classifiers, achieve the best compromise between locality and capacity.
Cognitive ScienceTask Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks
497 Citations1991Robert A. Jacobs, Michael I. Jordan +1 more
A novel modular connectionist architecture is presented in which the networks composing the architecture compete to learn the training patterns, and an outcome of the competition is that different networks learn different training patterns and, thus, learn to compute different functions.
PubMedLearning by statistical cooperation of self-interested neuron-like computing elements.
187 Citations1985Andy Barto
It is argued that some of the longstanding problems concerning adaptation and learning by networks might be solvable by this form of cooperativity, and computer simulation experiments are described that show how networks of self-interested components that are sufficiently robust can solve rather difficult learning problems.
Maximum Likelihood Competitive Learning
138 Citations1989Steven J. Nowlan
This work proposes to view competitive adaptation as attempting to fit a blend of simple probability generators to a set of data-points, and investigates one application of the soft competitive model, placement of radial basis function centers for function interpolation, and shows that the soft model can give better performance with little additional computational cost.
Journal of Cognitive NeuroscienceComputational Consequences of a Bias toward Short Connections
128 Citations1992Robert A. Jacobs, Michael I. Jordan
This paper presents simulations that show that systems that learn under the influence of an architectural bias have a number of desirable properties, including a tendency to decompose tasks into subtasks, to decouple the dynamics of recurrent subsystems, and to develop location-sensitive internal representations.
IEEE Transactions on Pattern Analysis and Machine IntelligenceThe Meta-Pi network: building distributed knowledge representations for robust multisource pattern recognition
106 Citations1992J.B. Hampshire, Alex Waibel
The authors present the Meta-Pi network, a multinetwork connectionist classifier that forms distributed low-level knowledge representations for robust pattern recognition, given random feature vectors generated by multiple statistically distinct sources.
Neural ComputationA Tree-Structured Algorithm for Reducing Computation in Networks with Separable Basis Functions
43 Citations1991Terence D. Sanger
A new algorithm for approximating continuous functions in high-dimensional input spaces and an example that predicts future values of the Mackey-Glass differential delay equation are presented.
IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews)On the Use of Different Speech Representations for Speaker Modeling
24 Citations2005Ke Chen
This work proposes an alternative approach to speaker modeling by the simultaneous use of different speech representations in an optimal way, and presents a parametric statistical model, generalized Gaussian mixture model (GGMM), to characterize a speaker identity based on different speech representation.
A Modular Connectionist Architecture For Learning Piecewise Control Strategies
22 Citations1991Robert A. Jacobs, Michael I. Jordan
This work describes a multi-network, or modular, connectionist architecture that learns to perform control tasks using a piecewise control strategy, where the architecture's networks compete to learn the training patterns.
Learning explicit rules in a neural network
7 Citations2002Clayton McMillan, Michael C. Mozer +1 more
The authors propose an architecture called RuleNet, which, based on knowledge of the task domain, allows for the extraction of symbolic condition-action rules from the connection strengths in a neural net by exploiting constraints inherent in the domain of symbolic string-to-string mappings.