Maximum Likelihood Competitive Learning
Published 1 January 1989
Steven J. Nowlan
Citations138
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work proposes to view competitive adaptation as attempting to fit a blend of simple probability generators to a set of data-points, and investigates one application of the soft competitive model, placement of radial basis function centers for function interpolation, and shows that the soft model can give better performance with little additional computational cost.
Abstract
One popular class of unsupervised algorithms are competitive algo-rithms. In the traditional view of competition, only one competitor, the winner, adapts for any given case. I propose to view compet-itive adaptation as attempting to fit a blend of simple probability generators (such as gaussians) to a set of data-points. The maxi-mum likelihood fit of a model of this type suggests a "softer " form of competition, in which all competitors adapt in proportion to the relative probability that the input came from each competitor. I investigate one application of the soft competitive model, place-ment of radial basis function centers for function interpolation, and show that the soft model can give better performance with little additional computational cost. 1
Keywords
Computer Science
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
Biological CyberneticsSelf-organized formation of topologically correct feature maps
9,519 Citations1982Teuvo Kohonen
In a simple network of adaptive physical elements which receives signals from a primary event space, the signal representations are automatically mapped onto a set of output responses in such a way that the responses acquire the same topological order as that of the primary events.
Complex SystemsRadial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks
3,439 Citations1988David S. Broomhead, David Lowe
The relationship between 'learning' in adaptive layered networks and the fitting of data with high dimensional surfaces is discussed, leading naturally to a picture of 'generalization in terms of interpolation between known data points and suggests a rational approach to the theory of such networks.
The Journal of the Acoustical Society of AmericaControl Methods Used in a Study of the Vowels
2,783 Citations1952Gordon E. Peterson, Harold L. Barney
Control methods used in the evaluation of effects of language and dialectal backgrounds and vocal and auditory characteristics of the individuals concerned in a vowel study program at Bell Telephone Laboratories are discussed.
Queensland's institutional digital repository (The University of Queensland)Mixture models : inference and applications to clustering
2,074 Citations1988Geoffrey J. McLachlan, K. E. Basford
ComputerSelf-organization in a perceptual network
1,514 Citations1988Ralph Linsker
It is shown that even a single developing cell of a layered network exhibits a remarkable set of optimization properties that are closely related to issues in statistics, theoretical physics, adaptive signal processing, the formation of knowledge representation in artificial intelligence, and information theory.
NeurocomputingProbabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition
1,202 Citations1990John S. Bridle
Two modifications are explained: probability scoring, which is an alternative to squared error minimisation, and a normalised exponential (softmax) multi-input generalisation of the logistic non- linearity of feed-forward non-linear networks with multiple outputs.
Cognitive ScienceFeature discovery by competitive learning
890 Citations1985David E. Rumelhart, David Zipser
Neural ComputationLayered Neural Networks with Gaussian Hidden Units as Universal Approximations
790 Citations1990Eric Hartman, James D. Keeler +1 more
A neural network with a single layer of hidden units of gaussian type is proved to be a universal approximator for real-valued maps defined on convex, compact sets of Rn.
Biological CyberneticsCognitron: A self-organizing multilayered neural network
727 Citations1975Kunihiko Fukushima
A new hypothesis for the organization of synapses between neurons is proposed: “The synapse from neuron x to neuron y is reinforced when x fires provided that no neuron in the vicinity of y is firing stronger than y”, and a new algorithm with which a multilayered neural network is effectively organized can be deduced.
A Theory of Networks for Approximation and Learning
716 Citations1989Tomaso Poggio, Federico Girosi
Proceedings of the National Academy of SciencesFrom basic network principles to neural architecture: emergence of spatial-opponent cells.
396 Citations1986R Linsker
This paper is the first of three that address the origin and organization of feature-analyzing cells in simple systems governed by biologically plausible development rules, and introduces the theory of "modular self-adaptive networks," of which this system is an example, and explicitly demonstrates the emergence of a layer of spatial-opponent cells.
Intellectica Revue de l Association pour la Recherche CognitiveModèles connexionnistes de l'apprentissage
182 Citations1987Y. Le Cun, Françoise Fogelman‐Soulié
Fast Learning in Multi-Resolution Hierarchies
147 Citations1988John Moody
A class of fast, supervised learning algorithms inspired by Albus's CMAC model that use local representations, hashing, and multiple scales of resolution to approximate functions which are piece-wise continuous are presented.
Neural Information Processing SystemsNeural Net and Traditional Classifiers
139 Citations1987William Y. C. Huang, Richard P. Lippmann
It is demonstrated that two-layer perceptron classifiers trained with back propagation can form both convex and disjoint decision regions.
An Optimality Principle for Unsupervised Learning
102 Citations1988Terence D. Sanger
An optimality principle is proposed for training an unsupervised feedforward neural network based upon maximal ability to reconstruct the input data from the network outputs and an algorithm which can be used to train either linear or nonlinear networks with certain types of nonlinearity.
Computer Speech & LanguageNeural networks and radial basis functions in classifying static speech patterns
100 Citations1990Mahesan Niranjan, F. Fallside
A review of several classification techniques similar to the method of radial basis functions is presented, and the class boundaries generated by the different methods are compared on simple two-dimensional examples.
IEEE International Conference on Neural NetworksMultilayer feedforward potential function network
56 Citations1988Sukhan Lee, Kil
The authors present a multilayer feedforward network, called the Gaussian potential function network (GPFN), performing association or classification based on a set of potentially fields synthesized over the domain of input space by a number of GaRussian potential function units (GPFUs).