The Meta-Pi network: building distributed knowledge representations for robust multisource pattern recognition
IEEE Transactions on Pattern Analysis and Machine IntelligencePublished 1 July 1992
J.B. Hampshire, Alex Waibel
Citations106
SJR quartileQ1
SJR score3.91
SNIP5.99
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The authors present the Meta-Pi network, a multinetwork connectionist classifier that forms distributed low-level knowledge representations for robust pattern recognition, given random feature vectors generated by multiple statistically distinct sources.
Abstract
The authors present the Meta-Pi network, a multinetwork connectionist classifier that forms distributed low-level knowledge representations for robust pattern recognition, given random feature vectors generated by multiple statistically distinct sources. They illustrate how the Meta-Pi paradigm implements an adaptive Bayesian maximum a posteriori classifier. They also demonstrate its performance in the context of multispeaker phoneme recognition in which the Meta-Pi superstructure combines speaker-dependent time-delay neural network (TDNN) modules to perform multispeaker /b,d,g/ phoneme recognition with speaker-dependent error rates of 2%. Finally, the authors apply the Meta-Pi architecture to a limited source-independent recognition task, illustrating its discrimination of a novel source. They demonstrate that it can adapt to the novel source (speaker), given five adaptation examples of each of the three phonemes.>
Keywords
Computer Science
NatureLearning representations by back-propagating errors
30,885 Citations1986David E. Rumelhart, Geoffrey E. Hinton +1 more
Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Proceedings of the IEEEA tutorial on hidden Markov models and selected applications in speech recognition
22,785 Citations1989L. R. Rabiner
Neural ComputationAdaptive Mixtures of Local Experts
4,988 Citations1991Robert A. Jacobs, Michael I. Jordan +2 more
A new supervised learning procedure for systems composed of many separate networks, each of which learns to handle a subset of the complete set of training cases, which is demonstrated to be able to be solved by a very simple expert network.
The MIT Press eBooksParallel Distributed Processing
3,227 Citations1987James L. McClelland, David E. Rumelhart +1 more
SIAM ReviewMixture Densities, Maximum Likelihood and the EM Algorithm
2,686 Citations1984Richard A. Redner, Homer F. Walker
This work discusses the formulation and theoretical and practical properties of the EM algorithm, a specialization to the mixture density context of a general algorithm used to approximate maximum-likelihood estimates for incomplete data problems.
IEEE Transactions on Acoustics Speech and Signal ProcessingPhoneme recognition using time-delay neural networks
2,625 Citations1989Alexander Waibel, Toshiyuki Hanazawa +3 more
The authors present a time-delay neural network (TDNN) approach to phoneme recognition which is characterized by two important properties: using a three-layer arrangement of simple computing units, a hierarchy can be constructed that allows for the formation of arbitrary nonlinear decision surfaces, which the TDNN learns automatically using error backpropagation.
Neural ComputationLearning in Artificial Neural Networks: A Statistical Perspective
922 Citations1989Halbert White
Concepts and analytical results from the literatures of mathematical statistics, econometrics, systems identification, and optimization theory relevant to the analysis of learning in artificial neural networks are reviewed.
IEEE Transactions on Neural NetworksThe multilayer perceptron as an approximation to a Bayes optimal discriminant function
840 Citations1990D.W. Ruck, Steven K. Rogers +3 more
The multilayer perceptron, when trained as a classifier using backpropagation, is shown to approximate the Bayes optimal discriminant function.
IEEE Communications MagazinePattern classification using neural networks
812 Citations1989Richard P. Lippmann
The author extends a previous review and focuses on feed-forward neural-net classifiers for static patterns with continuous-valued inputs, examining probabilistic, hyperplane, kernel, and exemplar classifiers.
IEEE International Conference on Neural NetworksProbabilistic neural networks for classification, mapping, or associative memory
586 Citations1988Specht
It can be shown that by replacing the sigmoid activation function often used in neural networks with an exponential function, a neural network can be formed which computes nonlinear decision boundaries, which yields decision surfaces which approach the Bayes optimal under certain conditions.
Machine LearningCONNECTIONIST LEARNING PROCEDURES11This chapter appeared in Volume 40 of Artificial Intelligence in 1989, reprinted with permission of North-Holland Publishing. It is a revised version of Technical Report CMU-CS-87-115, which has the same title and was prepared in June 1987 while the author was at Carnegie Mellon University. The research was supported by contract N00014-86-K-00167 from the Office of Naval Research and by grant IST-8520359 from the National Science Foundation.
523 Citations1990Geoffrey E. Hinton
These relatively simple, gradient-descent learning procedures work well for small tasks, and the new challenge is to find ways of improving their convergence rate and their generalization abilities so that they can be applied to larger, more realistic tasks.
TechnometricsEstimation of Parameters for a Mixture of Normal Distributions
474 Citations1966Victor Hasselblad
IEEE Transactions on Acoustics Speech and Signal ProcessingAn overview of the SPHINX speech recognition system
439 Citations1990K.-F. Lee, Hsiao-Wuen Hon +1 more
SPHINX is a system that demonstrates the feasibility of accurate, large-vocabulary, speaker-independent, continuous speech recognition, based on discrete hidden Markov models with LPC- (linear-predictive-coding) derived parameters.
IEEE Transactions on Pattern Analysis and Machine IntelligenceLinks between Markov models and multilayer perceptrons
340 Citations1990H. Bourlard, C. Wellekens
It is shown theoretically and experimentally that the outputs of the MLP approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities.
IEEE Transactions on Neural NetworksNeural network classification: a Bayesian interpretation
274 Citations1990Eric A. Wan
The relationship between minimizing a mean squared error and finding the optimal Bayesian classifier is reviewed and a number of confidence measures are proposed to evaluate the performance of the neural network classifier within a statistical framework.
IEEE Transactions on Neural NetworksA novel objective function for improved phoneme recognition using time-delay neural networks
204 Citations1990J.B. Hampshire, Alexander Waibel
International Conference on Acoustics, Speech, and Signal ProcessingA probabilistic approach to the understanding and training of neural network classifiers
173 Citations2002H. Gish
It is shown that training a neural network using a mean-square-error criterion gives network outputs that approximate posterior class probabilities and information-theoretic training criteria such as maximum mutual information and the Kullback-Liebler measure are investigated.
IEEE International Conference on Neural NetworksNeural network simulation at Warp speed: how we got 17 million connections per second
161 Citations1988Pomerleau, Gusciora +2 more
Results indicate that linear systolic array machines can be efficient neural network simulators and are about eight times faster at simulating the NETtalk text-to-speech network than the fastest back-propagation simulator previously reported in the literature.
IEEE Transactions on Acoustics Speech and Signal ProcessingModularity and scaling in large phonemic neural networks
155 Citations1989Alexander Waibel, Hirozumi Sawai +1 more
The authors train several small time-delay neural networks aimed at all phonemic subcategories and report excellent fine phonemic discrimination performance for all cases and propose several technique that make it possible to grow larger nets in an incremental and modular fashion without loss in recognition performance and without the need for excessive training time or additional data.
TechnometricsEstimation of Parameters for a Mixture of Normal Distributions
146 Citations1966Victor Hasselblad
IEEE Transactions on Pattern Analysis and Machine IntelligenceAutomatic pattern recognition: a study of the probability of error
144 Citations1988Luc Devroye
The Vapnik-Chervonenkis method can be used to choose the smoothing parameter in kernel-based rules, to choose k in the k-nearest neighbor rule, and to choose between parametric and nonparametric rules.
Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures
124 Citations1991Steven J. Nowlan
An unsupervised algorithm which is an alternative to the classical winner-take-all competitive algorithms and a supervised modular architecture in which a number of simple "expert" networks compete to solve distinct pieces of a large task are considered.
Bell System Technical JournalThe Design and Analysis of Pattern Recognition Experiments
113 Citations1962W. H. Highleyman
One important nonparametric result is that the proportion of the total sample used for testing the machine should never be less than that proportion used for designing the machine, and in some cases should be a good deal more.
International Conference on Acoustics, Speech, and Signal ProcessingOn semi-continuous hidden Markov modeling
52 Citations2002Xuedong Huang, K.-F. Lee +1 more
A simple method is investigated, to re-estimate the vector quantization codebook without continuous probability density function assumptions, and preliminary experiments show that such reestimation methods are as effective as the semicontinuous model, especially when the continuous probabilitydensity function assumption is inappropriate.
Task decomposition through competition in a modular connectionist architecture
40 Citations1990Robert A. Jacobs, Andrew G. Barto
IEEE Transactions on Acoustics Speech and Signal ProcessingDynamic speaker adaptation for feature-based isolated word recognition
36 Citations1987Richard M. Stern, Moshé J. Lasry
A set of dynamic adaptation procedures for updating expected feature values during recognition using maximum a posteriori probability (MAP) estimation techniques to update the mean vectors of sets of feature values on a speaker-by-speaker basis.
IEEE Transactions on Neural NetworksA note on least-squares learning procedures and classification by neural network models
28 Citations1991Patrick A. Shoemaker
The author demonstrates this with a straightforward proof in which class probability densities are regarded as primitives and which for simplicity does not rely on probability theory or statistics.
Syllable recognition using integrated neural networks
26 Citations1989Matsuoka, Hamada +1 more
Elsevier eBooksTemporal Backpropagation: An Efficient Algorithm for Finite Impulse Response Neural Networks
25 Citations1991Eric A. Wan
An efficient gradient descent algorithm is derived which will be shown to be a temporal generalization of the familiar backpropagation algorithm.
The use of probabilistic neural networks to improve solution times for hull-to-emitter correlation problems
24 Citations1989Maloney, Specht
The probabilistic neural network (PNN) paradigm has been applied successfully to the hull-to-emitter correlation (HULTEC) problem and tremendous speedup on training was achieved by the PNN implementation compared with the application of backpropagation networks.
Design of hierarchical perceptron structures and their application to the task of isolated-word recognition
16 Citations1989Kammerer, Kupper
Several design strategies for feedforward networks are examined within the scope of pattern classification and a hierarchical structure with pairwise training of two-class models is superior to a single uniform network for speaker-independent word recognition.
The Journal of the Acoustical Society of AmericaContext-modulated discrimination of similar vowels using second-order connectionist networks
11 Citations1989Raymond L. Watrous
Neural Information Processing SystemsApplications of Error Back-Propagation to Phonetic Classification
11 Citations1988Hong Leung, Victor W. Zue
The objective is to investigate the basic characteristics of back-propagation, and study how the framework of multi-layer perceptrons can be exploited in phonetic recognition, and explore issues such as integration of heterogeneous sources of information, conditions that can affect performance of phonetic classification, internal representations, and comparisons with traditional pattern classification techniques.
Task Decomposition Through Competition in a Modular Connectionist
10 Citations1990Robert A. Jacobs, Michael I. Jordan +1 more
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)The meta-generalized delta rule : a new algorithm for learning in connectionist networks
7 Citations2018Dean Pomerleau, Artificial Intelligence And Psychology Project.
A new connectionist architecture and learning algorithm, called the meta-generalized delta rule (MGDR), is an extension of the GDR to provide for learning the proper weights for meta-connections, which are connections from a unit to the connection between two other units.