Gaussian Mixture Models and Probabilistic Decision-Based Neural Networks for Pattern Classification: A Comparative Study
Neural Computing and ApplicationsPublished 1 August 1999
Ka Fai Cedric Yiu, Man‐Wai Mak, C. K. Li
Citations23
SJR quartileQ1
SJR score1.10
SNIP1.61
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data.
Abstract
Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
Neural NetworksProbabilistic neural networks
3,738 Citations1990Donald F. Specht
A probabilistic neural network that can compute nonlinear decision boundaries which approach the Bayes optimal is formed, and a fourlayer neural network of the type proposed can map any input pattern to any number of classifications.
Queensland's institutional digital repository (The University of Queensland)Mixture models : inference and applications to clustering
2,074 Citations1988Geoffrey J. McLachlan, K. E. Basford
Neural ComputationNeural Network Classifiers Estimate Bayesian <i>a posteriori</i> Probabilities
993 Citations1991Michael D. Richard, Richard P. Lippmann
Results of Monte Carlo simulations performed using multilayer perceptron (MLP) networks trained with backpropagation, radial basis function (RBF) networks, and high-order polynomial networks graphically demonstrate that network outputs provide good estimates of Bayesian probabilities.
IEEE Transactions on Neural NetworksThe multilayer perceptron as an approximation to a Bayes optimal discriminant function
840 Citations1990D.W. Ruck, Steven K. Rogers +3 more
The multilayer perceptron, when trained as a classifier using backpropagation, is shown to approximate the Bayes optimal discriminant function.
IEEE Communications MagazinePattern classification using neural networks
812 Citations1989Richard P. Lippmann
The author extends a previous review and focuses on feed-forward neural-net classifiers for static patterns with continuous-valued inputs, examining probabilistic, hyperplane, kernel, and exemplar classifiers.
IEEE Transactions on Neural NetworksFace recognition/detection by probabilistic decision-based neural network
578 Citations1997Shang‐Hung Lin, Sun‐Yuan Kung +1 more
The paper demonstrates a successful application of PDBNN to face recognition applications on two public (FERET and ORL) and one in-house (SCR) databases and experimental results on three different databases such as recognition accuracies as well as false rejection and false acceptance rates are elaborated.
Control Engineering PracticeDigital neural networks
478 Citations1994
This chapter discusses the development of unsupervised networks for association/Classification and their applications in neural nets, as well as some of the techniques used in conventional optimization techniques.
IEEE Transactions on Neural NetworksGrowing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques
306 Citations1997N.B. Karayiannis, Guihua Mi
The proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of GRBF networks, which include unsupervised algorithms for clustering and learning vector quantization, as well as learning algorithms for training single-layer linear neural networks.
IEEE Transactions on Neural NetworksMaximum likelihood training of probabilistic neural networks
207 Citations1994Roy L. Streit, Tod Luginbuhl
A maximum likelihood method is presented for training probabilistic neural networks (PNN's) using a Gaussian kernel, or Parzen window, and is a generalization of Fisher's method for linear discrimination.
IEEE Transactions on Neural NetworksA neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions
192 Citations1991Hans G. C. Tråvén
A method for designing near-optimal nonlinear classifiers, based on a self-organizing technique for estimating probability density functions when only weak assumptions are made about the densities, is described.
Neural ComputationA Probabilistic Resource Allocating Network for Novelty Detection
186 Citations1994Stephen Roberts, Lionel Tarassenko
A robust method for novelty detection is developed, which aims to minimize the number of heuristically chosen thresholds in the novelty decision process by growing a gaussian mixture model to form a representation of a training set of normal system states.
IEEE Transactions on Neural NetworksDecision-based neural networks with signal/image classification applications
164 Citations1995Sun‐Yuan Kung, J.S. Taur
A decision-based neural network is proposed, which combines the perceptron-like learning rule and hierarchical nonlinear network structure, which is confirmed by simulations conducted for several applications, including texture classification, OCR, and ECG analysis.
Neural Computing and ApplicationsProbability density estimation using a Gaussian clustering algorithm
11 Citations1996J. Ćwik, Jacek Koronacki
A version of the Tråvén's Gaussian clustering algorithm for normal mixture densities is studied and simulations suggest that the modified algorithm is a very promising method of estimating arbitrary continuous d-dimensional densities.
Maximum likelihood estimation of elliptical basis function parameters with application to speaker verification
4 Citations2002Man‐Wai Mak, Chengxiao Li +1 more
Experimental results show that smaller size EBF networks with basis function parameters determined by the EM algorithm outperform the large RBF networks trained in the conventional approach.