Minimum complexity estimator for RBF networks architecture selection

Proceedings of International Conference on Neural Networks (ICNN'96)Published 24 December 2002

L. Sardo, Josef Kittler

DOI10.1109/icnn.1996.548880

Citations8

TL;DR

In the present work each unknown density is approximated in terms of a linear combination of radial basis functions (RBFs) using a modified Kullback-Leibler distance as a criterion of optimality.

Abstract

The problem of nonparametric probability density estimation using neural networks methodologies is addressed here. We investigate a criterion that leads to an appropriate choice of the network architecture complexity. In the present work each unknown density is approximated in terms of a linear combination of radial basis functions (RBFs). Both the parameters of the approximating function and the number of RBFs units are estimated using a modified Kullback-Leibler distance as a criterion of optimality. This modification consists of the addition of a term that penalizes complex architectures. Experimental results show the reliability of the methodology.

Keywords

Computer ScienceEngineering

IEEE Transactions on Automatic ControlA new look at the statistical model identification

50,732 Citations1974Hirotugu Akaike

View PDF

AutomaticaModeling by shortest data description

5,944 Citations1978J. Rissanen

The number of digits it takes to write down an observed sequence x1,...,xN of a time series depends on the model with its parameters that one assumes to have generated the observed data.

Neural ComputationApproximation and Radial-Basis-Function Networks

819 Citations1993Jooyoung Park, Irwin W. Sandberg

It has been shown in recent papers that certain classes of radial-basis-function networks are broad enough for universal approximation, and results are considerably extended and sharpened.

IEEE Communications MagazinePattern classification using neural networks

812 Citations1989Richard P. Lippmann

The author extends a previous review and focuses on feed-forward neural-net classifiers for static patterns with continuous-valued inputs, examining probabilistic, hyperplane, kernel, and exemplar classifiers.

IEEE Transactions on Information TheoryMinimum complexity density estimation

497 Citations1991Andrew R. Barron, Thomas M. Cover

An index of resolvability is proved to bound the rate of convergence of minimum complexity density estimators as well as the information-theoretic redundancy of the corresponding total description length to demonstrate the statistical effectiveness of the minimum description-length principle as a method of inference.

IEEE Transactions on Neural NetworksA neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions

192 Citations1991Hans G. C. Tråvén

A method for designing near-optimal nonlinear classifiers, based on a self-organizing technique for estimating probability density functions when only weak assumptions are made about the densities, is described.

IEEE Transactions on Information TheoryApproximating discrete probability distributions

98 Citations1969H. H. Ku, S. Kullback

The method of minimum discrimination information estimation is applied to the problem of estimating an n -dimensional discrete probability distribution in terms of lower order marginal distributions and yields regular best asymptotically normal (RBAN) estimates.

A modeling approach to feature selection

43 Citations2002J. Sheinvald, Byron Dom +1 more

An information-theoretic approach is used to derive a new feature selection criterion capable of detecting features that are totally useless, and is proved to be consistent, i.e., it yields the true useless subset with probability 1 as the size of the training set grows to infinity.

Image and Vision ComputingSimultaneous learning of decision rules and important attributes for classification problems in image analysis

18 Citations1994Pavel Pudil, Jana Novovičová +1 more

A new method is presented to approximate class conditional densities by a mixture of parameterized densities to facilitate simultaneous decision rule inference and selection of discriminative features which characterize the image entities to be classified.

Probability estimation by feed-forward networks in continuous speech recognition

11 Citations2002Steve Renals, N. Morgan +1 more

The authors review the use of feedforward neural networks as estimators of probability densities in hidden Markov modelling and discusses problems with discriminative training, particularly the problem of dealing with unlabelled training data and the mismatch between model and data priors.