k-nearest-neighbor Bayes-risk estimation

IEEE Transactions on Information TheoryPublished 1 May 1975

Keinosuke Fukunaga, L.D. Hostetler

DOI10.1109/tit.1975.1055373

Source URL

Citations130

SJR quartileQ1

SJR score1.46

SNIP1.76

TL;DR

Nonparametric estimation of the Bayes risk R^\ast using a k -nearest-neighbor ( k -NN) approach is investigated and the mean-squared error of the conditional Bayes error estimate is reduced significantly.

Abstract

Nonparametric estimation of the Bayes risk R^\ast using a k -nearest-neighbor ( k -NN) approach is investigated. Estimates of the conditional Bayes error r(X) for use in an unclassified test sample approach to estimate R^\ast are derived using maximum-likelihood estimation techniques. By using the volume information as well as the class representations of the k -NN's to X , the mean-squared error of the conditional Bayes error estimate is reduced significantly. Simulations are presented to indicate the performance of the estimates using unclassified testing samples.

Keywords

Mathematics

IEEE Transactions on Information TheoryNearest neighbor pattern classification

16,134 Citations1967Thomas M. Cover, Peter E. Hart

The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.

The Annals of Mathematical StatisticsOn Estimation of a Probability Density Function and Mode

10,504 Citations1962Emanuel Parzen

View PDF

The Annals of Mathematical StatisticsA Nonparametric Estimate of a Multivariate Density Function

767 Citations1965D. O. Loftsgaarden, Charles P. Quesenberry

View PDF

IEEE Transactions on Information TheoryOptimization of k nearest neighbor density estimates

190 Citations1973Keinosuke Fukunaga, L.D. Hostetler

Nonparametric density estimation using the k -nearest-neighbor approach is discussed and a functional form for the optimum k in terms of the sample size, the dimensionality of the observation space, and the underlying probability distribution is obtained.

IEEE Transactions on Information TheoryNonparametric Bayes error estimation using unclassified samples

88 Citations1973Keinosuke Fukunaga, David Kessell

A pattern recognition system may be viewed as a decision rule which transforms measurements into class assignments and the Bayes error is the minimum achievable error, where the minimization is with respect to all decision rules.

Elsevier eBooksLEARNING IN PATTERN RECOGNITION

77 Citations1969Thomas M. Cover

A discussion is made of some nonparametric approaches to the problem of the classification of an unknown pattern when the only information on the underlying distributions associated with the various categories is that which can be obtained from a finite number of samples.

IEEE Transactions on ComputersEstimation of Classification Error

70 Citations1971Keinosuke Fukunaga, David Kessell

Methods of estimating the probability of error for the Bayes' classifier which must be designed and tested with a finite number of classified samples are discussed and the expected difference between estimates is discussed.

IEEE Transactions on Information TheoryNonparametric Bayes-risk estimation

20 Citations1971S. Fralick, Rodney J. Scott

Two nonparametric methods to estimate the Bayes risk using classified sample sets are described and compared and it is shown that both have acceptable small- sample properties; however, small-sample convergence of both estimators depends strongly on the choice of metric and local area or window size in the measurement space.

Journal of the American Statistical AssociationCubical and Spherical Estimation of Multivariate Probability Density

12 Citations1968Thomas A. Elkins

TechnometricsAdaptive Nonparametric Classification

9 Citations1969Chester R. Pelto

It is shown that an optimal single-hypersphere decision process exists, and that the proposed procedure converges stochastically to the optimal process.