On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates
The Annals of StatisticsPublished 1 September 1994Open access
Luc Devroye, László Györfi, Adam Krzyżak, Gábor Lugosi
Citations300
SJR quartileQ1
SJR score4.77
SNIP3.13
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that all modes of convergence in L 1 are equivalent if the regression variable is bounded and under the additional condition k/log n → ∞ the strong universal consistency of the estimate is obtained.
Abstract
Two results are presented concerning the consistency of the $k$-nearest neighbor regression estimate. We show that all modes of convergence in $L_1$ (in probability, almost sure, complete) are equivalent if the regression variable is bounded. Under the additional conditional $k/\\log n \\rightarrow \\infty$ we also obtain the strong universal consistency of the estimate.
Keywords
MathematicsEngineering
IEEE Transactions on Information TheoryNearest neighbor pattern classification
16,134 Citations1967Thomas M. Cover, Peter E. Hart
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Springer series in statisticsProbability Inequalities for sums of Bounded Random Variables
6,947 Citations1994Wassily Hoeffding
The Annals of Mathematical StatisticsA Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
3,567 Citations1952Herman Chernoff
Cambridge University Press eBooksOn the method of bounded differences
1,428 Citations1989Colin McDiarmid
Journal of the American Statistical AssociationNonparametric Density Estimation: The L 1 View.
802 Citations1987James R. Thompson, Luc Devroye +1 more
Measure and Integral
781 Citations2015Richard L. Wheeden, Antoni Zygmund +1 more
IEEE Transactions on Information TheoryEstimation by the nearest neighbor rule
358 Citations1968Thomas M. Cover
This paper is an extension of earlier work from the problem of classification by the nearest neighbor rule to that of estimation, where the unbounded loss functions in the estimation problem introduce additional problems concerning the convergence of the unconditional risk.
The Annals of StatisticsDistribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation
227 Citations1980Luc Devroye, T. Wagner
The Annals of StatisticsOn the Almost Everywhere Convergence of Nonparametric Regression Function Estimates
214 Citations1981Luc Devroye
The Annals of StatisticsThe Equivalence of Weak, Strong and Complete Convergence in $L_1$ for Kernel Density Estimates
145 Citations1983Luc Devroye
Theory of Probability and Its ApplicationsRemarks on Non-Parametric Estimates for Density Functions and Regression Curves
129 Citations1970É. A. Nadaraya
The Annals of StatisticsDistribution-Free Pointwise Consistency of Kernel Regression Estimate
116 Citations1984Włodzimierz Greblicki, Adam Krzyżak +1 more
Exponential Inequalities in Nonparametric Estimation
115 Citations1991Luc Devroye
The Annals of StatisticsAsymptotic Normality of Nearest Neighbor Regression Function Estimates
113 Citations1984Winfried Stute
The Annals of StatisticsConsistent Window Estimation in Nonparametric Regression
100 Citations1980Clifford H. Spiegelman, J. Sacks
Journal of Statistical Planning and InferenceAn equivalence theorem for L1 convergence of the kernel regression estimate
100 Citations1989Luc Devroye, Adam KrzyZdotak
IEEE Transactions on Information TheoryThe rates of convergence of kernel regression estimates and classification rules
64 Citations1986Adam Krzyżak
Both nonrecursive and recursive nonparametric regression estimates are studied and the rates of weak and strong convergence of kernel estimates, as well as corresponding multiple classification errors, are derived without assuming the existence of the density of the measurements.
Lecture notes in mathematicsEstimation de la regression par la methode des k points les plus proches avec noyau : quelques propriétés de convergence ponctuelle
53 Citations1980Gérard Collomb
IEEE Transactions on Information TheoryDistribution-free consistency of a nonparametric kernel regression estimate and classification
51 Citations1984Adam Krzyżak, M. Pawlak
It is shown that the kernel estimate of the regression E(Y|X = x) is weakly or strongly consistent for almost all x(\mu) , where \mu is the probability measure of X .
Journal of Multivariate AnalysisExponential bounds of mean error for the nearest neighbor estimates of regression functions
50 Citations1987L.C. Zhao
The Annals of StatisticsWeak Convergence of $k$-NN Density and Regression Estimators with Varying $k$ and Applications
44 Citations1987P. Bhattacharya, Y. P. Mack
Probability Theory and Related FieldsThe kernel estimate is relatively stable
33 Citations1988Luc Devroye
Universal Consistencies of a Regression Estimate for Unbounded Regression Functions
31 Citations1991László Györfi