Oracle inequalities for multi-fold cross validation
Statistics & DecisionsPublished 1 December 2006Open access
Aad van der Vaart, Sandrine Dudoit, Mark J. van der Laan
Citations158
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results are extended to penalized cross validation in order to control unbounded loss functions and applications include regression with squared and absolute deviation loss and classification under Tsybakov’s condition.
Abstract
We consider choosing an estimator or model from a given class by cross validation consisting of holding a nonneglible fraction of the observations out as a test set. We derive bounds that show that the risk of the resulting procedure is (up to a constant) smaller than the risk of an oracle plus an error which typically grows logarithmically with the number of estimators in the class. We extend the results to penalized cross validation in order to control unbounded loss functions. Applications include regression with squared and absolute deviation loss and classification under Tsybakov’s condition.
Keywords
MathematicsEngineering
IEEE Transactions on Automatic ControlA new look at the statistical model identification
50,732 Citations1974Hirotugu Akaike
Journal of the Royal Statistical Society Series B (Statistical Methodology)Cross-Validatory Choice and Assessment of Statistical Predictions
10,527 Citations1974M. Stone
Probability Theory and Related FieldsRisk bounds for model selection via penalization
667 Citations1999Andrew R. Barron, Lucien Birgé +1 more
It is shown that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve, which quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size.
The Annals of StatisticsOptimal aggregation of classifiers in statistical learning
577 Citations2004A. B. Tsybakov
The main result of the paper concerns optimal aggregation of classifiers: a classifier that automatically adapts both to the complexity and to the margin, and attains the optimal fast rates, up to a logarithmic factor.
The Annals of StatisticsModel Selection Via Multifold Cross Validation
535 Citations1993Ping Zhang
Two notions of multi-fold cross validation (MCV and MCV*) criteria are considered and it turns out that MCV indeed reduces the chance of overfitting.
The Annals of StatisticsAsymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set
521 Citations1987Ker-Chau Li
IEEE Transactions on Information TheoryMinimum complexity density estimation
497 Citations1991Andrew R. Barron, Thomas M. Cover
An index of resolvability is proved to bound the rate of convergence of minimum complexity density estimators as well as the information-theoretic redundancy of the corresponding total description length to demonstrate the statistical effectiveness of the minimum description-length principle as a method of inference.
Journal of the American Statistical AssociationThe Estimation of Prediction Error
458 Citations2004Bradley Efron
A Rao–Blackwell type of relation is derived in which nonparametric methods such as cross-validation are seen to be randomized versions of their covariance penalty counterparts.
ESAIM Probability and StatisticsTheory of Classification: a Survey of Some Recent Advances
416 Citations2005Stéphane Boucheron, Olivier Bousquet +1 more
The Annals of StatisticsSmooth discrimination analysis
385 Citations1999Enno Mammen, Alexandre B. Tsybakov
IEEE Transactions on Information TheoryRademacher penalties and structural risk minimization
371 Citations2001Vladimir Koltchinskii
This work suggests a penalty function to be used in various problems of structural risk minimization, based on the sup-norm of the so-called Rademacher process indexed by the underlying class of functions (sets), and obtains oracle inequalities for the theoretical risk of estimators, obtained by structural minimization of the empirical risk withRademacher penalties.
Journal of the American Statistical AssociationThe Variable Selection Problem
336 Citations2000Edward I. George
This vignette reviews some of the key developments that have led to the wide variety of approaches for the problem of subset selection in statistical applications.
Annales de la faculté des sciences de Toulouse MathématiquesSome applications of concentration inequalities to statistics
239 Citations2000Pascal Massart
Statistical MethodologyAsymptotics of cross-validated risk estimation in estimator selection and performance assessment
168 Citations2005Sandrine Dudoit, Mark J. van der Laan
A general framework for cross-validation is introduced and distributional properties ofCross-validated risk estimators in the context of estimator selection and performance assessment are derived.
Journal of EconometricsAsymptotic optimality of generalized C, cross-validation, and generalized cross-validation in regression with heteroskedastic errors
131 Citations1991Donald W. K. Andrews
The Annals of StatisticsOn prediction of individual sequences
97 Citations1999Nicolò Cesa‐Bianchi, Gábor Lugosi
Journal of the American Statistical AssociationThe Variable Selection Problem
55 Citations2000Edward I. George
Statistical MethodologyCross validation model selection criteria for linear regression based on the Kullback–Leibler discrepancy
20 Citations2005Simon Davies, Andrew A. Neath +1 more
An adjusted cross validation model selection criterion (PDCa) is developed which is asymptotically a minimum variance unbiased estimator of the Kullback–Leibler discrepancy which serves as the basis for the Akaike information criteria AIC and AICc.