Some applications of concentration inequalities to statistics
Annales de la faculté des sciences de Toulouse MathématiquesPublished 1 January 2000Open access
Pascal Massart
Citations239
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
Nous presentons quelques applications d'inegalites de con- centration a la resolution de problemes de selection de modeles en statis- tique.Nous etudions en detail deux exemples pour lesquels cette ap- proche s'avere particulirement fructueuse.Nous considerons tout d'abord le classique mais delicat probleme du choix d'un bon histogramme.Nous presentons un extrait de travail de Castellan sur la question, mettant en evidence que la structure meme des inegalites de concentration de Tala- grand pour des processus empiriques influence directement la construc- tion d'un critere de selection de type Akaike modifie.Nous presentons egalement un nouveau theoreme de selection de modeles bien adapte a la resolution de problemes d'apprentissage.Ce resultat per met de reinterpre- ter et d'ameliorer la methode dite de minimisation structurelle du risque due a Vapnik.ABSTRACT.-The purpose of this paper is to illustrate the power of con- centration inequalities by presenting some striking applications to various model selection problems in statistics.We shall study in details two main examples.We shall consider the old-standing problem of optimal selection of an histogram (following the lines of Castellan's work on this topic) for which the structure of Talagrand's concentration inequalities for empir- ical processes directly influences the construction of a modified Akaike criterion.We shall also present a new model selection theorem which can be applied to improve on Vapnik's structural minimization of the risk method for the statistical learning problem of pattern recognition.
Keywords
MathematicsDecision Sciences
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Springer series in statisticsInformation Theory and an Extension of the Maximum Likelihood Principle
17,886 Citations1998H. Akaike
BiometrikaIdeal spatial adaptation by wavelet shrinkage
7,813 Citations1994David L. Donoho, Iain M. Johnstone
A new principle for spatially-adaptive estimation: selective wavelet reconstruction with an oracle inequality is described and a practical spatially adaptive method, RiskShrink, which works by shrinkage of empirical wavelet coefficients is developed.
Springer series in statisticsProbability Inequalities for sums of Bounded Random Variables
6,947 Citations1994Wassily Hoeffding
Springer series in statisticsWeak Convergence and Empirical Processes
5,037 Citations1996Aad van der Vaart, Jon A. Wellner
This chapter discusses Convergence: Weak, Almost Uniform, and in Probability, which focuses on the part of Convergence of the Donsker Property which is concerned with Uniformity and Metrization.
Springer series in statisticsInformation Theory and an Extension of the Maximum Likelihood Principle
2,999 Citations1992H. Akaike
The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.
BiometrikaIdeal Spatial Adaptation by Wavelet Shrinkage
2,486 Citations1994David L. Donoho, Iain M. Johnstone
Cambridge University Press eBooksOn the method of bounded differences
1,428 Citations1989Colin McDiarmid
Probability in Banach Spaces: Isoperimetry and Processes
1,276 Citations1991Michel Ledoux, Michel Talagrand
Publications mathématiques de l IHÉSConcentration of measure and isoperimetric inequalities in product spaces
862 Citations1995Michel Talagrand
Journal of the American Statistical AssociationProbability Inequalities for the Sum of Independent Random Variables
756 Citations1962G. G. Bennett
Probability Theory and Related FieldsRisk bounds for model selection via penalization
667 Citations1999Andrew R. Barron, Lucien Birgé +1 more
It is shown that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve, which quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size.
Inventiones mathematicaeNew concentration inequalities in product spaces
517 Citations1996Michel Talagrand
Journal of the European Mathematical SocietyGaussian model selection
477 Citations2001Lucien Birgé, Pascal Massart
The purpose in this paper is to provide a general approach to model selection via penalization for Gaussian regression and to develop the point of view about this subject.
Lecture notes in statisticsMinimax Theory of Image Reconstruction
375 Citations1993A. P. Korostelev, A. B. Tsybakov
This book is concerned with a technique for image reconstruction known as the asymptotic minimax approach, which is based on non-parametric regression and change-point analysis, and presents results under the simplest assumptions.
BernoulliMinimum Contrast Estimators on Sieves: Exponential Bounds and Rates of Convergence
344 Citations1998Lucien Birgé, Pascal Massart +1 more
From Model Selection to Adaptive Estimation
335 Citations1997Lucien Birgé, Pascal Massart
Many different model selection information criteria can be found in the literature in various contexts including regression and density estimation to select among a given collection of parametric models that model which minimizes an empirical loss plus some penalty term which is proportional to the dimension of the model.
Journal of Mathematical SciencesExtremal properties of half-spaces for spherically invariant measures
332 Citations1978V. N. Sudakov, Boris Tsirelson
The Annals of ProbabilityAbout the constants in Talagrand's concentration inequalities for empirical processes
284 Citations2000Pascal Massart
Probability Theory and Related FieldsRates of convergence for minimum contrast estimators
221 Citations1993Lucien Birgé, Pascal Massart
The Annals of StatisticsApproximation of Density Functions by Sequences of Exponential Families
219 Citations1991Andrew R. Barron, Chyong-Hwa Sheu
Analogous convergence results for the relative entropy are shown to hold in general, for any class of log-density functions and sequence of finite-dimensional linear spaces having L2 and L.
IEEE Transactions on Information TheoryA simple proof of the blowing-up lemma (Corresp.)
184 Citations1986K. Marton
Random Structures and AlgorithmsA sharp concentration inequality with applications
184 Citations2000Stéphane Boucheron, G�bor Lugosi +1 more
Journal of the American Statistical AssociationProbability Inequalities for the Sum of Independent Random Variables
150 Citations1962George N. Bennett
ESAIM Probability and StatisticsHow many bins should be put in a regular histogram
148 Citations2006Lucien Birgé, Yves Rozenholc
The purpose in this paper is to give a fully automatic and simple method to choose the number of bins of the partition from the data based on a nonasymptotic evaluation of the performances of penalized maximum likelihood estimators in some exponential families due to Castellan and heavy simulations which allowed us to optimize the form of the penalty function.
The Annals of ProbabilityInformation inequalities and concentration of measure
100 Citations1997Amir Dembo
Probability Theory and Related FieldsModel selection for regression on a fixed design
96 Citations2000Yannick Baraud
This work considers some collection of finite dimensional linear spaces and the least-squares estimator built on a data driven selected model among this collection and deduce adaptivity properties from which the estimator from which it is derived holds under mild moment conditions on the errors.
The Annals of ProbabilityA Refinement of the KMT Inequality for the Uniform Empirical Process
55 Citations1987David M. Mason, W. R. van Zwet
ESAIM Probability and StatisticsModel selection for (auto-)regression with dependent data
45 Citations2001Yannick Baraud, Fabienne Comte +1 more
This paper considers a collection of finite dimensional linear spaces and estimates the regression function by a least-squares estimator built on a data driven selected linear space among the collection and shows that it is adaptive in the minimax sense over a large class of Besov balls.
IEEE Transactions on Information TheoryDensity estimation via exponential model selection
25 Citations2003Gwénaëlle Castellan
The rate of convergence of the squared Hellinger risk of the maximum-likelihood estimator is deduced in the regular case when the logarithm of the density belongs to some Besov space.
Springer Link (Chiba Institute of Technology)Model Selection for (Auto-) Regression with Dependent Data
3 Citations1999Yannick Baraud, Fabienne Comte +1 more