High-dimensional generalized linear models and the lasso
The Annals of StatisticsPublished 25 March 2008Open access
Sara A. van de Geer
Citations396
SJR quartileQ1
SJR score4.77
SNIP3.13
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A nonasymptotic oracle inequality is proved for the empirical risk minimizer with Lasso penalty for high-dimensional generalized linear models with Lipschitz loss functions, and the penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm.
Abstract
We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.
Keywords
Computer ScienceMathematics
Springer series in statisticsThe Elements of Statistical Learning
24,344 Citations2001Trevor Hastie, J. Friedman +1 more
IEEE Transactions on Information TheoryDe-noising by soft-thresholding
9,411 Citations1995David L. Donoho
The authors prove two results about this type of estimator that are unprecedented in several ways: with high probability f/spl circ/*/sub n/ is at least as smooth as f, in any of a wide variety of smoothness measures.
Communications on Pure and Applied MathematicsFor most large underdetermined systems of linear equations the minimal đ<sub>1</sub>ânorm solution is also the sparsest solution
2,541 Citations2006David L. Donoho
The techniques include the use of random proportional embeddings and almostâspherical sections in Banach space theory, and deviation bounds for the eigenvalues of random Wishart matrices.
The Annals of StatisticsHigh-dimensional graphs and variable selection with the Lasso
2,406 Citations2006Nicolai Meinshausen, Peter BĂŒhlmann
It is shown that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs and is hence equivalent to variable selection for Gaussian linear models.
Journal of the Royal Statistical Society Series B (Statistical Methodology)The Group Lasso for Logistic Regression
1,720 Citations2008Lukas Meier, Sara van de Geer +1 more
An efficient algorithm is presented, that is especially suitable for high dimensional problems, which can also be applied to generalized linear models to solve the corresponding convex optimization problem.
Communications on Pure and Applied MathematicsFor most large underdetermined systems of equations, the minimal đ<sub>1</sub>ânorm nearâsolution approximates the sparsest nearâsolution
645 Citations2006David L. Donoho
It is shown that for most Ί, if the optimally sparse approximation x0,Ï” is sufficiently sparse, then the solution x1, Ï” of the đ1âminimization problem is a good approximation to x0 ,Ï”.
The Annals of StatisticsOptimal aggregation of classifiers in statistical learning
577 Citations2004A. B. Tsybakov
The main result of the paper concerns optimal aggregation of classifiers: a classifier that automatically adapts both to the complexity and to the margin, and attains the optimal fast rates, up to a logarithmic factor.
The Annals of StatisticsLasso-type recovery of sparse representations for high-dimensional data
479 Citations2009Nicolai Meinshausen, Bin Yu
Even though the Lasso cannot recover the correct sparsity pattern, the estimator is still consistent in the â2-norm sense for fixed designs under conditions on (a) the number sn of non-zero components of the vector n and (b) the minimal singular values of the design matrices that are induced by selecting of order sn variables.
Computational Statistics & Data AnalysisRelaxed Lasso
462 Citations2006Nicolai Meinshausen
It is shown that the contradicting demands of an efficient computational procedure and fast convergence rates of the `2-loss can be overcome by a two-stage procedure, termed the relaxed Lasso.
The Annals of ProbabilityAbout the constants in Talagrand's concentration inequalities for empirical processes
284 Citations2000Pascal Massart
Comptes Rendus MathématiqueA Bennett concentration inequality and its application to suprema of empirical processes
277 Citations2002Olivier Bousquet
ESAIM Probability and StatisticsOn Talagrand's deviation inequalities for product measures
261 Citations1997Michel Ledoux
Electronic Journal of StatisticsSparsity oracle inequalities for the Lasso
240 Citations2007Florentina Bunea, Alexandre Tsybakov +1 more
It is shown that the penalized least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in terms of the number of non-zero components of the oracle vector, in nonparametric regression setting with random design.
Annales de la faculté des sciences de Toulouse MathématiquesSome applications of concentration inequalities to statistics
239 Citations2000Pascal Massart
Lecture notes in computer scienceAggregation and Sparsity Via â1 Penalized Least Squares
75 Citations2006Florentina Bunea, Alexandre B. Tsybakov +1 more
The Annals of StatisticsBest subset selection, persistence in high-dimensional statistical learning and optimization under l1 constraint
67 Citations2006Eitan Greenshtein
A high-dimensional simulation study of a "boosting type" classification procedure based on empirical risk minimization and shows that the best subset among those of size which is of order o(n/log(n).
BernoulliClassifiers of support vector machine type with \ell1 complexity regularization
45 Citations2006Bernadetta Tarigan, Sara A. van de Geer
Under certain conditions on the base functions, hinge loss with this complexity penalty is shown to lead to an oracle inequality involving both model complexity and margin.
BMC BioinformaticsPenalized likelihood for sparse contingency tables with an application to full-length cDNA libraries
37 Citations2007Corinne Dahinden, Giovanni Parmigiani +2 more
Methods to perform model selection and parameter estimation in log-linear models for the analysis of sparse contingency tables, to study the interaction of two or more factors and propose regularization methods that can be used successfully to detect complex interaction patterns among categorical variables in a broad range of biological problems involving categorical variable.