Estimation for High-Dimensional Linear Mixed-Effects Models Using ℓ1-Penalization
Published 1 January 2010
Jürg Schelldorfer
Citations148
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
We propose an $\ell_1$-penalized estimation procedure for high-dimensional linear mixed-effects models. The models are useful whenever there is a grouping structure among high-dimensional observations, i.e. for clustered data. We prove a consistency and an oracle optimality result and we develop an algorithm with provable numerical convergence. Furthermore, we demonstrate the performance of the method on simulated and a real high-dimensional data set.
Keywords
Mathematics
Journal of the Royal Statistical Society Series B (Statistical Methodology)Regression Shrinkage and Selection Via the Lasso
51,790 Citations1996Robert Tibshirani
A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Journal of Statistical SoftwareRegularization Paths for Generalized Linear Models via Coordinate Descent
17,125 Citations2010Jerome H. Friedman, Trevor Hastie +1 more
In comparative timings, the new algorithms are considerably faster than competing methods and can handle large problems and can also deal efficiently with sparse features.
PubMedRegularization Paths for Generalized Linear Models via Coordinate Descent.
13,977 Citations2010Jerome H. Friedman, Trevor Hastie +1 more
The Annals of StatisticsLeast angle regression
9,493 Citations2004Bradley Efron, Trevor Hastie +2 more
TechnometricsMixed-Effects Models in S and S-PLUS
9,355 Citations2001Josae C. Pinheiro, Douglas M. Bates +3 more
This paper presents a meta-modelling framework for fitting and extending the Basic LME Model to nonlinear Mixed-Effects models and describes the structure of Grouped Data.
BiometricsRandom-Effects Models for Longitudinal Data
8,915 Citations1982Nan M. Laird, James H. Ware
A unified approach to fitting these models, based on a combination of empirical Bayes and maximum likelihood estimation of model parameters and using the EM algorithm, is discussed.
Journal of the American Statistical AssociationThe Adaptive Lasso and Its Oracle Properties
7,633 Citations2006Hui Zou
A new version of the lasso is proposed, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the ℓ1 penalty, and the nonnegative garotte is shown to be consistent for variable selection.
Lecture notes in statisticsLinear Mixed Models for Longitudinal Data
3,525 Citations1997Geert Verbeke
Springer series in statisticsLinear Mixed Models for Longitudinal Data
2,641 Citations2000Geert Molenberghs, Geert Verbeke
Using data of 955 men, Brant et al showed that the average rates of increase of systolic blood pressure (SBP) are smallest in the younger age groups, and greatest in the older agegroups, and that obese individuals tend to have a higher SBP than non-obese individuals.
The Annals of StatisticsSimultaneous analysis of Lasso and Dantzig selector
2,536 Citations2009Peter J. Bickel, Ya’acov Ritov +1 more
It is shown that, under a sparsity scenario, the Lasso estimator and the Dantzig selector exhibit similar behavior and derive, in parallel, oracle inequalities for the prediction risk in the general nonparametric regression model as well as bounds on the l p estimation loss in the linear model.
The Annals of StatisticsHigh-dimensional graphs and variable selection with the Lasso
2,406 Citations2006Nicolai Meinshausen, Peter Bühlmann
It is shown that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs and is hence equivalent to variable selection for Gaussian linear models.
Wiley series in probability and statisticsGeneralized, Linear, and Mixed Models
2,368 Citations2000Charles E. McCulloch, S. R. Searle
Journal of the Royal Statistical Society Series B (Statistical Methodology)Stability Selection
2,135 Citations2010Nicolai Meinshausen, Peter Bühlmann
It is proved for the randomized lasso that stability selection will be variable selection consistent even if the necessary conditions for consistency of the original lasso method are violated.
On Model Selection Consistency of Lasso
2,015 Citations2006Peng Zhao, Bin Yu
It is proved that a single condition, which is called the Irrepresentable Condition, is almost necessary and sufficient for Lasso to select the true model both in the classical fixed p setting and in the large p setting as the sample size n gets large.
arXiv (Cornell University)The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
1,725 Citations2005Candes, Emmanuel, Tao, Terence
Is it possible to estimate β reliably based on the noisy data y?
Journal of the Royal Statistical Society Series B (Statistical Methodology)The Group Lasso for Logistic Regression
1,720 Citations2008Lukas Meier, Sara van de Geer +1 more
An efficient algorithm is presented, that is especially suitable for high dimensional problems, which can also be applied to generalized linear models to solve the corresponding convex optimization problem.
Springer series in statisticsStatistics for High-Dimensional Data
1,620 Citations2011Peter Bühlmann, Sara van de Geer
Statistics for High-Dimensional Data: Methods, Theory and Applications
1,560 Citations2011Peter Bhlmann, Sara van de Geer
Journal of Machine Learning ResearchOn Model Selection Consistency of Lasso
1,010 Citations2006ZhaoPeng, Yubin Yubin
The Annals of StatisticsOn the “degrees of freedom” of the lasso
968 Citations2007Hui Zou, Trevor Hastie +1 more
The number of nonzero coefficients is an unbiased estimate for the degrees of freedom of the lasso—a conclusion that requires no special assumption on the predictors and the unbiased estimator is shown to be asymptotically consistent.
IMA Journal of Numerical AnalysisA new approach to variable selection in least squares problems
847 Citations2000M. R. Osborne
A compact descent method for solving the constrained problem for a particular value of κ is formulated, and a homotopy method, in which the constraint bound κ becomes the Homotopy parameter, is developed to completely describe the possible selection regimes.
Mathematical ProgrammingA coordinate gradient descent method for nonsmooth separable minimization
784 Citations2007Paul Tseng, Sangwoon Yun
A (block) coordinate gradient descent method for solving this class of nonsmooth separable problems and establishes global convergence and, under a local Lipschitzian error bound assumption, linear convergence for this method.
Coordinate descent algorithms for lasso penalized regression
776 Citations2008Tong Wu, Kenneth Lange
UC BerkeleyLasso-type recovery of sparse representations for high-dimensional data
751 Citations2006Meinshausen, Nicolai, Yu, Bin
Electronic Journal of StatisticsOn the conditions used to prove oracle results for the Lasso
707 Citations2009Sara A. van de Geer, Peter Bühlmann
HIGH DIMENSIONAL VARIABLE SELECTION
554 Citations2009Wasserman, Larry, Roeder, Kathryn
Statistica SinicaAdaptive Lasso for sparse high-dimensional regression models
517 Citations2008Jian Huang, Shuangge Ma +1 more
The adaptive Lasso has the oracle property even when the number of covariates is much larger than the sample size, and under a partial orthogonality condition in which the covariates with zero coefficients are weakly correlated with the covariate with nonzero coefficients, marginal regression can be used to obtain the initial estimator.
Journal of the American Statistical Association<i>p</i>-Values for High-Dimensional Regression
446 Citations2009Nicolai Meinshausen, Lukas Meier +1 more
Inference across multiple random splits can be aggregated while maintaining asymptotic control over the inclusion of noise variables, and it is shown that the resulting p-values can be used for control of both family-wise error and false discovery rate.
Sparsity oracle inequalities for the Lasso
421 Citations2012Florentina Bunea, Alexandre B. Tsybakov +1 more
BernoulliPersistence in high-dimensional linear predictor selection and the virtue of overparametrization
359 Citations2004Eitan Greenshtein, Ya’acov Ritov
Under various sparsity assumptions on the optimal predictor there is “asymptotically no harm” in introducing many more explanatory variables than observations, and such practice can be beneficial in comparison with a procedure that screens in advance a small subset of explanatory variables.
Computational Statistics & Data AnalysisA new chi-square approximation to the distribution of non-negative definite quadratic forms in non-central normal variables
242 Citations2008Huan Liu, Yongqiang Tang +1 more
BiometricsJoint Variable Selection for Fixed and Random Effects in Linear Mixed‐Effects Models
230 Citations2010Howard D. Bondell, Arun Krishna +1 more
This method is based on a penalized joint log likelihood with an adaptive penalty for the selection and estimation of both the fixed and random effects and enjoys the Oracle property, in that, asymptotically it performs as well as if the true model was known beforehand.
Testℓ1-penalization for mixture regression models
194 Citations2010Nicolas Städler, Peter Bühlmann +1 more
This work considers a finite mixture of regressions model for high-dimensional inhomogeneous data where the number of covariates may be much larger than sample size and proposes an ℓ1-penalized maximum likelihood estimator in an appropriate parameterization.
ℓ1-Penalization for Mixture Regression Models
180 Citations2016Nicolas Städler
BiometricsFixed and Random Effects Selection in Mixed Effects Models
153 Citations2010Joseph G. Ibrahim, Hongtu Zhu +2 more
This work considers selecting both fixed and random effects in a general class of mixed effects models using maximum penalized likelihood (MPL) estimation along with the smoothly clipped absolute deviation (SCAD) and adaptive least absolute shrinkage and selection operator (ALASSO) penalty functions.
Electronic Journal of StatisticsThe adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso)
108 Citations2011Sara van de Geer, Peter Bühlmann +1 more
It is shown that for exact variable selection, the adaptive Lasso generally needs more severe beta-min conditions than thresholding, and a lower bound is provided for the Lasso with respect to false positive selections.
BiometricsVariable Selection for Semiparametric Mixed Models in Longitudinal Studies
61 Citations2009Xiao Ni, Daowen Zhang +1 more
A double-penalized likelihood approach for simultaneous model selection and estimation in semiparametric mixed models for longitudinal data, which provides valid inference for data with missing at random and will be more efficient if the specified model is correct.
arXiv (Cornell University)P-values for high-dimensional regression
37 Citations2008Nicolai Meinshausen, Lukas Meier +1 more
arXiv (Cornell University)The adaptive and the thresholded Lasso for potentially misspecified models
11 Citations2010Sara van de Geer, Peter Bühlmann +1 more