Bayesian adaptive lassos with non-convex penalization

The Bayesian interpretation of the Lasso is adopted as the maximum a posteriori (MAP) estimate of the regression coefficients, which have been given independent, double exponential prior distributions, and the properties of this approach are explored.

Abstract

The lasso (Tibshirani,1996) has sparked interest in the use of penalization of
\nthe log-likelihood for variable selection, as well as shrinkage. Recently, there have
\nbeen attempts to propose penalty functions which improve upon the Lassos properties
\nfor variable selection and prediction, such as SCAD (Fan and Li, 2001) and
\nthe Adaptive Lasso (Zou, 2006). We adopt the Bayesian interpretation of the Lasso
\nas the maximum a posteriori (MAP) estimate of the regression coefficients, which
\nhave been given independent, double exponential prior distributions. Generalizing
\nthis prior provides a family of adaptive lasso penalty functions, which includes the quasi-cauchy distribution (Johnstone and Silverman, 2005) as a special case.
\nThe properties of this approach are explored. We are particularly interested in the
\nmore variables than observations case of characteristic importance for data arising
\nin chemometrics, genomics and proteomics - to name but three. Our methodology
\ncan give rise to multiple modes of the posterior distribution and we show how
\nthis may occur even with the convex lasso. These multiple modes do no more
\nthan reflect the indeterminacy of the model. We give fast algorithms and suggest
\na strategy of using a set of perfectly fitting random starting values to explore
\ndifferent regions of the parameter space with substantial posterior support. Simulations
\nshow that our procedure provides significant improvements on a range of
\nestablished procedures and we provide an example from chemometrics.