A variational Bayesian framework for group feature selection

This work presents a novel hierarchical Bayesian formulation of a generalized linear model and estimates the posterior distribution of the parameters and hyper-parameters of the model within a completely Bayesian paradigm based on variational inference.

Abstract

In many machine learning and pattern analysis applications, grouping of features during model development and the selection of a small number of relevant groups can be useful to improve the interpretability of the learned parameters. Although this problem has been receiving a significant amount of attention lately, most of the approaches require the manual tuning of one or more hyper-parameters. In order to overcome this drawback, this work presents a novel hierarchical Bayesian formulation of a generalized linear model and estimates the posterior distribution of the parameters and hyper-parameters of the model within a completely Bayesian paradigm based on variational inference. All the required computations are analytically tractable. The performance and applicability of the proposed framework is demonstrated on synthetic and real world examples.