Improving the Mean Field Approximation Via the Use of Mixture Distributions

This work employs mixture models as posterior approximations, where each mixture component is a factorized distribution, and describes efficient methods for optimizing the Parameters in these models.

Abstract

Mean field methods provide computationally efficient approximations to posterior probability distributions for graphical models. Simple mean field methods make a completely factorized approximation to the posterior, which is unlikely to be accurate when the posterior is multimodal. Indeed, if the posterior is multi-modal, only one of the modes can be captured. To improve the mean field approximation in such cases, we employ mixture models as posterior approximations, where each mixture component is a factorized distribution. We describe efficient methods for optimizing the Parameters in these models.