Structured Variational Distributions in VIBES

Published 3 January 2003

Chris Bishop, John Winn

Citations34

TL;DR

This paper presents an extension of VIBES in which the variational posterior distribution corresponds to a sub-graph of the full probabilistic model, which can produce much closer approximations to the true posterior distribution.

Abstract

Variational methods are becoming increasingly popular for the approximate solution of complex probabilistic models in machine learning, computer vision, information retrieval and many other fields. Unfortunately, for every new application it is necessary first to derive the specific forms of the variational update equations for the particular probabilistic model being used, and then to implement these equations in applicationspecific software. Each of these steps is both time consuming and error prone. We have therefore recently developed a general purpose inference engine called VIBES [1] (`Variational Inference for Bayesian Networks&apos;) which allows a wide variety of probabilistic models to be implemented and solved variationally without recourse to coding. New models are specified as a directed acyclic graph using an interface analogous to a drawing package, and VIBES then automatically generates and solves the variational equations.

Keywords

Computer ScienceMathematics

Statistics and ComputingWinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility

5,395 Citations2000David J. Lunn, Andrew C. Thomas +2 more

How and why various modern computing concepts, such as object-orientation and run-time linking, feature in the software's design are discussed and how the framework may be extended.

Machine LearningAn Introduction to Variational Methods for Graphical Models

3,775 Citations1999Michael I. Jordan, Zoubin Ghahramani +2 more

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields), and describes a general framework for generating variational transformations based on convex duality.

View PDF

Machine LearningFactorial Hidden Markov Models

1,189 Citations1997Zoubin Ghahramani, Michael I. Jordan

A generalization of HMMs in which this state is factored into multiple state variables and is therefore represented in a distributed manner, and a structured approximation in which the the state variables are decoupled, yielding a tractable algorithm for learning the parameters of the model.

View PDF

Neural Information Processing SystemsA Variational Baysian Framework for Graphical Models

695 Citations1999Hagai Attias

Propagation Algorithms for Variational Bayesian Learning

288 Citations2000Zoubin Ghahramani, Matthew J. Beal

It is demonstrated how the belief propagation and the junction tree algorithms can be used in the inference step of variational Bayesian learning to infer the hidden state dimensionality of the state-space model in a variety of synthetic problems and one real high-dimensional data set.

Variational principal components

270 Citations1999Chris Bishop

This paper develops an alternative, variational formulation of Bayesian PCA, based on a factorial representation of the posterior distribution, which maximizes a rigorous lower bound on the marginal log probability of the observed data.

Exploiting Tractable Substructures in Intractable Networks

221 Citations1995Lawrence K. Saul, Michael I. Jordan

A refined mean field approximation for inference and learning in probabilistic neural networks is developed, and it is shown how to incorporate weak higher order interactions into a first-order hidden Markov model.

Perspectives in neural computingEnsemble Learning for Blind Image Separation and Deconvolution

164 Citations2000James W. Miskin, David Mackay

In this chapter, ensemble learning is applied to the problem of blind source separation and deconvolution of images by convolving mixtures with unknown blurring filters and then adding Gaussian noise.

Neural Information Processing SystemsVIBES: A Variational Inference Engine for Bayesian Networks

88 Citations2002Chris Bishop, David J. Spiegelhalter +1 more

A general purpose inference engine called VIBES ('Variational Inference for Bayesian Networks') which allows a wide variety of probabilistic models to be implemented and solved variationally without recourse to coding.

arXiv (Cornell University)Variational Approximations between Mean Field Theory and the Junction\n Tree Algorithm

80 Citations2013Wim Wiegerinck

View PDF