Bayesian Inference on Mixtures of Distributions
arXiv (Cornell University)Published 15 April 2008Open access
Kate Lee, Jean‐Michel Marin, Inria Saclay, Kerrie Mengersen
Citations37
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This survey covers state-of-the-art Bayesian techniques for the estimation of mixtures by studying new types of distributions, the multinomial, latent class and t distributions and sheds a new light on the computation of Bayes factors via the approximation of Chib (1995).
Abstract
This survey covers state-of-the-art Bayesian techniques for the estimation of mixtures. It complements the earlier Marin, Mengersen and Robert (2005) by studying new types of distributions, the multinomial, latent class and t distributions. It also exhibits closed form solutions for Bayesian inference in some discrete setups. Lastly, it sheds a new light on the computation of Bayes factors via the approximation of Chib (1995).
Keywords
Computer ScienceMathematics
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Journal of the Franklin InstituteAn introduction to probability theory and its applications
29,966 Citations1958
Wiley series in probability and statisticsFinite Mixture Models
7,410 Citations2018Geoffrey J. McLachlan, David Peel +1 more
The aim of this article is to provide an up-to-date account of the theory and methodological developments underlying the applications of finite mixture models.
TechnometricsMonte Carlo Statistical Methods
5,612 Citations2000Hoon Kim, Christian P. Robert +1 more
Journal of the American Statistical AssociationThe Calculation of Posterior Distributions by Data Augmentation
3,782 Citations1987Martin A. Tanner, Wing Hung Wong
If data augmentation can be used in the calculation of the maximum likelihood estimate, then in the same cases one ought to be able to use it in the computation of the posterior distribution of parameters of interest.
Journal of the Royal Statistical Society Series B (Statistical Methodology)On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)
1,913 Citations1997Sylvia Richardson, Peter J. Green
Journal of the American Statistical AssociationMarginal Likelihood from the Gibbs Output
1,812 Citations1995Siddhartha Chib
This work exploits the fact that the marginal density can be expressed as the prior times the likelihood function over the posterior density, so that Bayes factors for model comparisons can be routinely computed as a by-product of the simulation.
Springer series in statisticsFinite Mixture and Markov Switching Models
1,591 Citations2006Sylvia Frühwirth‐Schnatter
This book should help newcomers to the field to understand how finite mixture and Markov switching models are formulated, what structures they imply on the data, what they could be used for, and how they are estimated.
Bayesian AnalysisNested sampling for general Bayesian computation
1,564 Citations2006John Skilling
Nested sampling estimates directly how the likelihood function relates to prior mass, and invariance over monotonic re- labelling allows the method to deal with a class of phase-change problems which eectiv ely defeat thermal annealing.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Bayesian Model Choice: Asymptotics and Exact Calculations
1,239 Citations1994Alan E. Gelfand, Dipak K. Dey
A general predictive density is presented which includes all proposed Bayesian approaches the authors are aware of and using Laplace approximations they can conveniently assess and compare asymptotic behavior of these approaches.
Journal of the American Statistical AssociationMarginal Likelihood From the Metropolis–Hastings Output
1,094 Citations2001Siddhartha Chib, Ivan Jeliazkov
The proposed method is developed in the context of MCMC chains produced by the Metropolis–Hastings algorithm, whose building blocks are used both for sampling and marginal likelihood estimation, thus economizing on prerun tuning effort and programming.
On Bayesian Analysis of Mixtures with an Unknown Number of Components
937 Citations1997Sylvia Richardson, Peter J. Green
Journal of the American Statistical AssociationComputational and Inferential Difficulties with Mixture Posterior Distributions
568 Citations2000Gilles Celeux, Merrilee Hurn +1 more
This article proposes alternatives for Bayesian inference for permutation invariant posteriors, including a clustering device and alternative appropriate loss functions and shows that exploration of these modes can be imposed using tempered transitions.
The Annals of StatisticsBayesian analysis of mixture models with an unknown number of components—an alternative to reversible jump methods
523 Citations2000Matthew Stephens
Journal of the American Statistical AssociationPractical Bayesian Density Estimation Using Mixtures of Normals
480 Citations1997Kathryn Roeder, Larry Wasserman
Journal of the Royal Statistical Society Series B (Statistical Methodology)Efficient Construction of Reversible Jump Markov Chain Monte Carlo Proposal Distributions
316 Citations2003Stephen P. Brooks, Paolo Giudici +1 more
The main motivation for this paper is that, in complex problems, the probability that the Markov chain moves between such spaces may be prohibitively small, as the probability mass can be very thinly spread across the space.
Journal of the American Statistical AssociationReal-Parameter Evolutionary Monte Carlo With Applications to Bayesian Mixture Models
210 Citations2001Faming Liang, Wing Hung Wong
An evolutionary Monte Carlo algorithm to sample from a target distribution with real-valued parameters is proposed and it is confirmed that the real-coded evolutionary algorithm is a promising general approach for simulation and optimization.
Computational Statistics & Data AnalysisInterpretation and inference in mixture models: Simple MCMC works
166 Citations2006John Geweke
The mixture model likelihood function is invariant with respect to permutation of the components of the mixture, and simple and widely used Markov chain Monte Carlo algorithms with data augmentation reliably recover the entire posterior distribution.
American Journal of SociologyRole Conflict and Personality
136 Citations1951Samuel A. Stouffer, Jackson Toby
Canadian Journal of StatisticsA Bayesian method for classification and discrimination
135 Citations1992Michael Lavine, Mike West
The development involves application of an iterative resampling approach to Monte Carlo inference, commonly called Gibbs sampling, and demonstrates routine application of exact analyses of traditional normal-mixture models for classification and discrimination.
BiometrikaProperties of nested sampling
130 Citations2010ChopinNicolas, Christian P. Robert
It is established that nested sampling has an approximation error that vanishes at the standard Monte Carlo rate and that this error is asymptotically Gaussian, and it is shown that the asymPTotic variance of the nested sampling approximation typically grows linearly with the dimension of the parameter.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Asymptotic Inference for Mixture Models by Using Data-Dependent Priors
126 Citations2000L. H. Wasserman
Journal of the Royal Statistical Society Series B (Statistical Methodology)A Theory of Statistical Models for Monte Carlo Integration
125 Citations2003Augustine Kong, Peter McCullagh +3 more
Statistics and ComputingBayesian analysis of outlier problems using the Gibbs sampler
119 Citations1991Isabella Verdinelli, Larry Wasserman
It is shown that the Gibbs sampler brings considerable conceptual and computational simplicity to the problem of calculating posterior marginals and is notable for its ease of implementation.
BiometrikaModel choice in generalised linear models: a Bayesian approach via Kullback-Leibler projections
99 Citations1998Costas Goutis
A general Bayesian method of comparing models based on the Kullback-Leibler distance between two families of models, one nested within the other, which can judge whether or not a more parsimonious model is appropriate.
A Bayesian approach to the selection and testing of mixture models
76 Citations2003Johannes Berkhof, Iven Van Mechelen +1 more
This paper proposes a variant of Chib's estimator that accounts for the non-identifiability of the mixture components and proposes to extend the Bayes factor model with a hyperprior to reduce the prior sensitivity.
Statistics and ComputingReparameterization strategies for hidden Markov models and Bayesian approaches to maximum likelihood estimation
60 Citations1998Christian P. Robert, D. M. Titterington
A global approach to both Bayesian and likelihood treatments of the estimation of the parameters of a hidden Markov model in the cases of normal and Poisson distributions is synthesized.
Journal of Computational and Graphical StatisticsComputing Normalizing Constants for Finite Mixture Models via Incremental Mixture Importance Sampling (IMIS)
52 Citations2006Russell Steele, Adrian E. Raftery +1 more
The method performed well in simulations and in mixture problems in astronomy and medical research, and makes it easier to adjust tuning parameters in the course of estimation than standard Markov chain Monte Carlo algorithms.
Statistical MethodologyMixture models, latent variables and partitioned importance sampling
38 Citations2004George Casella, Christian P. Robert +1 more
An alternative approach to estimation in mixture models is examined, one based on a Rao–Blackwellization argument applied to a latent-variable-based estimator, and an alternative Monte Carlo sampling scheme is constructed that avoids trapping states.
Canadian Journal of StatisticsA fast distance‐based approach for determining the number of components in mixtures
28 Citations2003Sujit K. Sahu, Russell Cheng
The authors propose a procedure for determining the unknown number of components in mixtures by generalizing a Bayesian testing method proposed by Mengersen & Robert (1996), which exploits a fast collapsing approach which accelerates the search for the minimum number of Components by avoiding full refitting at each step.
Journal of Statistical Planning and InferenceA default Bayesian test for the number of components in a mixture
23 Citations2002Elı́as Moreno, Brunero Liseo
Statistics and ComputingDirect simulation for discrete mixture distributions
17 Citations2005Paul Fearnhead
The approach is based on directly calculating the posterior distribution using a set of recursions which are similar to those of the Forward-Backward algorithm, which is more practicable than existing perfect simulation methods for mixtures.
arXiv (Cornell University)Contemplating Evidence: properties, extensions of, and alternatives to Nested Sampling
13 Citations2008Nicolás Chopin, Christian P. Robert
It is established that nested sampling leads to an error that vanishes at the standard Monte Carlo rate O(N^-1/2), where N is a tuning parameter that is proportional to the computational effort, and that this error is asymptotically Gaussian.
Aerosol formation over the Boreal forest in Hyytiälä, Finland: monthly frequency and annual cycles – the roles of air mass characteristics and synoptic scale meteorology
10 Citations2006E. D. Nilsson, Markku Kulmala
arXiv (Cornell University)On variance stabilisation by double Rao-Blackwellisation
3 Citations2008Alessandra Iacobucci, Jean‐Michel Marin +1 more
A modified version of the original Population Monte Carlo algorithm that eliminates the influence of the transition particle via a double Rao-Blackwellisation is shown to improve the exploration of the modes through an large simulation experiment on posterior distributions of mean mixtures of distributions.