Staged Mixture Modelling and Boosting
arXiv (Cornell University)Published 12 December 2012Open access
Christopher Meek, Bo Thiesson, David Heckerman
Citations16
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper introduces and evaluates a data-driven staged mixture modeling tcchnique for building density, regression, and classification models and shows that the technique is qualitatively similar to boosting.
Abstract
In this paper, we introduce and evaluate a data-driven staged mixture modeling technique for building density, regression, and classification models. Our basic approach is to sequentially add components to a finite mixture model using the structural expectation maximization (SEM) algorithm. We show that our technique is qualitatively similar to boosting. This correspondence is a natural byproduct of the fact that we use the SEM algorithm to sequentially fit the mixture model. Finally, in our experimental evaluation, we demonstrate the effectiveness of our approach on a variety of prediction and density estimation tasks using real-world data.
Keywords
Computer Science
The Annals of StatisticsGreedy function approximation: A gradient boosting machine.
28,973 Citations2001Jerome H. Friedman
A general gradient descent boosting paradigm is developed for additive expansions based on any fitting criterion, and specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification.
Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
20,320 Citations1997Yoav Freund, Robert E. Schapire
The Annals of StatisticsAdditive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
6,842 Citations2000Jerome H. Friedman, Trevor Hastie +1 more
This work shows that this seemingly mysterious phenomenon of boosting can be understood in terms of well-known statistical principles, namely additive modeling and maximum likelihood, and develops more direct approximations and shows that they exhibit nearly identical results to boosting.
Learning Belief Networks in the Presence of Missing Values and Hidden Variables
391 Citations1997Nir Friedman
This paper proposes a new method for learning network structure from incomplete data based on an extension of the Expectation-Maximization (EM) algorithm for model selection problems that performs search for the best structure inside the EM procedure.
IEEE Transactions on Neural NetworksModeling the manifolds of images of handwritten digits
329 Citations1997Geoffrey E. Hinton, Peter Dayan +1 more
Two new methods for modeling the manifolds of digitized images of handwritten digits of principal components analysis and factor analysis are described, based on locally linear low-dimensional approximations to the underlying data manifold.
arXiv (Cornell University)A Bayesian Approach to Learning Bayesian Networks with Local Structure
326 Citations2013David Maxwell Chickering, David Heckerman +1 more
Mixture Density Estimation
236 Citations1999Jonathan Q. Li, Andrew R. Barron
For mixture density estimation, it is shown that a k-component mixture estimated by maximum likelihood (or by an iterative likelihood improvement that is introduced) achieves log-likelihood within order 1/k of the log- likelihood achievable by any convex combination.
Applied Physics Letters10.1162/153244301753344605
188 Citations2000
The mixtures-of-trees model is described, a probabilistic model for discrete multidimensional domains that generalizes the probabilistic trees of Chow and Liu (1968) in a different and complementary direction to that of Bayesian networks.
The Annals of StatisticsStratified exponential families: Graphical models and model selection
115 Citations2001Dan Geiger, David Heckerman +2 more
This work describes a hierarchy of exponential families which is useful for distinguishing types of graphical models and shows how to compute the dimension of a stratified exponential family.
Autoregressive Tree Models for Time-Series Analysis
65 Citations2002Christopher Meek, David M. Chickering +1 more
Microsoft Windows highly intelligent speech recognizer: Whisper
59 Citations2002Xuedong Huang, Alex Acero +4 more
The Whisper (Windows Highly Intelligent Speech Recognizer) represents significantly improved recognition efficiency, usability, and accuracy, when compared with the Sphinx-II system.
Efficient determination of dynamic split points in a decision tree
13 Citations2002David M. Chickering, Christopher Meek +1 more
A number of alternative approaches that generate a small number of candidate split points dynamically with little overhead are described, and it is argued that these approaches are preferable to pre-discretization, and experimental evidence that they yield probabilistic decision trees with the same prediction accuracy as the traditional dynamic approach.
Computationally Efficient Methods For Selecting Among Mixtures Of Graphical Models, With Discussion
4 Citations1999Bo Thiesson, C. E. Meek +2 more
It is argued that simple search-and-score algorithms are infeasible for a variety of problems, and a feasible approach in which parameter and structure search is interleaved and expected data is treated as real data is introduced.