Probabilistic topic models
Communications of the ACMPublished 23 March 2012
David M. Blei
Citations5,549
SJR quartileQ1
SJR score1.15
SNIP3.34
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Surveying a suite of algorithms that offer a solution to managing large document archives suggests they are well-suited to handle large amounts of data.
Abstract
Surveying a suite of algorithms that offer a solution to managing large document archives.
Keywords
Computer Science
GeneticsInference of Population Structure Using Multilocus Genotype Data
34,260 Citations2000Jonathan K. Pritchard, Matthew Stephens +1 more
A model-based clustering method for using multilocus genotype data to infer population structure and assign individuals to populations that can be applied to most of the commonly used genetic markers, provided that they are not closely linked.
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
NeurocomputingAdvances in neural information processing systems 7
22,300 Citations1997Krzysztof J. Cios, Mark E. Shields
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Machine LearningAn Introduction to Variational Methods for Graphical Models
3,775 Citations1999Michael I. Jordan, Zoubin Ghahramani +2 more
This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields), and describes a general framework for generating variational transformations based on convex duality.
A Bayesian Hierarchical Model for Learning Natural Scene Categories
3,613 Citations2005Li Fei-Fei, Pietro Perona
This work proposes a novel approach to learn and recognize natural scene categories by representing the image of a scene by a collection of local regions, denoted as codewords obtained by unsupervised learning.
Journal of the American Statistical AssociationHierarchical Dirichlet Processes
3,567 Citations2006Yee Whye Teh, Michael I. Jordan +2 more
This work considers problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups, and considers a hierarchical model, specifically one in which the base measure for the childDirichlet processes is itself distributed according to a Dirichlet process.
now publishers, Inc. eBooksGraphical Models, Exponential Families, and Variational Inference
3,154 Citations2007Martin J. Wainwright, Michael I. Jordan
Dynamic topic models
2,322 Citations2006David M. Blei, John Lafferty
A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections, and dynamic topic models provide a qualitative window into the contents of a large document collection.
arXiv (Cornell University)Probabilistic Latent Semantic Analysis
2,092 Citations2013Thomas Hofmann
This work proposes a widely applicable generalization of maximum likelihood model fitting by tempered EM, based on a mixture decomposition derived from a latent class model which results in a more principled approach which has a solid foundation in statistics.
Foundations and Trends® in Machine LearningGraphical Models, Exponential Families, and Variational Inference
1,780 Citations2008Martin J. Wainwright, Michael I. Jordan
The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.
The Annals of StatisticsBayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician
1,349 Citations1984Donald B. Rubin
arXiv (Cornell University)Supervised Topic Models
1,315 Citations2010David M. Blei, Jon McAuliffe
The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.
Uncertainty in Artificial IntelligenceThe author-topic model for authors and documents
1,313 Citations2004Michal Rosen‐Zvi, Thomas Griffiths +2 more
Online Learning for Latent Dirichlet Allocation
1,294 Citations2010Matthew D. Hoffman, Francis Bach +1 more
An online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA) based on online stochastic optimization with a natural gradient step is developed, which shows converges to a local optimum of the VB objective function.
OPAL (Open@LaTrobe) (La Trobe University)A correlated topic model of Science
1,137 Citations2018David M. Blei, John Lafferty
Modeling annotated data
1,070 Citations2003David M. Blei, Michael I. Jordan
Three hierarchical probabilistic mixture models which aim to describe annotated data with multiple types, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type.
Topic modeling
1,057 Citations2006Hanna Wallach
A hierarchical generative probabilistic model that incorporates both n-gram statistics and latent topic variables by extending a unigram topic model to include properties of a hierarchical Dirichlet bigram language model is explored.
Journal of the Royal Statistical Society Series A (General)Sampling and Bayes' Inference in Scientific Modelling and Robustness
936 Citations1980George E. P. Box
Predictive checking functions for transformation, serial correlation, bad values, and their relation with Bayesian options are considered, and robustness is seen from a Bayesian viewpoint and examples are given.
Journal of the ACMThe nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies
667 Citations2010David M. Blei, Thomas L. Griffiths +1 more
An application to information retrieval in which documents are modeled as paths down a random tree, and the preferential attachment dynamics of the nCRP leads to clustering of documents according to sharing of topics at multiple levels of abstraction.
Pachinko allocation
612 Citations2006Wei Li, Andrew McCallum
Improved performance of PAM is shown in document classification, likelihood of held-out data, the ability to support finer-grained topics, and topical keyword coherence.
Political AnalysisA Bayesian Hierarchical Topic Model for Political Texts: Measuring Expressed Agendas in Senate Press Releases
558 Citations2009Justin Grimmer
A statistical model is introduced that attends to the structure of political rhetoric when measuring expressed priorities: statements are naturally organized by author to simultaneously estimate the topics in the texts, as well as the attention political actors allocate to the estimated topics.
Integrating Topics and Syntax
492 Citations2004Thomas L. Griffiths, Mark Steyvers +2 more
This work presents a generative model that uses both kinds of dependencies, and can be used to simultaneously find syntactic classes and semantic topics despite having no representation of syntax or semantics beyond statistical dependency.
arXiv (Cornell University)On Smoothing and Inference for Topic Models
452 Citations2012Arthur Asuncion, Max Welling +2 more
arXiv (Cornell University)Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression
298 Citations2012David Mimno, Andrew McCallum
The Annals of Applied StatisticsHierarchical relational models for document networks
260 Citations2010Jonathan Chang, David M. Blei
The relational topic model (RTM), a hierarchical model of both network structure and node attributes, is developed, where the attributes of each document are its words, that is, discrete observations taken from a fixed vocabulary.
Unsupervised discovery of visual object class hierarchies
213 Citations2008Josef Šivic, Bryan Russell +3 more
This work proposes to group visual objects using a multi-layer hierarchy tree that is based on common visual elements by adapting to the visual domain the generative hierarchical latent Dirichlet allocation (hLDA) model previously used for unsupervised discovery of topic hierarchies in text.
Lecture notes in computer scienceVariational Extensions to EM and Multinomial PCA
163 Citations2002Wray Buntine
This paper begins with a review of the basic theory of the variational extension to the expectation-maximization algorithm, and then presents discrete component finding algorithms in that light.
arXiv (Cornell University)Syntactic Topic Models
155 Citations2010Jordan Boyd‐Graber, David M. Blei
The syntactic topic model (STM) is developed, a nonparametric Bayesian model of parsed documents that generates words that are both thematically and syntactically constrained, which combines the semantic insights of topic models with the syntactic information available from parse trees.
Statistical entity-topic models
147 Citations2006David Newman, Chaitanya Chemudugunta +1 more
New graphical models are presented that directly learn the relationship between topics discussed in news articles and entities mentioned in each article and it is shown how these entity-topic models, through a better understanding of the entity- topic relationships, are better at making predictions about entities.
A Language-based Approach to Measuring Scholarly Impact
122 Citations2010Sean Gerrish, David M. Blei
This work proposes using changes in the thematic content of documents over time to measure the importance of individual documents within the collection, and describes a dynamic topic model for both quantifying and qualifying the impact of these documents.
Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process
116 Citations2009Chong Wang, David M. Blei
An efficient Gibbs sampler for the sparseTM is developed that includes a general-purpose method for sampling from a Dirichlet mixture with a combinatorial number of components and will show that sparseTMs give better predictive performance with simpler inferred models.
Building and using a semantivisual image hierarchy
110 Citations2010Li-Jia Li, Chong Wang +3 more
This paper proposes a method to automatically discover the “semantivisual” image hierarchy by incorporating both image and tag information, and shows that humans find the constructed image hierarchy more effective than those solely based on texts or low-level visual features.
Accounting for burstiness in topic models
101 Citations2009Gabriel Doyle, Charles Elkan
A topic model is introduced that uses Dirichlet compound multinomial (DCM) distributions to model this burstiness phenomenon and achieves better held-out likelihood than standard latentDirichlet allocation (LDA).
Lecture notes in computer scienceDiscrete Component Analysis
90 Citations2006Wray Buntine, Aleks Jakulin
A unified theory for analysis of components in discrete data is presented, and the methods compared with techniques such as independent component analysis, non-negative matrix factorisation and latent Dirichlet allocation are compared.
Spherical Topic Models
78 Citations2010Joseph Reisinger, Austin Waters +2 more
The Spherical Admixture Model is introduced, a Bayesian topic model for arbitrary l2 normalized data that maintains the same hierarchical structure as Latent Dirichlet Allocation, but models documents as points on a high-dimensional spherical manifold, allowing a natural likelihood parameterization in terms of cosine distance.
APPLIED STATISTICS: A REVIEW
41 Citations2012D. R. Cox
Markov Topic Models
40 Citations2009Chong Wang, Bo Thiesson +2 more
Markov topic models are developed, a novel family of generative probabilistic models that can learn topics simultaneously from multiple corpora, such as papers from different conferences, and improve quantitative performance over the state of the art.
The Annals of Applied StatisticsApplied statistics: A review
32 Citations2007D. R. Cox
The main phases of applied statistical work are discussed in general terms and an attempt is made to extract some general notions.
A Bayesian Analysis of Dynamics in Free Recall
24 Citations2009Richard Socher, Samuel J. Gershman +4 more
A probabilistic model of human memory performance in free recall experiments is developed, conceptualizing memory retrieval as a dynamic latent variable model and using Bayesian inference to represent uncertainty and reason about the cognitive processes underlying memory.
IEEE Transactions on Pattern Analysis and Machine IntelligenceUnsupervised Organization of Image Collections: Taxonomies and Beyond
18 Citations2011Evgeniy Bart, Max Welling +1 more
A nonparametric Bayesian model, called TAX, is introduced, which can organize image collections into a tree-shaped taxonomy without supervision and associates each image with a path through the taxonomy.