Sequential Latent Dirichlet Allocation: Discover Underlying Topic Structures within a Document
Published 1 December 2010
Lan Du, Wray Buntine, Huidong Jin
Citations40
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
By taking into account the sequential structure within a document, the SeqLDA model has a higher fidelity over LDA in terms of perplexity (a standard measure of dictionary-based compressibility) and yields a nicer sequential topic structure than LDA.
Abstract
Understanding how topics within a document evolve over its structure is an interesting and important problem. In this paper, we address this problem by presenting a novel variant of Latent Dirichlet Allocation (LDA): Sequential LDA (SeqLDA). This variant
Keywords
Computer Science
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Proceedings of the National Academy of SciencesFinding scientific topics
6,017 Citations2004Thomas L. Griffiths, Mark Steyvers
A generative model for documents is described, introduced by Blei, Ng, and Jordan, and a Markov chain Monte Carlo algorithm is presented for inference in this model, which is used to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics.
Dynamic topic models
2,322 Citations2006David M. Blei, John Lafferty
A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections, and dynamic topic models provide a qualitative window into the contents of a large document collection.
Journal of the American Statistical AssociationGibbs Sampling Methods for Stick-Breaking Priors
1,556 Citations2001Hemant Ishwaran, Lancelot F. James
Two general types of Gibbs samplers that can be used to fit posteriors of Bayesian hierarchical models based on stick-breaking priors are presented and the blocked Gibbs sampler, based on an entirely different approach that works by directly sampling values from the posterior of the random measure.
Uncertainty in Artificial IntelligenceThe author-topic model for authors and documents
1,313 Citations2004Michal Rosen‐Zvi, Thomas Griffiths +2 more
The Annals of ProbabilityThe two-parameter Poisson-Dirichlet distribution derived from a stable subordinator
1,255 Citations1997Jim Pitman, Marc Yor
Topics over time
1,132 Citations2006Xuerui Wang, Andrew McCallum
An LDA-style topic model is presented that captures not only the low-dimensional structure of data, but also how the structure changes over time, showing improved topics, better timestamp prediction, and interpretable trends.
Neural Information Processing SystemsCorrelated Topic Models
929 Citations2005John Lafferty, David M. Blei
The correlated topic model (CTM) is developed, where the topic proportions exhibit correlation via the logistic normal distribution and a mean-field variational inference algorithm is derived for approximate posterior inference in this model, which is complicated by the fact that the Logistic normal is not conjugate to the multinomial.
Estimating a Dirichlet Distribution
778 Citations2000Thomas P. Minka
Journal of the ACMThe nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies
667 Citations2010David M. Blei, Thomas L. Griffiths +1 more
An application to information retrieval in which documents are modeled as paths down a random tree, and the preferential attachment dynamics of the nCRP leads to clustering of documents according to sharing of topics at multiple levels of abstraction.
Integrating Topics and Syntax
492 Citations2004Thomas L. Griffiths, Mark Steyvers +2 more
This work presents a generative model that uses both kinds of dependencies, and can be used to simultaneously find syntactic classes and semantic topics despite having no representation of syntax or semantics beyond statistical dependency.
On-line LDA: Adaptive Topic Models for Mining Text Streams with Applications to Topic Detection and Tracking
420 Citations2008Loulwah AlSumait, Daniel Barbará +1 more
A topic model that automatically captures the thematic patterns and identifies emerging topics of text streams and their changes over time and is comparable to, and sometimes better than, the original LDA in predicting the likelihood of unseen documents.
Detecting topic evolution in scientific literature
165 Citations2009Qi He, Bi Yu Chen +4 more
An iterative topic evolution learning framework is proposed by adapting the Latent Dirichlet Allocation model to the citation network and develop a novel inheritance topic model, which clearly shows that citations can help to understand topic evolution better.
A Bayesian Interpretation of Interpolated Kneser-Ney
108 Citations2006YW Teh
Lecture notes in computer scienceDiscrete Component Analysis
90 Citations2006Wray Buntine, Aleks Jakulin
A unified theory for analysis of components in discrete data is presented, and the methods compared with techniques such as independent component analysis, non-negative matrix factorisation and latent Dirichlet allocation are compared.
ProceedingsLatent Dirichlet Co-Clustering
87 Citations2006Mahdi Shafiei, Evangelos Milios
This work presents a generative model for simultaneously clustering documents and terms and presents efficient approximate inference techniques based on Markov Chain Monte Carlo method and a moment-matching algorithm for empirical Bayes parameter estimation.
Machine LearningA segmented topic model based on the two-parameter Poisson-Dirichlet process
75 Citations2010Lan Du, Wray Buntine +1 more
This paper takes the simplest form of structure, a document consisting of multiple segments, as the basis for a new form of topic model, and uses the marginalized posterior of a two-parameter Poisson-Dirichlet process (or Pitman-Yor process) to handle the hierarchical modelling.
Multiscale topic tomography
59 Citations2007Ramesh Nallapati, Susan Ditmore +2 more
A new probabilistic graphical model is proposed that employs non-homogeneous Poisson processes to model generation of word-counts and its modeling the evolution of topics at various time-scales of resolution, allowing the user to zoom in and out of the time-Scales.
arXiv (Cornell University)A Bayesian View of the Poisson-Dirichlet Process
48 Citations2010Wray Buntine, Marcus Hütter
A Bayesian interpretation of the Poisson-Dirichlet process is presented: it is a mixture using an improper and infinite dimensional Dirichlet distribution, and thus all its sampling properties fit naturally.
A Generative Probabilistic Model for Multi-label Classification
31 Citations2008Hongning Wang, Minlie Huang +1 more
The CoL model is presented to capture the correlation between classes and the underlying structures via the latent random variables in a supervised manner and develops a variational procedure to approximate the posterior distribution and employs the EM algorithm for the empirical Bayes parameter estimation.