<b>topicmodels</b>: An<i>R</i>Package for Fitting Topic Models
Journal of Statistical SoftwarePublished 1 January 2011Open access
Bettina Grün, Kurt Hornik
Citations1,088
SJR quartileQ1
SJR score3.21
SNIP4.61
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The R package topicmodels provides basic infrastructure for fitting topic models based on data structures from the text mining package tm to estimate the similarity between documents as well as between a set of specified keywords using an additional layer of latent variables.
Abstract
Topic models allow the probabilistic modeling of term frequency occurrences in documents. The fitted model can be used to estimate the similarity between documents as well as between a set of specified keywords using an additional layer of latent variables which are referred to as topics. The R package topicmodels provides basic infrastructure for fitting topic models based on data structures from the text mining package tm. The package includes interfaces to two algorithms for fitting topic models: the variational expectation-maximization algorithm provided by David M. Blei and co-authors and an algorithm using Gibbs sampling by Xuan-Hieu Phan and co-authors.
Keywords
Computer ScienceSocial Sciences
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Proceedings of the National Academy of SciencesFinding scientific topics
6,017 Citations2004Thomas L. Griffiths, Mark Steyvers
A generative model for documents is described, introduced by Blei, Ng, and Jordan, and a Markov chain Monte Carlo algorithm is presented for inference in this model, which is used to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics.
Statistics and ComputingWinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
5,395 Citations2000David J. Lunn, Andrew C. Thomas +2 more
How and why various modern computing concepts, such as object-orientation and run-time linking, feature in the software's design are discussed and how the framework may be extended.
JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
4,702 Citations2003Martyn Plummer
Journal of the American Statistical AssociationHierarchical Dirichlet Processes
3,567 Citations2006Yee Whye Teh, Michael I. Jordan +2 more
This work considers problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups, and considers a hierarchical model, specifically one in which the base measure for the childDirichlet processes is itself distributed according to a Dirichlet process.
now publishers, Inc. eBooksGraphical Models, Exponential Families, and Variational Inference
3,154 Citations2007Martin J. Wainwright, Michael I. Jordan
Machine LearningText Classification from Labeled and Unlabeled Documents using EM
2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more
This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.
Dynamic topic models
2,322 Citations2006David M. Blei, John Lafferty
A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections, and dynamic topic models provide a qualitative window into the contents of a large document collection.
Psychology Press eBooksHandbook of Latent Semantic Analysis
1,565 Citations2007Thomas K. Landauer
This book discusses Latent Semantic Analysis as a Theory of Meaning, its application in Cognitive Theory, and its applications in Educational Applications.
PubMedMixed Membership Stochastic Blockmodels.
1,526 Citations2008Edoardo M. Airoldi, David M. Blei +2 more
This paper describes a latent variable model of such data called the mixed membership stochastic blockmodel, which extends blockmodels for relational data to ones which capture mixed membership latent relational structure, thus providing an object-specific low-dimensional representation.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Approximate Bayesian Inference with the Weighted Likelihood Bootstrap
1,515 Citations1994Michael A. Newton, Adrian E. Raftery
The weighted likelihood bootstrap is introduced as a way to simulate approximately from a posterior distribution, and the second-order properties of the WLB are elucidated, which is a generalization of Rubin's Bayesian bootstrap.
arXiv (Cornell University)Supervised Topic Models
1,315 Citations2010David M. Blei, Jon McAuliffe
The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.
Online Learning for Latent Dirichlet Allocation
1,294 Citations2010Matthew D. Hoffman, Francis Bach +1 more
An online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA) based on online stochastic optimization with a natural gradient step is developed, which shows converges to a local optimum of the VB objective function.
Journal of Statistical SoftwareText Mining Infrastructure in<i>R</i>
1,145 Citations2008Ingo Feinerer, Kurt Hornik +1 more
The tm package is presented which provides a framework for text mining applications within R and techniques for count-based analysis methods, text clustering, text classification and string kernels are presented.
OPAL (Open@LaTrobe) (La Trobe University)A correlated topic model of Science
1,137 Citations2018David M. Blei, John Lafferty
LDA-based document models for ad-hoc retrieval
1,086 Citations2006Xing Wei, W. Bruce Croft
This paper proposes an LDA-based document model within the language modeling framework, and evaluates it on several TREC collections, and shows that improvements over retrieval using cluster-based models can be obtained with reasonable efficiency.
Probabilistic Topic Models
1,031 Citations2015Mark Steyvers, Thomas L. Griffiths
Neural Information Processing SystemsHierarchical Topic Models and the Nested Chinese Restaurant Process
938 Citations2003Thomas L. Griffiths, Michael I. Jordan +2 more
A Bayesian approach is taken to generate an appropriate prior via a distribution on partitions that allows arbitrarily large branching factors and readily accommodates growing data collections.
Journal of Statistical Software<b>statnet</b>: Software Tools for the Representation, Visualization, Analysis and Simulation of Network Data
917 Citations2008Mark S. Handcock, David R. Hunter +3 more
Statnet is a suite of software packages for statistical network analysis that provides a comprehensive framework for ERGM-based network modeling, including tools for model estimation, model evaluation, model- based network simulation, and network visualization.
MPG.PuRe (Max Planck Society)Clustering on the Unit Hypersphere using von Mises-Fisher Distributions
807 Citations2005Arindam Banerjee, Inderjit S. Dhillon +2 more
A generative mixture-model approach to clustering directional data based on the von Mises-Fisher distribution, which arises naturally for data distributed on the unit hypersphere, and derives and analyzes two variants of the Expectation Maximization framework for estimating the mean and concentration parameters of this mixture.
Learning to classify short and sparse text & web with hidden topics from large-scale data collections
739 Citations2008Xuan-Hieu Phan, Le-Minh Nguyen +1 more
A general framework for building classifiers that deal with short and sparse text & Web segments by making the most of hidden topics discovered from large-scale data collections that is general enough to be applied to different data domains and genres ranging from Web search results to medical text.
Pachinko allocation
612 Citations2006Wei Li, Andrew McCallum
Improved performance of PAM is shown in document classification, likelihood of held-out data, the ability to support finer-grained topics, and topical keyword coherence.
Fast collapsed gibbs sampling for latent dirichlet allocation
567 Citations2008Ian R. Porteous, David Newman +4 more
A novel collapsed Gibbs sampling method for the widely used latent Dirichlet allocation (LDA) model, which can be as much as 8 times faster than the standard collapsed Gibbs sampler for LDA and results in significant speedups on real world text corpora.
Relational Topic Models for Document Networks
485 Citations2009Jonathan Chang, David M. Blei
The relational topic model (RTM) is developed, a model of documents and the links between them that derives efficient inference and learning algorithms based on variational methods and evaluates the predictive performance of the RTM for large networks of scientific abstracts and web documents.
Studying the history of ideas using topic models
455 Citations2008David Hall, Daniel Jurafsky +1 more
Unsupervised topic modeling is applied to the ACL Anthology to analyze historical trends in the field of Computational Linguistics from 1978 to 2006, finding trends including the rise of probabilistic methods starting in 1988, a steady increase in applications, and a sharp decline of research in semantics and understanding between 1978 and 2001.
Distributed Algorithms for Topic Models
395 Citations2009David Newman, Arthur Asuncion +2 more
ACM Transactions on Information SystemsLearning author-topic models from text corpora
304 Citations2010Michal Rosen‐Zvi, Chaitanya Chemudugunta +3 more
The interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, parsing of abstracts by topics and authors, and detection of unusual papers by specific authors are discussed.
Journal of Statistical SoftwareA CLUE for CLUster Ensembles
261 Citations2005Kurt Hornik
The R package clue provides an extensible computational environment for creating and analyzing cluster ensembles, with basic data structures for representing partitions and hierarchies, and facilities for computing on these, including methods for measuring proximity and obtaining consensus and "secondary" clusterings.
Mixtures of hierarchical topics with Pachinko allocation
225 Citations2007David Mimno, Wei Li +1 more
H hierarchical PAM is presented---an enhancement that explicitly represents a topic hierarchy that can be seen as combining the advantages of hLDA's topical hierarchy representation with PAM's ability to mix multiple leaves of the topic hierarchy.
Journal of Statistical Software<b>movMF</b>: An<i>R</i>Package for Fitting Mixtures of von Mises-Fisher Distributions
126 Citations2014Kurt Hornik, Bettina Grün
The main fitting function of the R package movMF, which contains functionality to draw samples from finite mixtures of von Mises-Fisher distributions and to fit these models using the expectation-maximization algorithm for maximum likelihood estimation, is described and illustrated.
Connections between the lines
121 Citations2009Jonathan Chang, Jordan Boyd‐Graber +1 more
A novel probabilistic topic model is presented to analyze text corpora and infer descriptions of its entities and of relationships between those entities and it is shown qualitatively and quantitatively that the model can construct and annotate graphs of relationships and make useful predictions.
Lecture notes in computer scienceA Generic Approach to Topic Models
72 Citations2009Gregor Heinrich
The benefit of this representation of topic models as "mixture networks" is its straight-forward mapping to inference equations and algorithms, which is shown with the derivation and implementation of a generic Gibbs sampling algorithm.
Tensor Space Learning for Analyzing Activity Patterns from Video Sequences
59 Citations2007Liang Wang, Christopher Leckie +3 more
A tensor space representation for analyzing human activity patterns in monocular videos that can effectively identify human activities with intra- and inter-person variations on both temporal and spatial scales is described.
Lecture notes in computer scienceExploring LDA-Based Document Model for Geographic Information Retrieval
10 Citations2008Zhisheng Li, Chong Wang +3 more
An LDA-based document model to improve geographic information retrieval by inheriting the LDA model with text retrieval model is proposed and has been evaluated on GeoCLEF2007 collection.
Lecture notes in computer scienceVariational Bayes for Generic Topic Models
7 Citations2009Gregor Heinrich, Michael Goesele
The article contributes a derivation of variational Bayes for a large class of topic models by generalising from the well-known model of latent Dirichlet allocation, leading to inference algorithms for models that so far have used Gibbs sampling exclusively.