PLDA: Parallel Latent Dirichlet Allocation for Large-Scale Applications
Lecture notes in computer sciencePublished 1 January 2009
Yi Wang, Hongjie Bai, Matt Stanton, Wen-Yen Chen, Edward Yi Chang
Citations190
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
PLDA smooths out storage and computation bottlenecks and provides fault recovery for lengthy distributed computations and can be applied to large, real-world applications and achieves good scalability.
Abstract
This paper presents PLDA, our parallel implementation of Latent Dirichlet Allocation on MPI and MapReduce. PLDA smooths out storage and computation bottlenecks and provides fault recovery for lengthy distributed computations. We show that PLDA can be applied to large, real-world applications and achieves good scalability. We have released MPI-PLDA to open source at http://code.google.com/p/plda under the Apache License.
Keywords
Computer Science
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Communications of the ACMMapReduce
18,538 Citations2008Jay B. Dean, Sanjay Ghemawat
This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.
Proceedings of the National Academy of SciencesFinding scientific topics
6,017 Citations2004Thomas L. Griffiths, Mark Steyvers
A generative model for documents is described, introduced by Blei, Ng, and Jordan, and a Markov chain Monte Carlo algorithm is presented for inference in this model, which is used to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics.
Discourse ProcessesAn introduction to latent semantic analysis
4,817 Citations1998Thomas K. Landauer, Peter W. Foltz +1 more
The adequacy of LSA's reflection of human knowledge has been established in a variety of ways, for example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word‐word and passage‐word lexical priming data.
The MIT Press eBooksMap-Reduce for Machine Learning on Multicore
1,253 Citations2007Cheng-Tao Chu, Sang Kyun Kim +5 more
This work shows that algorithms that fit the Statistical Query model can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers and shows basically linear speedup with an increasing number of processors.
The International Journal of High Performance Computing ApplicationsOptimization of Collective Communication Operations in MPICH
909 Citations2005Rajeev Thakur, Rolf Rabenseifner +1 more
The work on improving the performance of collective communication operations in MPICH is described, with results indicating that to achieve the best performance for a collective communication operation, one needs to use a number of different algorithms and select the right algorithm for a particular message size and number of processes.
Fast collapsed gibbs sampling for latent dirichlet allocation
567 Citations2008Ian R. Porteous, David Newman +4 more
A novel collapsed Gibbs sampling method for the widely used latent Dirichlet allocation (LDA) model, which can be as much as 8 times faster than the standard collapsed Gibbs sampler for LDA and results in significant speedups on real world text corpora.
Expectation-propagation for the generative aspect model
442 Citations2002Thomas P. Minka, John Lafferty
arXiv (Cornell University)Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments
440 Citations2013Alexandrin Popescul, Lyle Ungar +2 more
It is shown that secondary content information can often be used to overcome sparsity and appropriate mixture models incorporating secondary data produce significantly better quality recommenders than k-nearest neighbors (k-NN).
Learning to Probabilistically Identify Authoritative Documents
252 Citations2000David Cohn, Huan Chang
A model of document citation that learns to identify hubs and authorities in a set of linked documents, such as pages retrieved from the world wide web, or papers retrieved from a research paper archive, provides probabilistic estimates that have clear semantics.
PSVM: Parallelizing Support Vector Machines on Distributed Computers
249 Citations2011Kaihua Zhu
A parallel SVM algorithm (PSVM), which reduces memory use through performing a row-based, approximate matrix factorization, and which loads only essential data to each machine to perform parallel computation.
Distributed Inference for Latent Dirichlet Allocation
220 Citations2007David Newman, Padhraic Smyth +2 more
Using five real-world text corpora, it is shown that distributed learning works very well for LDA models, i.e., perplexity and precision-recall scores for distributed learning are indistinguishable from those obtained with single-processor learning.
OPTIMOL: automatic Online Picture collecTion via Incremental MOdel Learning
197 Citations2007Li-Jia Li, Gang Wang +1 more
Neural Information Processing SystemsAsynchronous Distributed Learning of Topic Models
148 Citations2008Padhraic Smyth, Max Welling +1 more
It is demonstrated that the asynchronous algorithms presented are able to learn global topic models that are statistically as accurate as those learned by the standard LDA and HDP samplers, but with significant improvements in computation time and memory.
Collaborative filtering for orkut communities
143 Citations2009Wen-Yen Chen, Jon-Chyuan Chu +4 more
Empirical comparisons show that LDA performs consistently better than ARM for the community recommendation task when recommending a list of 4 or more communities, however, for recommendation lists of up to 3 communities, ARM is still a bit better.
Proceedings - IEEE International Parallel and Distributed Processing SymposiumAvoiding communication in sparse matrix computations
134 Citations2008J. Demmel, Mark Frederick Hoemmen +2 more
This paper focuses on an alternative building block for sparse iterative solvers, the "matrix powers kernel" [x, Ax, A2x, ..., Akx], and shows that by organizing computations around this kernel, it can achieve near-minimal communication costs.
Tensor Space Learning for Analyzing Activity Patterns from Video Sequences
59 Citations2007Liang Wang, Christopher Leckie +3 more
A tensor space representation for analyzing human activity patterns in monocular videos that can effectively identify human activities with intra- and inter-person variations on both temporal and spatial scales is described.
Parallelized Variational EM for Latent Dirichlet Allocation: An Experimental Evaluation of Speed and Scalability
33 Citations2007Ramesh Nallapati, William W. Cohen +1 more
This work builds parallel implementations of the variational EM algorithm for LDA in a multiproces- sor architecture as well as a distributed setting and indicates that while both the implementations achieve speed-ups, the distributed version achieves dramatic improvements in both speed and scalability.
Memory bounded inference in topic models
12 Citations2008Ryan G. Gomes, Max Welling +1 more
This work addresses the question of what type of algorithms and statistical techniques support learning from very large datasets over long stretches of time through a memory bounded version of a variational EM algorithm that approximates inference in a topic model.