LDAvis: A method for visualizing and interpreting topics
Published 1 January 2014Open access
Carson Sievert, Kenneth E. Shirley
Citations1,354
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
LDAvis, a web-based interactive visualization of topics estimated using Latent Dirichlet Allocation that is built using a combination of R and D3, and a novel method for choosing which terms to present to a user to aid in the task of topic interpretation is proposed.
Abstract
We present LDAvis, a web-based interac-tive visualization of topics estimated using Latent Dirichlet Allocation that is built us-ing a combination of R and D3. Our visu-alization provides a global view of the top-ics (and how they differ from each other), while at the same time allowing for a deep inspection of the terms most highly asso-ciated with each individual topic. First, we propose a novel method for choosing which terms to present to a user to aid in the task of topic interpretation, in which we define the relevance of a term to a topic. Second, we present results from a user study that suggest that ranking terms purely by their probability under a topic is suboptimal for topic interpretation. Last, we describe LDAvis, our visualization system that allows users to flexibly explore topic-term relationships using relevance to better understand a fitted LDA model. 1
Keywords
Computer Science
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Proceedings of the National Academy of SciencesFinding scientific topics
6,017 Citations2004Thomas L. Griffiths, Mark Steyvers
A generative model for documents is described, introduced by Blei, Ng, and Jordan, and a Markov chain Monte Carlo algorithm is presented for inference in this model, which is used to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics.
IEEE Transactions on Visualization and Computer GraphicsD³ Data-Driven Documents
3,153 Citations2011M. G. Bostock, V. Ogievetsky +1 more
This work shows how representational transparency improves expressiveness and better integrates with developer tools than prior approaches, while offering comparable notational efficiency and retaining powerful declarative components.
Reading Tea Leaves: How Humans Interpret Topic Models
1,874 Citations2009Jonathan Chang, Sean Gerrish +3 more
New quantitative methods for measuring semantic meaning in inferred topics are presented, showing that they capture aspects of the model that are undetected by previous measures of model quality based on held-out likelihood.
ScholarWorks@UMassAmherst (University of Massachusetts Amherst)Optimizing Semantic Coherence in Topic Models
1,248 Citations2011David Mimno, Hanna Wallach +3 more
A novel statistical topic model based on an automated evaluation metric based on this metric that significantly improves topic quality in a large-scale document collection from the National Institutes of Health (NIH).
Neural Information Processing SystemsHierarchical Topic Models and the Nested Chinese Restaurant Process
938 Citations2003Thomas L. Griffiths, Michael I. Jordan +2 more
A Bayesian approach is taken to generate an appropriate prior via a distribution on partitions that allows arbitrarily large branching factors and readily accommodates growing data collections.
Termite
414 Citations2012Jason Chuang, Christopher D. Manning +1 more
A novel saliency measure for selecting relevant terms and a seriation algorithm that both reveals clustering structure and promotes the legibility of related terms are contributed to Termite, a visual analysis tool for assessing topic model quality.
Interpretation and trust
244 Citations2012Jason Chuang, Daniel Ramage +2 more
A novel similarity measure for text collections based on a notion of "word-borrowing" that arose from an iterative design process and a set of design recommendations that describe how they promote interpretable and trustworthy visual analysis tools.
Proceedings of the International AAAI Conference on Web and Social MediaVisualizing Topic Models
207 Citations2021Allison J. B. Chaney, David Blei
This paper creates a navigator of the documents, allowing users to explore the hidden structure that a topic model discovers, and reveals meaningful patterns in a collection, helping end-users explore and understand its contents in new ways.
Evaluating topic models for digital libraries
205 Citations2010David Newman, Youn Noh +3 more
This large-scale user study includes over 70 human subjects evaluating and scoring almost 500 topics learned from collections from a wide range of genres and domains and shows how scoring model -- based on pointwise mutual information of word-pair using Wikipedia, Google and MEDLINE as external data sources - performs well at predicting human scores.
arXiv (Cornell University)Summarizing topical content with word frequency and exclusivity
158 Citations2012Jonathan Bischof, Edoardo M. Airoldi
Lecture notes in computer scienceTopic Significance Ranking of LDA Generative Models
155 Citations2009Loulwah AlSumait, Daniel Barbará +2 more
This paper presents the first automated unsupervised analysis of LDA models to identify junk topics from legitimate ones, and to rank the topic significance.
ACM Transactions on Intelligent Systems and TechnologyTopicNets
143 Citations2012Brynjar Gretarsson, John O’Donovan +5 more
A discussion of the design and implementation choices for each visual analysis technique is presented, followed by a discussion of three diverse use cases in which TopicNets enables fast discovery of information that is otherwise hard to find.
Topic Model Diagnostics: Assessing Domain Relevance via Topical Alignment
97 Citations2013Jason Chuang, Sonal Gupta +2 more
This work compares 10,000 topic model variants to 200 expert-provided domain concepts, and demonstrates how the framework can inform choices of model parameters, inference algorithms, and intrinsic measures of topical quality.
arXiv (Cornell University)Visualizing Topics with Multi-Word Expressions
90 Citations2009David M. Blei, John Lafferty
A new method for visualizing topics, the distributions over terms that are automatically extracted from large text corpora using latent variable models, based on a language model of arbitrary length expressions, which outperforms the more standard use of $\chi^2$ and likelihood ratio tests.
Topic Models and Metadata for Visualizing Text Corpora
13 Citations2013Justin Snyder, Rebecca Knowles +3 more
A new web-based tool that integrates topics learned from an unsupervised topic model in a faceted browsing experience that the user can manage topics, filter documents by topic and summarize views with metadata and topic graphs is presented.