Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression

arXiv (Cornell University)Published 13 June 2012Open access

David Mimno, Andrew McCallum

DOI10.48550/arxiv.1206.3278

Citations298

Abstract

Although fully generative models have been successfully used to model the contents of text documents, they are often awkward to apply to combinations of text data and document metadata. In this paper we propose a Dirichlet-multinomial regression (DMR) topic model that includes a log-linear prior on document-topic distributions that is a function of observed features of the document, such as author, publication venue, references, and dates. We show that by selecting appropriate features, DMR topic models can meet or exceed the performance of several previously published topic models designed for specific data.

Keywords

Computer Science

Journal of Machine Learning ResearchLatent dirichlet allocation

27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more

Mathematical ProgrammingOn the limited memory BFGS method for large scale optimization

8,529 Citations1989Dong C. Liu, Jorge Nocedal

The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.

arXiv (Cornell University)Supervised Topic Models

1,315 Citations2010David M. Blei, Jon McAuliffe

The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.

View PDF

Uncertainty in Artificial IntelligenceThe author-topic model for authors and documents

1,313 Citations2004Michal Rosen‐Zvi, Thomas Griffiths +2 more

Topics over time

1,132 Citations2006Xuerui Wang, Andrew McCallum

An LDA-style topic model is presented that captures not only the low-dimensional structure of data, but also how the structure changes over time, showing improved topics, better timestamp prediction, and interpretable trends.

Modeling annotated data

1,070 Citations2003David M. Blei, Michael I. Jordan

Three hierarchical probabilistic mixture models which aim to describe annotated data with multiple types, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type.

Topic modeling

1,057 Citations2006Hanna Wallach

A hierarchical generative probabilistic model that incorporates both n-gram statistics and latent topic variables by extending a unigram topic model to include properties of a hierarchical Dirichlet bigram language model is explored.

A comparison of statistical significance tests for information retrieval evaluation

717 Citations2007Mark D. Smucker, James Allan +1 more

It is discovered that there is little practical difference between the randomization, bootstrap, and t tests and their use should be discontinued for measuring the significance of a difference between means.

View PDF

Pachinko allocation

612 Citations2006Wei Li, Andrew McCallum

Improved performance of PAM is shown in document classification, likelihood of held-out data, the ability to support finer-grained topics, and topical keyword coherence.

Proceedings of the National Academy of SciencesMixed-membership models of scientific publications

445 Citations2004Elena A. Erosheva, Stephen E. Fienberg +1 more

This work explores an internal soft-classification structure of articles based only on semantic decompositions of abstracts and bibliographies of PNAS articles, and compares it with the formal discipline classifications.