Relational Topic Models for Document Networks
Published 15 April 2009
Jonathan Chang, David M. Blei
Citations485
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The relational topic model (RTM) is developed, a model of documents and the links between them that derives efficient inference and learning algorithms based on variational methods and evaluates the predictive performance of the RTM for large networks of scientific abstracts and web documents.
Abstract
We develop the relational topic model (RTM), a model of documents and the links between them. For each pair of documents, the RTM models their link as a binary random variable that is conditioned on their contents. The model can be used to summarize a network of documents, predict links between them, and predict words within them. We derive efficient inference and learning algorithms based on variational methods and evaluate the predictive performance of the RTM for large networks of scientific abstracts and web documents. 1
Keywords
Computer SciencePhysics and Astronomy
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Journal of the ACMAuthoritative sources in a hyperlinked environment
9,060 Citations1999Jon Kleinberg
This work proposes and test an algorithmic formulation of the notion of authority, based on the relationship between a set of relevant authoritative pages and the set of “hub pages” that join them together in the link structure, and has connections to the eigenvectors of certain matrices associated with the link graph.
Machine LearningAn Introduction to Variational Methods for Graphical Models
3,775 Citations1999Michael I. Jordan, Zoubin Ghahramani +2 more
This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields), and describes a general framework for generating variational transformations based on convex duality.
Journal of the American Statistical AssociationLatent Space Approaches to Social Network Analysis
1,899 Citations2002Peter D. Hoff, Adrian E. Raftery +1 more
This work develops a class of models where the probability of a relation between actors depends on the positions of individuals in an unobserved “social space,” and proposes Markov chain Monte Carlo procedures for making inference on latent positions and the effects of observed covariates.
PubMedMixed Membership Stochastic Blockmodels.
1,526 Citations2008Edoardo M. Airoldi, David M. Blei +2 more
This paper describes a latent variable model of such data called the mixed membership stochastic blockmodel, which extends blockmodels for relational data to ones which capture mixed membership latent relational structure, thus providing an object-specific low-dimensional representation.
arXiv (Cornell University)Supervised Topic Models
1,315 Citations2010David M. Blei, Jon McAuliffe
The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.
Information RetrievalAutomating the Construction of Internet Portals with Machine Learning
1,265 Citations2000Andrew Kachites McCallum, Kamal Nigam +2 more
New research in reinforcement learning, information extraction and text classification that enables efficient spidering, the identification of informative text segments, and the population of topic hierarchies are described.
PsychometrikaLogit Models and Logistic Regressions for Social Networks: I. An Introduction to Markov Graphs and <i>p</i>
1,264 Citations1996Stanley Wasserman, Philippa Pattison
A large class of models, including several generalizations of stochastic block models, as well as models parameterizing global tendencies towards clustering and centralization, and individual differences in such tendencies are described and extended.
Modeling annotated data
1,070 Citations2003David M. Blei, Michael I. Jordan
Three hierarchical probabilistic mixture models which aim to describe annotated data with multiple types, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type.
Learning to extract symbolic knowledge from the World Wide Web
675 Citations1998Mark Craven, Dan DiPasquo +5 more
The goal of the research described here is to automatically create a computer understandable world wide knowledge base whose content mirrors that of the World Wide Web, and several machine learning algorithms for this task are described.
Link Prediction in Relational Data
442 Citations2003Ben Taskar, Ming-fai Wong +2 more
It is shown that the collective classification approach of RMNs, and the introduction of subgraph patterns over link labels, provide significant improvements in accuracy over flat classification, which attempts to predict each link in isolation.
The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity
439 Citations2000David Cohn, Thomas Hofmann
A joint probabilistic model for modeling the contents and inter-connectivity of document collections such as sets of web pages or research paper archives is described, based on a Probabilistic factor decomposition.
Joint latent topic models for text and citations
413 Citations2008Ramesh Nallapati, Amr Ahmed +2 more
This work addresses the problem of joint modeling of text and citations in the topic modeling framework with two different models called the Pairwise-Link-LDA and the Link-PLSA-Lda models, which combine the LDA and PLSA models into a single graphical model.
Topic modeling with network regularization
396 Citations2008Qiaozhu Mei, Deng Cai +2 more
A novel solution to the problem of topic modeling with network structure (TMN) is proposed, which regularizes a statistical topic model with a harmonic regularizer based on a graph structure in the data.
Physical Review LettersBayesian Approach to Network Modularity
308 Citations2008Jake M. Hofman, Chris H. Wiggins
This work presents an efficient, principled, and interpretable technique for inferring module assignments and for identifying the optimal number of modules in a given network, implemented using a variational technique developed only in the past decade.
ScholarWorks@UMassAmherst (University of Massachusetts Amherst)Topic and role discovery in social networks
291 Citations2005Andrew McCallum, Andrés Corrada-Emmanuel +1 more
Computer Physics CommunicationsThe structure and function of networks
269 Citations2002M. E. J. Newman
The author reviews some of the interesting issues in this area of characterization and modeling of networks and recounts some recent work on these issues by himself and by others.
Unsupervised prediction of citation influences
240 Citations2007Laura Dietz, Steffen Bickel +1 more
A probabilistic topic model is devised that explains the generation of documents and incorporates the aspects of topical innovation and topical inheritance via citations, and its ability to predict the strength of influence of citations against manually rated citations is evaluated.
Learning Probabilistic Models of Relational Structure
217 Citations2001Lise Getoor, Nir Friedman +2 more
Journal of the American Statistical AssociationVariational Inference for Large-Scale Models of Discrete Choice
174 Citations2010Michael Braun, Jon McAuliffe
Extensive simulations, along with an analysis of real-world data, demonstrate that variational methods achieve accuracy competitive with Markov chain Monte Carlo at a small fraction of the computational cost.
Proceedings of the International AAAI Conference on Web and Social MediaLink-PLSA-LDA: A New Unsupervised Model for Topics and Influence of Blogs
125 Citations2021Ramesh Nallapati, William W. Cohen
A new model that can be used to provide a user with highly influential blog postings on the topic of the user's interest is proposed, called Link-PLSA-LDA, that combines PLSA and LDA into a single framework, and explicitly models the topical relationship between the linking and the linked document.
Group and topic discovery from relations and text
122 Citations2005Xuerui Wang, Natasha Mohanty +1 more
A probabilistic generative model of entity relationships and textual attributes that simultaneously discovers groups among the entities and topics among the corresponding text and shows that in comparison with traditional, separate latent-variable models for words or Blockstructures for votes, the Group-Topic model's joint inference improves both the groups and topics discovered.
Discovering Latent Classes in Relational Data
82 Citations2004Charles Kemp, Thomas L. Griffiths +1 more
A domain-general framework for learning abstract relational knowledge, which goes beyond previous category-learning models in psychology and applies in two specific domains: learning the structure of kinship systems and learning causal theories.
arXiv (Cornell University)Component models for large networks
22 Citations2008Janne Sinkkonen, Janne Aukia +1 more
This work introduces an alternative, interaction component model for communities (ICMc), where the whole network is a bag of links, stemming from different components, which assumes assortativity and finds community-like structures like the earlier methods motivated by physics.