Incorporating domain knowledge into topic modeling via Dirichlet Forest priors

This work incorporates domain knowledge about the composition of words that should have high or low probability in various topics using a novel Dirichlet Forest prior in a LatentDirichlet Allocation framework.

Abstract

Users of topic modeling methods often have knowledge about the composition of words that should have high or low probability in various topics. We incorporate such domain knowledge using a novel Dirichlet Forest prior in a Latent Dirichlet Allocation framework. The prior is a mixture of Dirichlet tree distributions with special structures. We present its construction, and inference via collapsed Gibbs sampling. Experiments on synthetic and real datasets demonstrate our model's ability to follow and generalize beyond user-specified domain knowledge.