Information Extraction with HMM Structures Learned by Stochastic Optimization

This paper demonstrates that extraction accuracy strongly depends on the selection of structure, and presents an algorithm for automatically finding good structures by stochastic optimization, which finds HMM models that almost always out-perform a fixed model, and have superior average performance across tasks.

Abstract

Recent research has demonstrated the strong performance of hidden Markov models applied to information extrac-tion—the task of populating database slots with correspond-ing phrases from text documents. A remaining problem, however, is the selection of state-transition structure for the model. This paper demonstrates that extraction accuracy strongly depends on the selection of structure, and presents an algorithm for automatically finding good structures by stochastic optimization. Our algorithm begins with a simple model and then performs hill-climbing in the space of pos-sible structures by splitting states and gauging performance on a validation set. Experimental results show that this tech-nique finds HMM models that almost always out-perform a fixed model, and have superior average performance across tasks.