Semi-Markov Conditional Random Fields for Information Extraction
Published 1 December 2004
Sunita Sarawagi, William W. Cohen
Citations617
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Intuitively, a semi-CRF on an input sequence x outputs a "segmentation" of x, in which labels are assigned to segments rather than to individual elements of xi, and transitions within a segment can be non-Markovian.
Abstract
We describe semi-Markov conditional random fields (semi-CRFs), a conditionally trained version of semi-Markov chains. Intuitively, a semi-CRF on an input sequence x outputs a “segmentation ” of x, in which labels are assigned to segments (i.e., subsequences) of x rather than to individual elements xi of x. Importantly, features for semi-CRFs can measure properties of segments, and transitions within a segment can be non-Markovian. In spite of this additional power, exact learning and inference algorithms for semi-CRFs are polynomial-time—often only a small constant factor slower than conventional CRFs. In experiments on five named entity recognition problems, semi-CRFs generally outperform conventional CRFs. 1
Keywords
Computer Science
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Mathematical ProgrammingOn the limited memory BFGS method for large scale optimization
8,529 Citations1989Dong C. Liu, Jorge Nocedal
The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.
Neural NetworksStacked generalization
7,478 Citations1992David H. Wolpert
The conclusion is that for almost any real-world generalization problem one should use some version of stacked generalization to minimize the generalization error rate.
Artificial IntelligenceBetween MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
3,146 Citations1999Richard S. Sutton, Doina Precup +1 more
It is shown that options enable temporally abstract knowledge and action to be included in the reinforcement learning frame- work in a natural and general way and may be used interchangeably with primitive actions in planning methods such as dynamic pro- gramming and in learning methodssuch as Q-learning.
Discriminative training methods for hidden Markov models
1,889 Citations2002Michael Collins
Experimental results on part-of-speech tagging and base noun phrase chunking are given, in both cases showing improvements over results for a maximum-entropy tagger.
A comparison of string distance metrics for name-matching tasks
1,375 Citations2003William W. Cohen, Pradeep Ravikumar +1 more
This work investigates a number of different metrics proposed by different communities, including edit-distance metrics, fast heuristic string comparators, token-based distance metrics, and hybrid methods, and finds the best-performing method is a hybrid scheme combining a TFIDF weighting scheme with the Jaro-Winkler string-distance scheme.
Shallow parsing with conditional random fields
1,251 Citations2003Fei Sha, Fernando Pereira
This work shows how to train a conditional random field to achieve performance as good as any reported base noun-phrase chunking method on the CoNLL task, and better than any reported single model.
Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons
1,161 Citations2003Andrew McCallum, Wei Li
This work has shown that conditionally-trained models, such as conditional maximum entropy models, handle inter-dependent features of greedy sequence modeling in NLP well.
Dynamic conditional random fields
746 Citations2004Charles Sutton, Khashayar Rohanimanesh +1 more
A comparison of algorithms for maximum entropy parameter estimation
667 Citations2002Robert Malouf
A number of algorithms for estimating the parameters of ME models are considered, including iterative scaling, gradient ascent, conjugate gradient, and variable metric methods.
Discriminative probabilistic models for relational data
637 Citations2002Ben Taskar, Pieter Abbeel +1 more
Exploiting dictionaries in named entity extraction
224 Citations2004William W. Cohen, Sunita Sarawagi
A semi-Markov extraction process is formalized, which is based on sequentially classifying segments of several adjacent words, rather than single words, and provides a more natural formulation of the NER problem than sequential word classification.
Automatic segmentation of text into structured records
212 Citations2001Vinayak Borkar, Kaustubh Deshmukh +1 more
A tool DATAMOLD is described that learns to automatically extract structure when seeded with a small number of training examples and enhances on Hidden Markov Models (HMM) to build a powerful probabilistic model that corroborates multiple sources of information.
Exploiting diverse knowledge sources via maximum entropy in named entity recognition
210 Citations1998Andrew Borthwick, John Sterling +2 more
This paper describes a novel statistical namedentity recognition system built around a maximum entity framework using the framework of maximum entropy theory and utilizing a flexible object-based architecture to make use of an extraordinarily diverse range of knowledge sources in making its tagging decisions.
Hierarchical hidden Markov models for information extraction
142 Citations2003Marios Skounakis, Mark Craven +1 more
This work proposes and evaluates an approach that is based on using hierarchical hidden Markov models to represent the grammatical structure of the sentences being processed, which results in more accurate models than several baseline HMM approaches.
Applied Physics Letters10.1162/153244304322972685
99 Citations2000
An algorithm, RAPIER, that uses pairs of sample documents and filled templates to induce pattern-match rules that directly extract fillers for the slots in the template, and presents encouraging experimental results on two domains.
Semi-Markov Models and Applications
90 Citations2011Jacques Janssen, Nikolaos Limnios
OPAL (Open@LaTrobe) (La Trobe University)Coordination in Teams: Evidence from a Simulated Management Game
43 Citations2018Robert E. Kraut, Susan R. Fussell +2 more
Segmental semi-markov models and applications to sequence analysis
23 Citations2002Xianping Ge, Padhraic Smyth
This dissertation derives inference and learning procedures for the segmental semi-Markov model, and proposes novel learning algorithms and applications within this framework for real-world supervised and unsupervised learning problems.
Relational Markov Networks for Collective Information Extraction
21 Citations2004Razvan Bunescu and Raymond J. Mooney
A new IE method is presented that employs Relational Markov Networks, which can represent arbitrary dependencies between extractions, which allows for “collective information extraction” that exploits the mutual influence between possible extractions.