Solving the problem of cascading errors
Published 1 January 2006Open access
Jenny Rose Finkel, Christopher D. Manning, Andrew Y. Ng
Citations115
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel architecture is presented, which models these pipelines as Bayesian networks, with each low level task corresponding to a variable in the network, and then it is performed approximate inference to find the best labeling.
Abstract
The end-to-end performance of natural language processing systems for compound tasks, such as question answering and textual entailment, is often hampered by use of a greedy 1-best pipeline architecture, which causes errors to propagate and compound at each stage. We present a novel architecture, which models these pipelines as Bayesian networks, with each low level task corresponding to a variable in the network, and then we perform approximate inference to find the best labeling. Our approach is extremely simple to apply but gains the benefits of sampling the entire distribution over labels at each stage in the pipeline. We apply our method to two tasks -- semantic role labeling and recognizing textual entailment -- and achieve useful performance gains from the superior pipeline architecture.
Keywords
Computer Science
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Foundations of statistical natural language processing
9,996 Citations1999Christopher D. Manning, Hinrich Schütze
Accurate unlexicalized parsing
3,055 Citations2003Dan Klein, Christopher D. Manning
It is demonstrated that an unlexicalized PCFG can parse much more accurately than previously shown, by making use of simple, linguistically motivated state splits, which break down false independence assumptions latent in a vanilla treebank grammar.
Lecture notes in computer scienceThe PASCAL Recognising Textual Entailment Challenge
1,626 Citations2006Ido Dagan, Oren Glickman +1 more
A maximum-entropy-inspired parser
1,496 Citations2000Eugene Charniak
A new parser for parsing down to Penn tree-bank style parse trees that achieves 90.1% average precision/recall for sentences of length 40 and less and 89.5% when trained and tested on the previously established sections of the Wall Street Journal treebank is presented.
Introduction to the CoNLL-2005 shared task
649 Citations2005Xavier Carreras, Lluı́s Màrquez
The specification and goals of the task are introduced, the data sets and evaluation methods are described, and a general overview of the 19 systems that have contributed to the task is presented, providing a comparative description and results.
Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling
396 Citations2004Xavier Carreras, Lluı́s Màrquez
The specification and goal of the task are introduced, the data sets and evaluation methods are described, and a general overview of the systems that have contributed to the task is presented, providing comparative description.
Better<i>k</i>-best parsing
300 Citations2005Liang Huang, David Chiang
It is shown how the improved output of the efficient algorithms for k-best trees in the framework of hypergraph parsing has the potential to improve results from parse reranking systems and other applications.
Joint learning improves semantic role labeling
182 Citations2005Kristina Toutanova, Aria Haghighi +1 more
This work shows how to build a joint model of argument frames, incorporating novel features that model these interactions into discriminative log-linear models, achieving error reduction of 22% on all arguments and 32% on core arguments over a state-of-the art independent classifier for gold-standard parse trees on PropBank.
Learning to recognize features of valid textual entailments
140 Citations2006Bill MacCartney, Trond Grenager +3 more
This paper advocates a new architecture for textual inference in which finding a good alignment is separated from evaluating entailment, and proposes a pipelined approach where alignment is followed by a classification step, in which features represent high-level characteristics of the entailment problem.
Parsing inside-out
134 Citations1998Stuart M. Shieber, Joshua Goodman
The Probabilistic Feature Grammar formalism is presented, which achieves state of the art accuracy, and can compute inside and outside probabilities of probabilistic Context-Free Grammars.
Generalized inference with multiple semantic role labeling systems
130 Citations2005P. Koomen, Vasin Punyakanok +2 more
An approach to semantic role labeling (SRL) that takes the output of multiple argument classifiers and combines them into a coherent predicate-argument output by solving an optimization problem is presented.
ScholarWorks@UMassAmherst (University of Massachusetts Amherst)An integrated, conditional model of information extraction and coreference with application to citation matching
126 Citations2004Ben Wellner, Andrew McCallum +2 more
A Method for Disjunctive Constraint Satisfaction
95 Citations1991John T. Maxwell, Ronald M. Kaplan
A distinctive property of many current grammatical formalisms is their use of feature equality constraints to express a wide variety of grammatical dependencies.
A joint model for semantic role labeling
56 Citations2005Aria Haghighi, Kristina Toutanova +1 more
A semantic role labeling system submitted to the closed track of the CoNLL-2005 shared task implements a joint model that captures dependencies among arguments of a predicate using log-linear models in a discriminative re-ranking framework.
Packed rewriting for mapping semantics to kr
56 Citations2005Dick Crouch
The prime goal of this paper is to show how the free-choice packing mechanisms developed for managing ambiguity in non-context free parsing can profitably be extended beyond syntactic representations.
Joint parsing and semantic role labeling
56 Citations2005Charles Sutton, Andrew McCallum
This paper jointly performs parsing and semantic role labeling, using a probabilistic SRL system to rerank the results of a ProbabilisticParser, because a locally-trained SRL model can return inaccurate probability estimates.
Dynamic programming for parsing and estimation of stochastic unification-based grammars
54 Citations2001Stuart Geman, Mark Johnson
A graph-based dynamic programming algorithm for calculating statistics from the packed UBG parse representations of Maxwell and Kaplan (1995) which does not require enumerating all parses.
Lecture notes in computer scienceApplications of Finite-State Transducers in Natural Language Processing
48 Citations2001Lauri Karttunen
This paper is a review of some of the major applications of finite- state transducers in Natural Language Processing ranging from morphological analysis to finite-state parsing.
Joint Parsing and Semantic Role Labeling
22 Citations2005Charles Sutton, Andrew McCallum
The problem of computing the most probable tree in data-oriented parsing and stochastic tree grammars
20 Citations1995Rens Bod
It is shown that in STSG, in contrast with stochastic context-free grammar, the Viterbi algorithm cannot be used for computing a most probable tree of a string, and proposes a simple modification ofViterbi which allows by means of a "select-random" search to estimate the most probableTree of astring in polynomial time.
Convergence rates of the Voting Gibbs classifier, with application to Bayesian feature selection
19 Citations2001Andrew Y. Ng, Michael I. Jordan
The Voting Gibbs classiier is studied, which is the extension of this scheme to the full Monte Carlo setting, in which N samples are drawn from the posterior and new inputs are classiied by voting the N resulting classiiers.
Effective statistical models for syntactic and semantic disambiguation
9 Citations2005Christopher D. Manning, Kristina Toutanova
A novel representation of parse trees that connects the words of the sentence with the hidden syntactic structure in a direct way and a more accurate model, which captures the knowledge that the semantic frame of a verb is a joint structure with strong dependencies between arguments, are proposed.