Parsing the WSJ using CCG and log-linear models
Published 1 January 2004Open access
Stephen Clark, James Curran
Citations279
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A parallel implementation of the L-BFGS optimisation algorithm is described, which runs on a Beowulf cluster allowing the complete Penn Treebank to be used for estimation and a new efficient parsing algorithm for CCG which maximises expected recall of dependencies is developed.
Abstract
This paper describes and evaluates log-linear parsing models for Combinatory Categorial Grammar (CCG). A parallel implementation of the L-BFGS optimisation algorithm is described, which runs on a Beowulf cluster allowing the complete Penn Treebank to be used for estimation. We also develop a new efficient parsing algorithm for CCG which maximises expected recall of dependencies. We compare models which use all CCG derivations, including non-standard derivations, with normal-form models. The performances of the two models are comparable and the results are competitive with existing wide-coverage CCG parsers.
Keywords
Computer Science
Numerical Optimization
9,247 Citations1999Jorge Nocedal, Stephen J. Wright
Computational LinguisticsA maximum entropy approach to natural language processing
3,120 Citations1996Adam Berger, Vincent J. Della Pietra +1 more
A maximum-likelihood approach for automatically constructing maximum entropy models is presented and how to implement this approach efficiently is described, using as examples several problems in natural language processing.
The MIT Press eBooksThe Syntactic Process
1,610 Citations2000Mark Steedman
The book covers topics in formal linguistics, intonational phonology, computational linguistics and experimental psycholinguistics, presenting them as an integrated theory of the language faculty in a form accessible to readers from any of those fields.
Shallow parsing with conditional random fields
1,251 Citations2003Fei Sha, Fernando Pereira
This work shows how to train a conditional random field to achieve performance as good as any reported base noun-phrase chunking method on the CoNLL task, and better than any reported single model.
A new statistical parser based on bigram lexical dependencies
619 Citations1996Michael Collins
A new statistical parser which is based on probabilities of dependencies between head-words in the parse tree, which trains on 40,000 sentences in under 15 minutes and can be improved to over 200 sentences a minute with negligible loss in accuracy.
A Gaussian Prior for Smoothing Maximum Entropy Models
347 Citations1999Stanley F. Chen, Ronald Rosenfeld
Over a large number of data sets, it is found that an ME smoothing method proposed to us by Lafferty performs as well as or better than all other algorithms under consideration.
Parsing the wall street journal using a Lexical-Functional Grammar and discriminative estimation techniques
280 Citations2001Stefan Riezler, Tracy Holloway King +4 more
The model combines full and partial parsing techniques to reach full grammar coverage on unseen data, and on a gold standard of manually annotated f-structures for a subset of the WSJ treebank, reaches 79% F-score.
Corpus Variation and Parser Performance
273 Citations2001Daniel Gildea
This work examines how other types of text might affect parser performance, and how portable parsing models are across corpora by comparing results for the Brown and WSJ corpora, and considers which parts of the parser’s probability model are particularly tuned to the corpus on which it was trained.
Generative models for statistical parsing with Combinatory Categorial Grammar
209 Citations2001Julia Hockenmaier, Mark Steedman
This paper compares a number of generative probability models for a wide-coverage Combinatory Categorial Grammar (CCG) parser and finds a significant improvement from modeling word-word dependencies.
Parsing algorithms and metrics
139 Citations1996Joshua Goodman
Two new algorithms are presented: the "Labelled Recall Algorithm," which maximizes the expected Labelled Recall Rate, and the "Bracketed Recall Al algorithm", which maximized the Bracketed recall Rate.
The importance of supertagging for wide-coverage CCG parsing
138 Citations2004Stephen Clark, James Curran
This paper describes the role of supertagging in a wide-coverage CCG parser which uses a log-linear model to select an analysis and shows that large increases in speed can be obtained by tightly integrating the supertagger with the CCG grammar and parser.
Investigating GIS and smoothing for maximum entropy taggers
102 Citations2003James Curran, Stephen Clark
It is shown analytically and empirically that the correction feature, assumed to be required for the correctness of GIS, is unnecessary, and the use of a Gaussian prior and a simple cutoff for smoothing is explored.
Building deep dependency structures with a wide-coverage CCG parser
101 Citations2001Stephen Clark, Julia Hockenmaier +1 more
A wide-coverage statistical parser that uses Combinatory Categorial Grammar (CCG) to derive dependency structures and correctly recovers over 80% of labelled dependencies, and around 90% of unlabelled dependencies.
Maximum entropy estimation for feature forests
100 Citations2002Yusuke Miyao, Tsujii Jun'ichi
An algorithm is proposed for maximum entropy modeling that enables probabilistic modeling of complete structures, such as transition sequences in Markov models and parse trees, without dividing them into independent sub-events.
Efficient normal-form parsing for combinatory categorial grammar
99 Citations1996Jason Eisner
The parser is proved to find exactly one parse in each semantic equivalence class of allowable parses; that is, spurious ambiguity is shown to be both safely and completely eliminated.
Log-linear models for wide-coverage CCG parsing
72 Citations2003Stephen Clark, James Curran
This paper describes log-linear parsing models for Combinatory Categorial Grammar and describes a parellelised implementation which runs on a Beowulf cluster and allows the complete WSJ Penn Treebank to be used for estimation.
Parsing with generative models of predicate-argument structure
64 Citations2003Julia Hockenmaier
This paper argues that probabilistic parsers should model the dependencies in the predicate-argument structure, as in the model of Clark et al. (2002), and defines a generative model for CCG derivations that captures these dependencies, including bounded and unbounded long-range dependencies.
Parse Disambiguation for a Rich HPSG Grammar
46 Citations2002Kristina Toutanova, Christopher D. Manning +3 more
This paper explores building probabilistic models for parse disambiguation using the Redwoods HPSG treebank, and describes generative and discriminative models using analogous features and compares their performance on the disambigsuation task.