A Minimal Span-Based Neural Constituency Parser
Published 1 January 2017Open access
Mitchell Stern, Jacob Andreas, Dan Klein
Citations201
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents a minimal neural model for constituency parsing based on independent scoring of labels and spans that is compatible with classical dynamic programming techniques, but also admits a novel greedy top-down inference algorithm based on recursive partitioning of the input.
Abstract
In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans. We show that this model is not only compatible with classical dynamic programming techniques, but also admits a novel greedy top-down inference algorithm based on recursive partitioning of the input. We demonstrate empirically that both prediction schemes are competitive with recent work, and when combined with basic extensions to the scoring model are capable of achieving state-of-the-art single-model performance on the Penn Treebank (91.79 F1) and strong performance on the French Treebank (82.23 F1).
Keywords
Computer Science
UvA-DARE (University of Amsterdam)Adam: A Method for Stochastic Optimization
84,783 Citations2014Diederik P. Kingma, Jimmy Ba
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)
50,318 Citations2021Mandi, Jayanta, Canoy, Rocsildes +2 more
A simple numeric simulation of DNA-co-polymerized hydrogel shape change and a genetic algorithm that generates and selects large batches of material designs that compete with one another to evolve and converge on optimal objective-matching designs are constructed.
arXiv (Cornell University)Neural Machine Translation by Jointly Learning to Align and Translate
14,565 Citations2014Dzmitry Bahdanau
Understanding the difficulty of training deep feedforward neural networks
12,692 Citations2010Xavier Glorot, Yoshua Bengio
The objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future.
Building a Large Annotated Corpus of English: The Penn Treebank
7,528 Citations1993Mitchell P. Marcus
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)LOL: An Investigation into Cybernetic Humor, or: Can Machines Laugh?
3,084 Citations2016Alex Graves, Gervasi, Vincenzo +1 more
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.
Accurate unlexicalized parsing
3,055 Citations2003Dan Klein, Christopher D. Manning
It is demonstrated that an unlexicalized PCFG can parse much more accurately than previously shown, by making use of simple, linguistically motivated state splits, which break down false independence assumptions latent in a vanilla treebank grammar.
Feature-rich part-of-speech tagging with a cyclic dependency network
2,851 Citations2003Kristina Toutanova, Dan Klein +2 more
A new part-of-speech tagger is presented that demonstrates the following ideas: explicit use of both preceding and following tag contexts via a dependency network representation, broad use of lexical features, and effective use of priors in conditional loglinear models.
A Fast and Accurate Dependency Parser using Neural Networks
1,879 Citations2014Danqi Chen, Christopher D. Manning
This work proposes a novel way of learning a neural network classifier for use in a greedy, transition-based dependency parser that can work very fast, while achieving an about 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets.
Computational LinguisticsHead-Driven Statistical Models for Natural Language Parsing
1,860 Citations2003Michael Collins
Three statistical models for natural language parsing are described, leading to approaches in which a parse tree is represented as the sequence of decisions corresponding to a head-centered, top-down derivation of the tree.
arXiv (Cornell University)Universal Dependencies for Multilingual Open Information Extraction
704 Citations2021Timothy Dozat, Christopher D. Manning
This paper uses a larger but more thoroughly regularized parser than other recent BiLSTM-based approaches, with biaffine classifiers to predict arcs and labels, and shows which hyperparameter choices had a significant effect on parsing accuracy, allowing it to achieve large gains over other graph-based approach.
Improved Inference for Unlexicalized Parsing
610 Citations2007Slav Petrov, Dan Klein
A novel coarse-to-fine method in which a grammar’s own hierarchical projections are used for incremental pruning, including a method for efficiently computing projections of a grammar without a treebank is presented.
Transactions of the Association for Computational LinguisticsSimple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
593 Citations2016Eliyahu Kiperwasser, Yoav Goldberg
The effectiveness of the BiLSTM approach is demonstrated by applying it to a greedy transition-based parser as well as to a globally optimized graph-basedparser.
arXiv (Cornell University)Deep Biaffine Attention for Neural Dependency Parsing
552 Citations2016Timothy Dozat, Christopher D. Manning
Learning structured prediction models
462 Citations2005Ben Taskar, Vassil Chatalbashev +2 more
This work considers large margin estimation in a broad range of prediction models where inference involves solving combinatorial optimization problems, for example, weighted graph-cuts or matchings, and relies on the expressive power of convex optimization problems to compactly capture inference or solution optimality in structured prediction models.
Sequence-to-Sequence Learning as Beam-Search Optimization
438 Citations2016Sam Wiseman, Alexander M. Rush
This work introduces a model and beam-search training scheme, based on the work of Daume III and Marcu (2005), that extends seq2seq to learn global sequence scores and shows that this system outperforms a highly-optimized attention-basedseq2seq system and other baselines on three different sequence to sequence tasks: word ordering, parsing, and machine translation.
arXiv (Cornell University)Grammar as a Foreign Language
403 Citations2014Oriol Vinyals, Łukasz Kaiser +4 more
arXiv (Cornell University)DyNet: The Dynamic Neural Network Toolkit
344 Citations2017Graham Neubig, Chris Dyer +23 more
DyNet is a toolkit for implementing neural network models based on dynamic declaration of network structure that has an optimized C++ backend and lightweight graph representation and is designed to allow users to implement their models in a way that is idiomatic in their preferred programming language.
Efficient, Feature-based, Conditional Random Field Parsing
180 Citations2008Jenny Rose Finkel, Alex Kleeman +1 more
This work presents the first general, featurerich discriminative parser, based on a conditional random field model, which has been successfully scaled to the full WSJ parsing data, and achieves state-of-the-art results.
Graph-based Dependency Parsing with Bidirectional LSTM
168 Citations2016Wenhui Wang, Baobao Chang
A neural network model for graph-based dependency parsing which utilizes Bidirectional LSTM (BLSTM) to capture richer contextual information instead of using high-order factorization, and enable the model to use much fewer features than previous work.
Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
119 Citations2016James Cross, Liang Huang
A new shift-reduce system whose stack contains merely sentence spans, represented by a bare minimum of LSTM features, which is the first provably optimal dynamic oracle for constituency parsing, which runs in amortized O(1) time, compared to O(n^3) oracles for standard dependency parsing.
Discriminative training of a neural network statistical parser
110 Citations2004James Henderson
It is shown how a parser can be trained with a discriminative learning method while still parameterizing the problem according to a generative probability model, resulting in state-of-the-art levels of performance.
Transactions of the Association for Computational LinguisticsTraining Deterministic Parsers with Non-Deterministic Oracles
108 Citations2013Yoav Goldberg, Joakim Nivre
Experimental evaluation on a wide range of data sets clearly shows that using dynamic oracles to train greedy parsers gives substantial improvements in accuracy, unlike other techniques like beam search.
Neural CRF Parsing
94 Citations2015Greg Durrett, Dan Klein
A parsing model that combines the exact dynamic programming of CRF parsing with the rich nonlinear featurization of neural net approaches, using nonlinear potentials computed via a feedforward neural network.
Introducing the SPMRL 2014 Shared Task on Parsing Morphologically-rich Languages
93 Citations2014Djamé Seddah, Sandra Kübler +1 more
This paper provides a short overview of the 2014 SPMRL shared task goals, data sets, and evaluation setup and describes the description of participating systems and the analysis of their results as part of (Seddah et al., 2014).
arXiv (Cornell University)A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
74 Citations2015Peilu Wang, Yao Qian +3 more
This work proposes to use BLSTM-RNN for a unified tagging solution that can be applied to various tagging tasks including part-of-speech tagging, chunking and named entity recognition, requiring no task specific knowledge or sophisticated feature engineering.
Less Grammar, More Features
64 Citations2014David Hall, Greg Durrett +1 more
This work presents a parser that relies primarily on extracting information directly from surface spans rather than on propagating information through enriched grammar structure, and shows that, in both syntactic parsing and sentiment analysis, many broad linguistic trends can be captured via surface features.
Training with Exploration Improves a Greedy Stack LSTM Parser
58 Citations2016Miguel Ballesteros, Yoav Goldberg +2 more
This work adapts the greedy Stack-LSTM dependency parser to support a training-with-exploration procedure using dynamic oracles instead of cross-entropy minimization, which improves parsing accuracies for both English and Chinese.
Transactions of the Association for Computational LinguisticsShift-Reduce Constituent Parsing with Neural Lookahead Features
49 Citations2017Jiangming Liu, Yue Zhang
A bidirectional LSTM model is built, which leverages full sentence information to predict the hierarchy of constituents that each word starts and ends and gives the highest reported accuracies for fully-supervised parsing.
Optimal Shift-Reduce Constituent Parsing with Structured Perceptron
11 Citations2015Le Quang Thang, Hiroshi Noji +1 more
A constituent shift-reduce parser with a structured perceptron that finds the optimal parse in a practical runtime and achieves 91.1 F1 on a standard English experiment, a level which cannot be reached by other beam-based systems even with large beam sizes.