Phylogenetic Grammar Induction
Published 11 July 2010
Taylor Berg-Kirkpatrick, Dan Klein
Citations60
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Joint induction in the multilingual model substantially outperforms independent learning, with larger gains both from more articulated phylogenies and as well as from increasing numbers of languages.
Abstract
We present an approach to multilingual grammar induction that exploits a phylogeny-structured model of parameter drift. Our method does not require any translated texts or token-level alignments. Instead, the phylogenetic prior couples languages at a parameter level. Joint induction in the multilingual model substantially outperforms independent learning, with larger gains both from more articulated phylogenies and as well as from increasing numbers of languages. Across eight languages, the multilingual approach gives error reductions over the standard monolingual DMV averaging 21.1 % and reaching as high as 39%. 1
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
Mathematical ProgrammingOn the limited memory BFGS method for large scale optimization
8,529 Citations1989Dong C. Liu, Jorge Nocedal
The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.
Building a Large Annotated Corpus of English: The Penn Treebank
7,528 Citations1993Mitchell P. Marcus
Computational LinguisticsHead-Driven Statistical Models for Natural Language Parsing
1,860 Citations2003Michael Collins
Three statistical models for natural language parsing are described, leading to approaches in which a parse tree is represented as the sequence of decisions corresponding to a head-centered, top-down derivation of the tree.
ArXiv.orgFrustratingly Easy Domain Adaptation
1,395 Citations2009Hal Daumé
This work describes an approach to domain adaptation that is appropriate exactly in the case when one has enough “target” data to do slightly better than just using only “source’ data.
CoNLL-X shared task on multilingual dependency parsing
976 Citations2006Sabine Buchholz, Erwin Marsi
How treebanks for 13 languages were converted into the same dependency format and how parsing performance was measured is described and general conclusions about multi-lingual parsing are drawn.
Corpus-based induction of syntactic structure
453 Citations2004Dan Klein, Christopher D. Manning
This work presents a generative model for the unsupervised learning of dependency structures and describes the multiplicative combination of this dependency model with a model of linear constituency that works and is robust cross-linguistically.
Painless Unsupervised Learning with Features
224 Citations2010Taylor Berg-Kirkpatrick, Alexandre Bouchard‐Côté +2 more
This work shows how features can easily be added to standard generative models for unsupervised learning, without requiring complex new training methods, and applies this technique to part-of-speech induction, grammar induction, word alignment, and word segmentation.
Hierarchical Bayesian domain adaptation
175 Citations2009Jenny Rose Finkel, Christopher D. Manning
It is demonstrated that allowing different values for these hyperparameters significantly improves performance over both a strong baseline and (Daume III, 2007) within both a conditional random field sequence model for named entity recognition and a discriminatively trained dependency parser.
Parameter estimation for probabilistic finite-state transducers
167 Citations2001Jason Eisner
A "parameterized FST" paradigm is formulated and training algorithms for it are given, including a general bookkeeping trick ("expectation semirings") that cleanly and efficiently computes expectations and gradients.
Building a large-scale annotated Chinese corpus
135 Citations2002Nianwen Xue, Fu-Dong Chiou +1 more
This paper addresses issues related to building a large-scale Chinese corpus and tries to answer four questions: how to speed up annotation, how to maintain high annotation quality, for what purposes is the corpus applicable, and finally what future work the authors anticipate.
Dependency grammar induction via bitext projection constraints
129 Citations2009Kuzman Ganchev, Jennifer Gillenwater +1 more
This work considers generative and discriminative models for dependency grammar induction that use word-level alignments and a source language parser to constrain the space of possible target trees and evaluates the approach on Bulgarian and Spanish CoNLL shared task data and shows it consistently outperform unsupervised methods and can outperform supervised learning for limited training data.
Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction
124 Citations2009Shay B. Cohen, Noah A. Smith
A family of priors over probabilistic grammar weights is presented, called the shared logistic normal distribution, enabling factored covariance between the probabilities of different derivation events in the probabilism grammar, providing a new way to encode prior knowledge about an unknown grammar.
Optimization with EM and expectation-conjugate-gradient
122 Citations2003Ruslan Salakhutdinov, Sam T. Roweis +1 more
Two novel algorithms for maximum likelihood estimation of latent variable models are proposed and empirical results showing that they can substantially outperform standard EM in terms of speed of convergence in certain cases are reported.
Two languages are better than one (for syntactic parsing)
101 Citations2008David Burkett, Dan Klein
It is shown that jointly parsing a bitext can substantially improve parse quality on both sides, and the resulting bitext parser outperforms state-of-the-art monolingual parser baselines in a maximum entropy bitext parsing model.
Two statistical parsing models applied to the Chinese Treebank
86 Citations2000Daniel M. Bikel, David Chiang
The first-ever results of applying statistical parsing models to the newly-available Chinese Treebank are presented, with the former performing at 69% precision, 75% recall, and the latter at 77% precision and 78% recall.
Unsupervised multilingual grammar induction
81 Citations2009Benjamin Snyder, Tahira Naseem +1 more
A generative Bayesian model is formulated which seeks to explain the observed parallel data through a combination of bilingual and monolingual parameters, and loosely binds parallel trees while allowing language-specific syntactic structure.
Parser adaptation and projection with quasi-synchronous grammar features
69 Citations2009David A. Smith, Jason Eisner
This work connects two scenarios in structured learning: adapting a parser trained on one corpus to another annotation style, and projecting syntactic annotations from one language to another, and proposes quasi-synchronous grammar (QG) features for these structured learning tasks.
Experiments in parallel-text based grammar induction
52 Citations2004Jonas Kuhn
This paper discusses the use of statistical word alignment over multiple parallel texts for the identification of string spans that cannot be constituents in one of the languages, and presents experimental results with an evaluation against the Penn Tree-bank.
Empirical Methods in Natural Language ProcessingA Probabilistic Approach to Diachronic Phonology
49 Citations2007Alexandre Bouchard, Percy Liang +2 more
A probabilistic model of diachronic phonology in which individual word forms undergo stochastic edits along the branches of a phylogenetic tree is presented and results validating the model are presented.
Agreement-Based Learning
40 Citations2007Percy Liang, Dan Klein +1 more
An objective function is proposed for the approach, EM-style algorithms for parameter estimation are derived, and their effectiveness on three challenging real-world learning tasks is demonstrated.
Adding more languages improves unsupervised multilingual part-of-speech tagging
37 Citations2009Benjamin Snyder, Tahira Naseem +2 more
A non-parametric Bayesian model is proposed that connects related tagging decisions across languages through the use of multilingual latent variables and shows that performance improves steadily as the number of languages increases.