A Dataset of Syntactic-Ngrams over Time from a Very Large Corpus of English Books
Published 1 June 2013
Yoav Goldberg, Jon Orwant
Citations135
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A dataset of syntactic-ngrams (counted dependency-tree fragments) based on a corpus of 3.5 million English books includes temporal information, facilitating new kinds of research into lexical semantics over time.
Abstract
We created a dataset of syntactic-ngrams (counted dependency-tree fragments) based on a corpus of 3.5 million English books. The dataset includes over 10 billion distinct items covering a wide range of syntactic configurations. It also includes temporal information, facilitating new kinds of research into lexical semantics over time. This paper describes the dataset, the syntactic representation, and the kinds of information provided. 1
Keywords
Computer Science
Building a Large Annotated Corpus of English: The Penn Treebank
7,528 Citations1993Mitchell P. Marcus
ScienceQuantitative Analysis of Culture Using Millions of Digitized Books
3,107 Citations2010Jean-Baptiste Michel, Yuan Shen +12 more
This work surveys the vast terrain of ‘culturomics,’ focusing on linguistic and cultural phenomena that were reflected in the English language between 1800 and 2000, and shows how this approach can provide insights about fields as diverse as lexicography, the evolution of grammar, collective memory, the adoption of technology and the pursuit of fame.
Journal of Artificial Intelligence ResearchFrom Frequency to Meaning: Vector Space Models of Semantics
2,883 Citations2010Peter D. Turney, Patrick Pantel
The goal in this survey is to show the breadth of applications of VSMs for semantics, to provide a new perspective on VSMs, and to provide pointers into the literature for those who are less familiar with the field.
Automatic retrieval and clustering of similar words
1,555 Citations1998Dekang Lin
A word similarity measure based on the distributional pattern of words allows the automatically constructed thesaurus to be significantly closer to WordNet than Roget Thesaurus is.
Language Resources and EvaluationThe WaCky wide web: a collection of very large linguistically processed web-crawled corpora
1,168 Citations2009Marco Baroni, Silvia Bernardini +2 more
UkWaC, deWaC and itWaC are introduced, three very large corpora of English, German, and Italian built by web crawling, and the methodology and tools used in their construction are described.
The Stanford typed dependencies representation
922 Citations2008Marie-Catherine de Marneffe, Christopher D. Manning
This paper examines the Stanford typed dependencies representation, which was designed to provide a straightforward description of grammatical relations for any user who could benefit from automatic text understanding, and considers the underlying design principles of the Stanford scheme.
Computational LinguisticsDependency-Based Construction of Semantic Space Models
701 Citations2007Sebastian Padó, Mirella Lapata
This article presents a novel framework for constructing semantic spaces that takes syntactic relations into account, and introduces a formalization for this class of models, which allows linguistic knowledge to guide the construction process.
Computational LinguisticsDistributional Memory: A General Framework for Corpus-Based Semantics
662 Citations2010Marco Baroni, Alessandro Lenci
The Distributional Memory approach is shown to be tenable despite the constraints imposed by its multi-purpose nature, and performs competitively against task-specific algorithms recently reported in the literature for the same tasks, and against several state-of-the-art methods.
Meeting of the Association for Computational LinguisticsOpen Information Extraction Using Wikipedia
550 Citations2010Fei Wu, Daniel S. Weld
WOE is presented, an open IE system which improves dramatically on TextRunner's precision and recall and is a novel form of self-supervised learning for open extractors -- using heuristic matches between Wikipedia infobox attribute values and corresponding sentences to construct training data.
DIRT @SBT@discovery of inference rules from text
548 Citations2001Dekang Lin, Patrick Pantel
This paper proposes an unsupervised method for discovering inference rules from text, based on an extended version of Harris' Distributional Hypothesis, which states that words that occurred in the same contexts tend to be similar.
RECERCAT (Consorci de Serveis Universitaris de Catalunya)Simple Semi-supervised Dependency Parsing
444 Citations2008Terry Koo, Xavier Carreras +1 more
This work focuses on the problem of lexical representation, introducing features that incorporate word clusters derived from a large unannotated corpus, and shows that the cluster-based features yield substantial gains in performance across a wide range of conditions.
Syntactic Annotations for the Google Books NGram Corpus
376 Citations2012Yuri Lin, Jean-Baptiste Michel +4 more
A new edition of the Google Books Ngram Corpus, which describes how often words and phrases were used over a period of five centuries, in eight languages, is presented, which will facilitate the study of linguistic trends, especially those related to the evolution of syntax.
A structured vector space model for word meaning in context
362 Citations2008Katrin Erk, Sebastian Padó
A novel structured vector space model is presented that makes it possible to integrate syntax into the computation of word meaning in context and performs at and above the state of the art for modeling the contextual adequacy of paraphrases.
Transition-based Dependency Parsing with Rich Non-local Features
329 Citations2011Yue Zhang, Joakim Nivre
This paper shows that it can improve the accuracy of transition-based dependency parsers by considering even richer feature sets than those employed in previous systems by improving the accuracy in the standard Penn Treebank setup and rivaling the best results overall.
A New String-to-Dependency Machine Translation Algorithm with a Target Dependency Language Model
254 Citations2008Libin Shen, Jinxi Xu +1 more
A novel string-todependency algorithm for statistical machine translation that employs a target dependency language model during decoding to exploit long distance word relations, which are unavailable with a traditional n-gram language model.
Annotated Gigaword
227 Citations2012Courtney Napoles, Matthew R. Gormley +1 more
This work has created layers of annotation on the English Gigaword v.5 corpus to render it useful as a standardized corpus for knowledge extraction and distributional semantics, and provides to the community a public reference set based on current state-of-the-art syntactic analysis and coreference resolution.
A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing
195 Citations2008Yue Zhang, Stephen Clark
A Latent Dirichlet Allocation Method for Selectional Preferences
155 Citations2010Alan Ritter, Oren Etzioni
LDA-SP, which utilizes LinkLDA to model selectional preferences, combines the benefits of previous approaches: like traditional class-based approaches, it produces human-interpretable classes describing each relation's preferences, but it is competitive with non-class-based methods in predictive power.
Computational LinguisticsA Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences
101 Citations2010Katrin Erk, Sebastian Padó +1 more
This work presents a vector space–based model for selectional preferences that predicts plausibility scores for argument headwords and obtains consistent benefits from using the disambiguation and semantic role information provided by a semantically tagged primary corpus.
Latent Variable Models of Selectional Preference
100 Citations2010Diarmuid Ó Séaghdha
Three models related to Latent Dirichlet Allocation, a proven method for modelling document-word cooccurrences, are presented and evaluated on datasets of human plausibility judgements and perform very competitively, especially for infrequent predicate-argument combinations.
QuestionBank
77 Citations2006John Judge, Aoife Cahill +1 more
Using QuestionBank training data improves parser performance to 89.75% labelled bracketing f-score, and a new method for recovering empty nodes and their antecedents (capturing long distance dependencies) from parser output in CFG trees using LFG f-structure reentrancies is introduced.
Improving dependency parsing with subtrees from auto-parsed data
69 Citations2009Wenliang Chen, Jun’ichi Kazama +2 more
First, a baseline parser is used to parse large-scale unannotated data, then subtrees from dependency parse trees in the auto-parsed data are extracted, and new subtree-based features for parsing algorithms are constructed.
Web-Scale Features for Full-Scale Parsing
40 Citations2011Mohit Bansal, Dan Klein
This work first presents a method for generating web count features that address the full range of syntactic attachments, and integrates these features into full-scale dependency and constituent parsers.
Clustering words by syntactic similarity improves dependency parsing of predicate-argument structures
27 Citations2009Kenji Sagae, Andrew S. Gordon
An approach for deriving syntactic word clusters from parsed text, grouping words according to their unlexicalized syntactic contexts is presented and the use of these syntactic clusters in leveraging a large corpus of trees generated by a high-accuracy parser to improve the accuracy of another parser based on a different formalism.
Domain Adaptation of a Dependency Parser with a Class-Class Selectional Preference Model
10 Citations2012Raphael Cohen, Yoav Goldberg +1 more
This paper uses Latent Dirichlet Allocations (LDA) to learn a domain-specific Selectional Preference model in the target domain using un-annotated data and applies this method for adapting Easy First, a fast non-directional parser trained on WSJ, to the biomedical domain.
Attacking Parsing Bottlenecks with Unlabeled Data and Relevant Factorizations
8 Citations2012Emily Pitler
By including unlabeled data features into a factorization of the problem which matches the representation of prepositions and conjunctions, this work achieves a new state-of-the-art for English dependencies with 93.55% correct attachments on the current standard.