Web-scale N-gram models for lexical disambiguation
Published 11 July 2009
Shane Bergsma, Dekang Lin, Randy Goebel
Citations84
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents a unified view of using web counts for lexical disambiguation with supervised and unsupervised systems that combine information from multiple and overlapping segments of context.
Abstract
Web-scale data has been used in a diverse range of language research. Most of this research has used web counts for only short, fixed spans of context. We present a unified view of using web counts for lexical disambiguation. Unlike previous approaches, our supervised and unsupervised systems combine information from multiple and overlapping segments of context. On the tasks of preposition selection and context-sensitive spelling correction, the supervised system reduces disambiguation error by 20-24 % over the current state-of-the-art. 1
Keywords
Computer Science
Support vector machine learning for interdependent and structured output spaces
1,260 Citations2004Ioannis Tsochantaridis, Thomas Hofmann +2 more
This paper proposes to generalize multiclass Support Vector Machine learning in a formulation that involves features extracted jointly from inputs and outputs, and demonstrates the versatility and effectiveness of the method on problems ranging from supervised grammar learning and named-entity recognition, to taxonomic text classification and sequence alignment.
Scaling to very very large corpora for natural language disambiguation
685 Citations2001Michele Banko, Eric Brill
This paper examines methods for effectively exploiting very large corpora when labeled data comes at a cost, and evaluates the performance of different learning methods on a prototypical natural language disambiguation task, confusion set disambigsuation.
Large Language Models in Machine Translation
550 Citations2007Thorsten Brants, Ashok C. Popat +3 more
Systems, methods, and computer program products for machine translation are provided for backoff score determination as a function of a backoff factor and a relative frequency of a corresponding backoff n-gram in the corpus.
Decision lists for lexical ambiguity resolution
362 Citations1994David Yarowsky
This paper presents a statistical decision procedure for lexical ambiguity resolution that exploits both local syntactic patterns and more distant collocational evidence, generating an efficient, effective, and highly perspicuous recipe for resolving a given ambiguity.
Machine LearningA Winnow-Based Approach to Context-Sensitive Spelling Correction
282 Citations1999Andrew R. Golding, Dan Roth
This work presents an algorithm combining variants of Winnow and weighted-majority voting, and applies it to a problem in the aforementioned class: context-sensitive spelling correction, and finds that WinSpell achieves accuracies significantly higher than BaySpell was able to achieve in either the pruned or unpruned condition.
arXiv (Cornell University)Learning to Resolve Natural Language Ambiguities: A Unified Approach
175 Citations1998Dan Roth
An extensive experimental comparison of the approach with other methods on several well studied lexical disambiguation tasks such as context-sensitive spelling correction, prepositional phrase attachment and part of speech tagging shows that it outperforms other methods tried for these tasks or performs comparably to the best.
The ups and downs of preposition error detection in ESL writing
174 Citations2008Joel Tetreault, Martin Chodorow
A methodology for detecting preposition errors in the writing of non-native English speakers and addressing the problem of annotation and evaluation by showing how current approaches of using only one rater can skew system evaluation.
ACM Transactions on Speech and Language ProcessingWeb-based models for natural language processing
170 Citations2005Mirella Lapata, Frank Keller
It is argued that Web-based models should be used as a baseline for, rather than an alternative to, standard supervised models in Natural Language Processing tasks, and generally fail to outperform supervised state-of-the-art models trained on smaller corpora.
Detection of grammatical errors involving prepositions
156 Citations2007Martin Chodorow, Joel Tetreault +1 more
A maximum entropy classifier combined with rule-based filters are used to detect preposition errors in a corpus of student essays to help develop an NLP application that can reliably detect these types of errors.
A method for word sense disambiguation of unrestricted text
145 Citations1999Rada Mihalcea, Dan Moldovan
This paper presents a method that attempts to disambiguate all the nouns, verbs, adverbs and adjectives in a text, using the senses provided in WordNet, and reports an average accuracy of 80% for the first ranked sense, and 91%.
Using Contextual Speller Techniques and Language Modeling for ESL Error Correction
140 Citations2008Michael Gamon, Jianfeng Gao +5 more
A decisiontree approach inspired by contextual spelling systems for detection and correction suggestions, and a large language model trained on the Gigaword corpus to provide additional information to filter out spurious suggestions are used.
Lecture notes in computer scienceReal-Word Spelling Correction with Trigrams: A Reconsideration of the Mays, Damerau, and Mercer Model
93 Citations2008Amber Wilcox-O’Hearn, Graeme Hirst +1 more
The trigram method is found to be superior, even on content words, and optimizing over sentences gives better results than variants of the algorithm that optimize over fixed-length windows.
SemEval-2007 task 06
76 Citations2007Ken Litkowski, Orin Hargraves
The SemEval-2007 task to disambiguate prepositions was designed as a lexical sample task, and the data generated in the task provides ample opportunitites for further investigations of preposition behavior.
Scaling Up Context-Sensitive Text Correction
71 Citations2001Andrew J. Carlson, Jeffrey Rosen +1 more
This paper examines and offers solutions to several issues relating to scaling up a context sensitive text correction system and suggests methods to reduce the memory requirements while maintaining a high level of performance and shows how to significantly increase the coverage to realistic levels.
Web Text Corpus for Natural Language Processing
56 Citations2006Vinci Liu, James Curran
A Web Corpus is created by downloading web pages to create a topic-diverse collection of 10 billion words of English and it is shown that for context-sensitive spelling correction the Web Corpus results are better than using a search engine.
Automatically acquiring models of preposition use
45 Citations2007Rachele De Felice, Stephen Pulman
This paper proposes a machine-learning based approach to predict accurately, given a syntactic and semantic context, which preposition is most likely to occur in that context.
Distributional Identification of Non-Referential Pronouns
36 Citations2008Shane Bergsma, Dekang Lin +1 more
An automatic approach to determining whether a pronoun in text refers to a preceding noun phrase or is instead nonreferential, which achieves the highest performance yet attained on this important task.
Data Analysis Project: Leveraging Massive Textual Corpora Using n-Gram Statistics
11 Citations2008J. Andrew Carlson, Tom M. Mitchell +1 more
Algorithms that use a database of frequency counts for sequences of tokens in a teraword Web corpus to correct spelling mistakes and to extract a list of instances of some category given only the name of the target category are explored.