Web Text Corpus for Natural Language Processing
Published 1 April 2006
Vinci Liu, James Curran
Citations56
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A Web Corpus is created by downloading web pages to create a topic-diverse collection of 10 billion words of English and it is shown that for context-sensitive spelling correction the Web Corpus results are better than using a search engine.
Abstract
Web text has been successfully used as training data for many NLP applications. While most previous work accesses web text through search engine hit counts, we created a Web Corpus by downloading web pages to create a topic-diverse collection of 10 billion words of English. We show that for context-sensitive spelling correction the Web Corpus results are better than using a search engine. For thesaurus extraction, it achieved similar overall results to a corpus of newspaper text. With many more words available on the web, better results can be obtained by collecting much larger web corpora.
Keywords
Computer Science
NatureAccessibility of information on the web
1,356 Citations1999Steve Lawrence, C. Lee Giles
As the web becomes a major communications medium, the data on it must be made more accessible, and search engines need to make the data more accessible.
Scaling to very very large corpora for natural language disambiguation
685 Citations2001Michele Banko, Eric Brill
This paper examines methods for effectively exploiting very large corpora when labeled data comes at a cost, and evaluates the performance of different learning methods on a prototypical natural language disambiguation task, confusion set disambigsuation.
ScholarlyCommons (University of Pennsylvania)Maximum entropy models for natural language ambiguity resolution
432 Citations1998Adwait Ratnaparkhi, Mitchell P. Marcus
This thesis demonstrates that several important kinds of natural language ambiguities can be resolved to state-of-the-art accuracies using a single statistical modeling technique based on the principle of maximum entropy.
Computational LinguisticsUsing the Web to Obtain Frequencies for Unseen Bigrams
357 Citations2003Frank Keller, Mirella Lapata
It is shown that the Web can be employed to obtain frequencies for bigrams that are unseen in a given corpus by querying a search engine.
Machine LearningA Winnow-Based Approach to Context-Sensitive Spelling Correction
282 Citations1999Andrew R. Golding, Dan Roth
This work presents an algorithm combining variants of Winnow and weighted-majority voting, and applies it to a problem in the aforementioned class: context-sensitive spelling correction, and finds that WinSpell achieves accuracies significantly higher than BaySpell was able to achieve in either the pruned or unpruned condition.
Computer NetworksOn near-uniform URL sampling
276 Citations2000Monika Henzinger, Allan Heydon +2 more
This paper suggests ways of improving sampling based on random walks of the Web graph to make the samples closer to uniform and suggests a natural test bed based onrandom graphs for testing the effectiveness of the procedures.
Edinburgh Research Archive (University of Edinburgh)From Distributional to Semantic Similarity
247 Citations2004James Curran
This dissertation describes how to extract contexts from a corpus of over 2 billion words and introduces a new context-weighted approximation algorithm with bounded complexity in context vector size that significantly reduces the system runtime with only a minor performance penalty.
Randomized algorithms and NLP
210 Citations2005Deepak Ravichandran, Patrick Pantel +1 more
The power of randomized algorithm is explored to address the challenge of working with very large amounts of data and the running time from quadratic to practically linear in the number of elements to be computed.
Frequency estimates for statistical word similarity measures
197 Citations2003Egidio L. Terra, Charles L. A. Clarke
A comparative study of two methods for estimating word co-occurrence frequencies required by word similarity measures, generated from a terabyte-sized corpus of Web data.
ACM Transactions on Speech and Language ProcessingWeb-based models for natural language processing
170 Citations2005Mirella Lapata, Frank Keller
It is argued that Web-based models should be used as a baseline for, rather than an alternative to, standard supervised models in Natural Language Processing tasks, and generally fail to outperform supervised state-of-the-art models trained on smaller corpora.
SZTAKI Publication Repository (Hungarian Academy of Sciences)Creating open language resources for Hungarian
79 Citations2004Péter Halácsy, András Kornai +4 more
An extensive crawl of the .hu domain yielded a raw dataset of over 18m web pages, and the methods used to detect and remove duplicates, low quality, foreign, and mixed language documents are discussed.
Zurich Open Repository and Archive (University of Zurich)Exploiting the WWW as a corpus to resolve PP attachment ambiguities
68 Citations2001Martin Volk
Finding the correct attachment site for prepositional phrases (PPs) is one of the hardest problems when parsing natural languages.
arXiv (Cornell University)A Bayesian hybrid method for context-sensitive spelling correction
66 Citations1996Andrew R. Golding
This paper takes Yarowsky's work as a starting point, applying decision lists to the problem of context-sensitive spelling correction, and finds that further improvements can be obtained by taking into account not just the single strongest piece of evidence, but ALL the available evidence.
Using the web in machine learning for other-anaphora resolution
53 Citations2003Natalia N. Modjeska, Katja Markert +1 more
Besides morpho-syntactic, recency, and semantic features based on existing lexical knowledge resources, the algorithm obtains additional semantic knowledge from the Web via lexico-Syntactic patterns that are specific to other-anaphors.
The impact of corpus size on question answering performance
31 Citations2002Charles L. A. Clarke, Gordon V. Cormack +3 more
Using the question answering system, questions from the TREC 2001 evaluation were executed over a series of Web data collections, with the sizes of the collections increasing from 25 gigabytes up to nearly a terabyte.
A study of using search engine page hits as a proxy for n-gram frequencies
22 Citations2005Preslav Nakov, Marti A. Hearst
Using a particular NLP task, the variability in the n-gram counts across different search engines as well as for the same search engine across time is compared, finding that although there are measurable differences, they are not statistically significantly different for the task examined.