Improving Query Spelling Correction Using Web Search Results
Empirical Methods in Natural Language ProcessingPublished 1 June 2007
Qing Chen, Mu Li, Ming Zhou
Citations91
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel method is proposed for use of web search results to improve the existing query spelling correction models solely based on query logs by leveraging the rich information on the web related to the query and its top-ranked candidate.
Abstract
Traditional research on spelling correction in natural language processing and infor-mation retrieval literature mostly relies on pre-defined lexicons to detect spelling er-rors. But this method does not work well for web query spelling correction, because there is no lexicon that can cover the vast amount of terms occurring across the web. Recent work showed that using search query logs helps to solve this problem to some extent. However, such approaches cannot deal with rarely-used query terms well due to the data sparseness problem. In this paper, a novel method is proposed for use of web search results to improve the existing query spelling correction models solely based on query logs by leveraging the rich information on the web related to the query and its top-ranked candidate. Ex-periments are performed based on real-world queries randomly sampled from search engine’s daily logs, and the results show that our new method can achieve 16.9 % relative F-measure improvement and 35.4 % overall error rate reduction in comparison with the baseline method. 1
Keywords
Computer Science
Computational LinguisticsA maximum entropy approach to natural language processing
3,120 Citations1996Adam Berger, Vincent J. Della Pietra +1 more
A maximum-likelihood approach for automatically constructing maximum entropy models is presented and how to implement this approach efficiently is described, using as examples several problems in natural language processing.
ACM SIGIR ForumA taxonomy of web search
1,923 Citations2002Andrei Broder
This taxonomy of web searches is explored and how global search engines evolved to deal with web-specific needs is discussed.
Communications of the ACMA technique for computer detection and correction of spelling errors
1,573 Citations1964Fred J. Damerau
The method described assumes that a word which cannot be found in a dictionary has at most one error, which might be a wrong, missing or extra letter or a single transposition.
ACM SIGIR ForumQuary Expansion Using Local and Global Document Analysis
1,269 Citations2017Jinxi Xu, W. Bruce Croft
It is shown that, although global analysis haa some advantages, local analysia is generally more effective than global techniques and using global analysis techniques.
ACM Computing SurveysTechniques for automatically correcting words in text
1,256 Citations1992Karen Kukich
Research aimed at correcting words in text has focused on three progressively more difficult problems: nonword error detection; (2) isolated-word error correction; and (3) context-dependent work correction, which surveys documented findings on spelling error patterns.
The Annals of Mathematical StatisticsGeneralized Iterative Scaling for Log-Linear Models
1,203 Citations1972J. N. Darroch, D. Ratcliff
ACM SIGIR ForumDocument Language Models, Query Models, and Risk Minimization for Information Retrieval
774 Citations2017John Lafferty, ChengXiang Zhai
A framework for information retrieval that combines document models and query models using a probabilistic ranking function based on Bayesian decision theory is presented and an operational retrieval model that extends recent developments in the language modeling approach to information retrieval is suggested.
An improved error model for noisy channel spelling correction
537 Citations2000Eric Brill, Robert C. Moore
A new channel model for spelling correction, based on generic string to string edits, is described, which gives significant performance improvements compared to previously proposed models.
Information Processing & ManagementContext based spelling correction
267 Citations1991Eric Mays, Fred J. Damerau +1 more
Experimental results show that this statistical technique is capable of detecting 76% of simple spelling errors and correcting 73%.
A spelling correction program based on a noisy channel model
265 Citations1990Mark D. Kernighan, Kenneth Church +1 more
A new program is described, correct, which takes words rejected by the Unix® spell program, proposes a list of candidate corrections, and sorts them by probability, and the probability scores are the novel contribution of this work.
Empirical Methods in Natural Language ProcessingSpelling Correction as an Iterative Process that Exploits the Collective Knowledge of Web Users.
212 Citations2004Silviu Cucerzan, Eric Brill
This paper presents an approach that uses an iterative transformation of the input query strings into other strings that correspond to more and more likely queries according to statistics extracted from internet search query logs.
Pronunciation modeling for improved spelling correction
190 Citations2001Kristina Toutanova, Robert C. Moore
The proposed method builds an explicit error model for word pronunciations by modeling pronunciation similarities between words and achieves a substantial performance improvement over the previous best performing models for spelling correction.
Statistics and ComputingProbability scoring for spelling correction
180 Citations1991Kenneth Church, William A. Gale
This paper describes a new program, CORRECT, which takes words rejected by the Unix® SPELL program, proposes a list of candidate corrections, and sorts them by probability score, and finds that human judges were extremely reluctant to cast a vote given only the information available to the program.
Automatic Rule Acquisition for Spelling Correction
123 Citations1997Lidia Mangu, Eric Brill
This paper describes a new approach to automatically learning linguistic knowledge for spelling correction that is captured in a small set of easily understood rules, as opposed to a large set of opaque features and weights.
Learning a spelling error model from search query logs
109 Citations2005Hafiz Farooq Ahmad, Grzegorz Kondrak
This paper investigates using the Expectation Maximization algorithm to learn edit distance weights directly from search query logs, without relying on a corpus of paired words.
arXiv (Cornell University)Applying Winnow to Context-Sensitive Spelling Correction
95 Citations1996Andrew R. Golding, Dan Roth
This paper applies a Winnow-based algorithm to a task in natural language: context-sensitive spelling correction, and finds that Winnow is better than Bayes at adapting to the unfamiliar test set, using a strategy for combining learning on the training set with unsupervised learning on the test set.
Text induced spelling correction
44 Citations2004Martin Reynaert
TISC, a language-independent and context-sensitive spelling checking and correction system designed to facilitate the automatic removal of non-word spelling errors in large corpora, is presented and the implemented prototype is described and evaluated.