Spelling correction in the PubMed search engine
Information RetrievalPublished 31 August 2006Open access
W. John Wilbur, Won Bae Kim, Natalie Xie
Citations31
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The methodology developed is based on the noisy channel model for spelling correction and makes use of statistics harvested from user logs to estimate the probabilities of different types of edits that lead to misspellings.
Abstract
It is known that users of internet search engines often enter queries with misspellings in one or more search terms. Several web search engines make suggestions for correcting misspelled words, but the methods used are proprietary and unpublished to our knowledge. Here we describe the methodology we have developed to perform spelling correction for the PubMed search engine. Our approach is based on the noisy channel model for spelling correction and makes use of statistics harvested from user logs to estimate the probabilities of different types of edits that lead to misspellings. The unique problems encountered in correcting search engine queries are discussed and our solutions are outlined.
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
Speech and language processing
3,589 Citations2010Dan Jurafsky, James Martin +1 more
It is now clear that HAL’s creator, Arthur C. Clarke, was a little optimistic in predicting when an artificial agent such as HAL would be avail-able.
Communications of the ACMA technique for computer detection and correction of spelling errors
1,573 Citations1964Fred J. Damerau
The method described assumes that a word which cannot be found in a dictionary has at most one error, which might be a wrong, missing or extra letter or a single transposition.
ACM Computing SurveysTechniques for automatically correcting words in text
1,256 Citations1992Karen Kukich
Research aimed at correcting words in text has focused on three progressively more difficult problems: nonword error detection; (2) isolated-word error correction; and (3) context-dependent work correction, which surveys documented findings on spelling error patterns.
ACM SIGIR ForumAnalysis of a very large web search engine query log
1,159 Citations1999Craig Silverstein, Hannes Marais +2 more
It is shown that web users type in short queries, mostly look at the first 10 results only, and seldom modify the query, suggesting that traditional information retrieval techniques may not work well for answering web search requests.
Journal of the American Society for Information Science and TechnologySearching the web: The public and their queries
938 Citations2001Amanda Spink, Dietmar Wolfram +2 more
It is found that most people use few search terms, few modified queries, view few Web pages, and rarely use advanced search features, and the language of Web queries is distinctive.
Agglomerative clustering of a search engine query log
767 Citations2000Doug Beeferman, Adam Berger
Journal of the American Society for Information Science and TechnologySearching the web: The public and their queries
606 Citations2001Amanda Spink, Dietmar Wolfram +2 more
ACM Computing SurveysApproximate String Matching
586 Citations1980Patrick A. V. Hall, Geoff R. Dowling
An improved error model for noisy channel spelling correction
537 Citations2000Eric Brill, Robert C. Moore
A new channel model for spelling correction, based on generic string to string edits, is described, which gives significant performance improvements compared to previously proposed models.
ACM Transactions on Information SystemsQuery clustering using user logs
370 Citations2002
A new query clustering method that makes use of user logs which allow us to identify the documents the users have selected for a query and shows that a combination of both keywords and user logs is better than using either method alone.
International Statistical ReviewIntroduction to Probability Theory and Statistical Inference
246 Citations1976Marius Iosifescu, H. J. Larson
Journal of the American Society for Information Science and TechnologyRelevant term suggestion in interactive web search based on contextual information in query session logs
243 Citations2003Chien‐Kang Huang, Lee‐Feng Chien +1 more
Experimental results show that the proposed approach to relevant term extraction and term suggestion can provide organized and highly relevant terms, and can exploit the contextual information in a user's query session to make more effective suggestions.
Statistics and ComputingProbability scoring for spelling correction
180 Citations1991Kenneth Church, William A. Gale
This paper describes a new program, CORRECT, which takes words rejected by the Unix® SPELL program, proposes a list of candidate corrections, and sorts them by probability score, and finds that human judges were extremely reluctant to cast a vote given only the information available to the program.
Journal of the American Society for Information Science and TechnologyMining longitudinal web queries: Trends and patterns
175 Citations2003Peiling Wang, Michael W. Berry +1 more
The purpose of the study is to understand Web users' query behavior, to identify problems encountered by these Web users, and to develop appropriate techniques for optimization of query analysis and mining.
Software Practice and ExperienceFinding approximate matches in large lexicons
139 Citations1995Justin Zobel, Philip Dart
This paper shows how to use string matching techniques in conjunction with lexicon indexes to find approximate matches in a large lexicon, and proposes methods for combining these techniques, and shows experimentally that these combinations yield good retrieval effectiveness while keeping index size and retrieval time low.
PubMedPubMed: bridging the information gap.
113 Citations2001Johanna McEntyre, David J. Lipman
On-line literature searches of bibliographic databases such as PubMed ([www.ncbi.nlm.nih.gov/entrez/query.fcgi] are now integral to the lives of clinicians.
Program electronic library and information systemsPHONIX: The algorithm
99 Citations1990T.N. Gadd
PHONIX complements the more conventional information retrieval techniques used by URICA, and the algorithm used is explained as a simple, easy to apply, set of rules.
“User revealment”—a comparison of initial queries and ensuing question development in online searching and in human reference interactions
75 Citations1999Ragnar Nordlie
This paper compares user’s opening queries and their search progression in two sets of data: logs of end-user online searches and user-librarian reference interactions, and applies a communication model to explain how users are induced to reveal their search intention in the dialogue with the intermediary.
ACM Transactions on Information SystemsThe use of dynamic contexts to improve casual internet searching
39 Citations2003Gondy Leroy, Ann Lally +1 more
The genetic algorithm with negative expansion tripled recall and doubled precision for low achievers, but high achievers displayed an opposed trend and seemed to be hindered in this condition, and the effect of other conditions was less substantial.
TechnometricsIntroduction to Probability Theory and Statistical Inference
22 Citations1970James H. Stapleton, Harold J. Larson