Context sensitive stemming for web search
Published 23 July 2007
Fuchun Peng, Nawaaz Ahmed, Xin Li, Yumao Lu
Citations92
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A context sensitive stemming method that performs a context sensitive document matching for expanded variants of words in documents and serves as a safeguard against spurious stemming, and it turns out to be very important for improving precision.
Abstract
Traditionally, stemming has been applied to Information Retrieval tasks by transforming words in documents to the their root form before indexing, and applying a similar transformation to query terms. Although it increases recall, this naive strategy does not work well for Web Search since it lowers precision and requires a significant amount of additional computation.
Keywords
Computer Science
Modern Information Retrieval
11,544 Citations1999Ricardo Baeza‐Yates, Berthier Ribeiro‐Neto
Program electronic library and information systemsAn algorithm for suffix stripping
8,136 Citations1980Martin Porter
An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
ACM Transactions on Information SystemsCumulated gain-based evaluation of IR techniques
4,634 Citations2002Kalervo Järvelin, Jaana Kekäläinen
This article proposes several novel measures that compute the cumulative gain the user obtains by examining the retrieval result up to a given ranked position, and test results indicate that the proposed measures credit IR methods for their ability to retrieve highly relevant documents and allow testing of statistical significance of effectiveness differences.
Computer Speech & LanguageAn empirical study of smoothing techniques for language modeling
2,077 Citations1999Stanley F. Chen, Joshua Goodman
Automatic retrieval and clustering of similar words
1,555 Citations1998Dekang Lin
A word similarity measure based on the distributional pattern of words allows the automatically constructed thesaurus to be significantly closer to WordNet than Roget Thesaurus is.
Journal of the American Society for Information ScienceImproving retrieval performance by relevance feedback
1,504 Citations1990Gerard Salton, Chris Buckley
Relevance feedback is an automatic process, introduced over 20 years ago, designed to produce query formulations following an initial retrieval operation to demonstrate the effectiveness of the various methods.
ACM SIGIR ForumQuary Expansion Using Local and Global Document Analysis
1,269 Citations2017Jinxi Xu, W. Bruce Croft
It is shown that, although global analysis haa some advantages, local analysia is generally more effective than global techniques and using global analysis techniques.
Improving web search ranking by incorporating user behavior information
1,041 Citations2006Eugene Agichtein, Eric Brill +1 more
It is shown that incorporating user behavior data can significantly improve ordering of top results in real web search setting, improving the accuracy of a competitive web search ranking algorithms by as much as 31% relative to the original performance.
Query expansion using local and global document analysis
825 Citations1996Jinxi Xu, W. Bruce Croft
It is shown that using global analysis techniques, such as word contezt and phrase structure, on the local aet of documents produces re-sults that are both more effective and more predictable than simple local feedback.
Viewing morphology as an inference process
661 Citations1993Robert Krovetz
The role of morphological analysis in word sense disambiguation, and in identifying lexical semantic relationships in a machine-readable dictionary, is described.
Generating query substitutions
614 Citations2006Rosie Jones, Benjamin Rey +2 more
A model for selecting between candidates is built, by using a number of features relating the query-candidate pair, and by fitting the model to human judgments of relevance of query suggestions, which improves the quality of the candidates generated.
Proceedings of the 17th international conference on Computational linguistics -Automatic retrieval and clustering of similar words
550 Citations1998Dekang Lin
Journal of the American Society for Information ScienceHow effective is suffixing?
388 Citations1991Donna Harman
Three measures were selected which evaluate performance at given rank cutoff points, such as those cor- responding to a screenful of document titles,such as those responding to the lists of the top ranked documents.
Journal of the American Society for Information ScienceStemming algorithms: A case study for detailed evaluation
371 Citations1996David A. Hull
A case study of stemming algorithms is described which describes a number of novel approaches to evaluation and demonstrates their value.
Journal of DocumentationON TERM SELECTION FOR QUERY EXPANSION
318 Citations1990Stephen Robertson
A quantitative argument is sketches a quantitative argument which suggests that the two purposes of relevance feedback system and retrieval purposes require different weighting formulae.
ACM Transactions on Information SystemsCorpus-based stemming using cooccurrence of word variants
303 Citations1998Jinxi Xu, W. Bruce Croft
This work proposes a technique for using corpus-based word variant cooccurrence statistics to modify or create a stemmer, and demonstrates the viability of this technique and its advantages relative to conventional approaches that only employ morphological rules.
Using terminological feedback for web search refinement
248 Citations2003Peter Anick
Evidence is found that a subset of those users presented with terminological feedback do make effective use of it on a continuing basis of the AltaVista search engine.
Viewing stemming as recall enhancement
163 Citations1996Wessel Kraaij, Renée Pohlmann
Results show that linguistic stemming restricted to inflection yields a significant improvement over full linguistic and non-linquistic stemming, both in average Precision and R-Recall.
Semantic term matching in axiomatic approaches to information retrieval
127 Citations2006Hui Fang, ChengXiang Zhai
This paper shows that semantic term matching can be naturally incorporated into the axiomatic retrieval model through defining the primitive weighting function based on a semantic similarity function of terms, and shows that such extension can be efficiently implemented as query expansion.
Journal of Information ScienceAn evaluation of some conflation algorithms for information retrieval
118 Citations1981Martin Lennon, David S. Peirce +2 more
Comparative experiments with a range of keyword dictionaries and with the Cranfield document test collection suggest that there is relatively little difference in the performance of conflation algorithms despite the widely disparate means by which they have been developed and byWhich they operate.
A framework for selective query expansion
97 Citations2004Steve Cronen-Townsend, Yun Zhou +1 more
It is shown how comparison between language models of the unexpanded and expanded retrieval results can be used to predict when the expanded retrieval has strayed from the original sense of the query.
Query Segmentation for Web Search
62 Citations2003Knut Magne Risvik, Tomasz Mikolajewski +1 more
This paper describes a query segmentation method for search engines supporting inverse lookup of words and phrases, using data mining in query logs and document corpora to produce segment candidates and compute connexity measures.
Mining dependency relations for query expansion in passage retrieval
52 Citations2006Renxu Sun, Chai-Huat Ong +1 more
Two novel query expansion techniques that make use of dependency relation analysis to extract contextual terms and relations from external corpuses are presented and used to enhance the performance of density based and relation based passage retrieval frameworks respectively.
Information RetrievalWord normalization and decompounding in mono- and bilingual IR
42 Citations2006Eija Airio
In the monolingual runs, retrieval in a lemmatized compound index gives almost as good results as retrieving in a decompounded index, but in the bilingual runs differences are found: retrieval in an lemm atized decompoundedindex performs better than retrieval inA lemmAtized compoundIndex.
International ACM SIGIR Conference on Research and Development in Information RetrievalTerm conflation for information retrieval
32 Citations1984William B. Frakes
Based on results of two experiments concerned with term conflation for information retrieval, and the CATALOG retrieval system designed utilizing the results of the experiments,Term conflation can be automated in a retrieval system with no average loss of performance, thus allowing easier and user access to the system.
ACM SIGIR ForumFast and effective query refinement
16 Citations1997Bienvenido Vélez, Ron J. Weiss +2 more