How do search engines respond to some non-English queries?
Journal of Information SciencePublished 23 December 2004
Judit Bar‐Ilan, Tatyana Gutman
Citations54
SJR quartileQ1
SJR score0.63
SNIP1.31
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
These findings are rather disturbing, since for example Google is very popular in non-English speaking countries as well, and users are either not aware of what they miss when using search tools that do not take into account the structure and the special characteristics of the specific language or have no alternatives to use these search engines.
Abstract
In this article, we explored the capabilities of search engines for non-English languages. As a test case, we considered four languages: Russian, French, Hungarian and Hebrew. For each of these languages we tested three general search engines: Alta Vista, FAST and Google and some local search engines. Our results indicate that in the examined cases the general search engines ignore the special characteristics of non-English languages, and sometimes they do not even handle diacritics well. These findings are rather disturbing, since for example Google is very popular in non-English speaking countries as well, and users are either not aware of what they miss when using search tools that do not take into account the structure and the special characteristics of the specific language or have no alternatives but to use these search engines.
Keywords
Computer Science
ACM SIGIR ForumAnalysis of a very large web search engine query log
1,159 Citations1999Craig Silverstein, Hannes Marais +2 more
It is shown that web users type in short queries, mostly look at the first 10 results only, and seldom modify the query, suggesting that traditional information retrieval techniques may not work well for answering web search requests.
Journal of the American Society for Information ScienceHow effective is suffixing?
388 Citations1991Donna Harman
Three measures were selected which evaluate performance at given rank cutoff points, such as those cor- responding to a screenful of document titles,such as those responding to the lists of the top ranked documents.
Improving stemming for Arabic information retrieval
360 Citations2002Leah S. Larkey, Lisa Ballesteros +1 more
Several light stemmers based on heuristics and a statistical stemmer based on co-occurrence for Arabic retrieval and the retrieval effectiveness of these stemmers and of a morphological analyzer on the TREC-2001 data were compared.
The effects of query structure and dictionary setups in dictionary-based cross-language information retrieval
259 Citations1998Ari Pirkola
There was only a slight difference in performance between the original English queries and the best crosslanguage queries, i.e., structured queries with medical dictionary and general dictionary translation.
Encyclopedia of Machine Learning and Data MiningCross-Language Information Retrieval
246 Citations2017Claude Sammut, Geoffrey I. Webb
This chapter reviews research and practice in CLIR that allows users to state queries in their native language and retrieve documents in any other language supported by the system.
Journal of the American Society for Information ScienceThe effectiveness of stemming for natural-language access to Slovene textual data
167 Citations1992Mirko Popovič, Peter Willett
The use of stemming on Slovene-language documents and queries is reported, and it is demonstrated that the use of an appropriate stemming algorithm results in a large, and statistically significant, increase in retrieval effectiveness when compared with nonconflated processing.
Journal of the American Society for Information ScienceA stemming procedure and stopword list for general French corpora
125 Citations1999Jacques Savoy
The tools needed to establish a general stopword list and a stemming procedure in the French language databases are presented and some retrieval experiments that have been carried out using two medium-sized French language test collections are evaluated.
Journal of the American Society for Information Science and TechnologyVox populi: The public searching of the web
114 Citations2001Dietmar Wolfram, Amanda Spink +2 more
Comparaison de deux etudes faites en 1997 and 1999 sur le mode d'interrogation du Web par 200000 utilisateurs du moteur Excite ainsi que sur les sujets d'Interrogation (loisirs, sante, education...).
Information RetrievalCross-Language Evaluation Forum: Objectives, Results, Achievements
112 Citations2003Martín Braschler, C Peters
The main lessons learned during this period are summarized, the state-of-the-art of the research reported in the CLEF experiments are outlined and proposals for future directions in system evaluation aimed at meeting emerging needs are made.
Journal of the American Society for Information ScienceComparing words, stems, and roots as index terms in an Arabic Information Retrieval system
100 Citations1994Ibrahim A. Al‐Kharashi, Martha Evens
These experiments revealed that using roots and using stems as index terms gives better retrieval results than using words.
Lecture notes in computer scienceShallow Morphological Analysis in Monolingual Information Retrieval for Dutch, German, and Italian
94 Citations2002Christof Monz, Maarten de Rijke
The focus of the experiments was on the effects of morphological analyses, such as stemming and compound splitting, on retrieval effectiveness, and found improvements to be around 25% for German and as much as 69% for Dutch.
arXiv (Cornell University)Estimation of English and non-English Language Use on the WWW
94 Citations2000Gregory Grefenstette, Julien Nioche
A technique for estimating the size of a language-specific corpus given the frequency of commonly occurring words in the corpus is presented and this technique is applied to estimating the number of words available through Web browsers for given languages.
Journal of the American Society for Information ScienceStemming methodologies over individual query words for an Arabic Information Retrieval System
74 Citations1999Hani Abu‐Salem, Mahmoud Al‐Omari +1 more
This article investigates how to improve the performance of an Arabic Information Retrieval System (Arabic-IRS) by imposing the retrieval method over individual words of a query depending on the importance of the WORD, the STEM, or the ROOT of the query terms in the database.
Statistical morphological disambiguation for agglutinative languages
72 Citations2000Dilek Hakkani‐Tür, Kemal Oflazer +1 more
Journal of the American Society for Information ScienceStemming methodologies over individual query words for an Arabic Information Retrieval System
58 Citations1999Hani Abu‐Salem, Mahmoud Al‐Omari +1 more
Program electronic library and information systemsSuffix stripping with modern Greek
55 Citations1995Theodore Kalamboukis
This work examines modern Greek texts and examines the plural inflection of the English noun while in modern Greek there are 41 different inflectional suffixes.
Artificial Intelligence ReviewHebrew Computational Linguistics: Past and Future
51 Citations2004Shuly Wintner
Text REtrieval ConferenceThe TREC-2002 Arabic/English CLIR Track
51 Citations2002Douglas W. Oard, Fredric C. Gey
Nine teams participated in the TREC-2002 cross-language information retrieval track, which focused on retrieving Arabic language documents based on 50 topics that were originally prepared in English.
Lecture notes in computer scienceLexical and Algorithmic Stemming Compared for 9 European Languages with Hummingbird SearchServerTM at CLEF 2003
47 Citations2004Stephen Tomlinson
Hummingbird participated in the monolingual information retrieval tasks of the Cross-Language Evaluation Forum (CLEF) 2003: for natural language queries in 9 European languages, find all the relevant documents (with high precision) in the CLEF 2003 document sets.
Information Processing & ManagementAspects of Swedish morphology and semantics from the perspective of mono- and cross-language information retrieval
47 Citations2001Turid Hedlund, Ari Pirkola +1 more
The results suggest that part-of-speech tagging might be useful in Swedish IR due to the high frequency of homographic words, and publicly available morphological analysis tools used for normalization and compound splitting have pitfalls that might decrease the effectiveness of IR and CLIR.
Information RetrievalFrom Plain Character Strings to Meaningful Words: Producing Better Full Text Databases for Inflectional and Compounding Languages with Morphological Analysis Software
35 Citations2001Riitta Alkula
Special attention is focused on the characteristics of highly inflectional languages, and how morphological structure of a language should be taken into account, when designing and developing information retrieval systems.
LibriInformation Retrieval from Full-Text Arabic Databases: Can Search Engines Designed for English Do the Job?
31 Citations2001Haidar Moukdad, Andrew Large
The research investigates the impact on retrieval of prefixes in Arabic and questions the assumption that search engines designed with English in mind will work as well with different language structures.
Lecture notes in computer scienceReport on CLEF-2003 Monolingual Tracks: Fusion of Probabilistic Models for Effective Monolingual Retrieval
26 Citations2004Jacques Savoy
This paper proposes more effective and general stopword lists for the Swedish, Finnish and Russian languages, along with an improved, more efficient and simpler stemming procedure for these three languages, and suggests a combined search approach based on a data fusion strategy that would work with various European languages.
Journal of the ACMKEDMA—Linguistic Tools for Retrieval Systems
22 Citations1978Rony Attar, Yaacov Choueka +2 more
In a full-text natural-language retrieval system, frequent need for automatic hngulst~c analysis arises, e.g for keyword expansion in a search process, content analysis, or automatic construction of concordances.
The International Information & Library ReviewWeb Search Engines for Polish Information Retrieval: Questions of Search Capabilities and Retrieval Performance
19 Citations2000Marek Sroka
Lecture notes in computer scienceMorphological Disambiguation for Hebrew Search Systems
15 Citations1999David Carmel, Yoelle Maarek
A Hebrew Morphological Disambiguator (HMD or Hemed for short) based on statistical data gathered from large Hebrew corpora is described and how to integrate HMD with a search engine to enable linguistic search for Hebrew is shown.
CLEF (Working Notes)UC Berkeley at CLEF 2003 - Russian Language Experiments and Domain-Specific Cross-Language Retrieval
13 Citations2003Vivien Petras, Natalia Perelman +1 more
It is found that Berkeley's group 1's thesaurus matching technique compares to conventional machine translation for Russian and German against English retrieval and outperforms machinetranslation for English to German retrieval.
The International Information & Library ReviewWeb Search Engines for Polish Information Retrieval: Questions of Search Capabilities and Retrieval Performance
9 Citations2000Marek Sroka
Lecture notes in computer scienceA Search Engine for Indian Languages
7 Citations2000Ashwani Mujoo, Manoj Kumar Malviya +2 more
The design issues of a Search Engine for Indian Languages are described and the implementation of two Search Engines for Indian languages, one for documents in ISCII and the other for documentsIn Unicode are described.
CLEF (Working Notes)Multilingual experiments of UTA at CLEF 2003 The impact of different merging strategies and word normalizing tools
4 Citations2003Eija Airio, Heikki Keskustalo +2 more
The tests of University of Tampere show that stemming is a good normalizing method for English, which is a language with weak morphology, and that stemming given remarkably worse results, the index built with the morphological analyzer gave better results.