On the instability of web search engines
Published 12 April 2000
Erik Selberg, Oren Etzioni
Citations35
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The output of major WWW search engines was analyzed and the results led to some surprising observations about their stability.
Abstract
The output of major WWW search engines was analyzed and the results led to some surprising observations about their stability. Twenty-five queries were issued repeatedly to the engines and the results were compared. After one month, the top ten results returned by eight out of nine engines had changed by more than fifty percent. Furthermore, five out of the nine engines returned over a third of their URLs intermittently during the month.
Keywords
Computer Science
NatureAccessibility of information on the web
1,356 Citations1999Steve Lawrence, C. Lee Giles
As the web becomes a major communications medium, the data on it must be made more accessible, and search engines need to make the data more accessible.
ScienceSearching the World Wide Web
974 Citations1998Steve Lawrence, C. Lee Giles
The coverage and recency of the major World Wide Web search engines was analyzed, yielding some surprising results, including a lower bound on the size of the indexable Web of 320 million pages.
Computer Networks and ISDN SystemsA technique for measuring the relative size and overlap of public Web search engines
393 Citations1998Krishna Bharat, Andrei Broder
A standardized, statistical way of measuring search engine coverage and overlap through random queries is described that can be implemented by third-party evaluators using only public query interfaces and suggests the size of the static, public Web as of November was over 200 million pages.
Rate of change and other metrics: a live study of the world wide web
318 Citations1997Fred Douglis, Anja Feldmann +2 more
The potential benefit of a shared proxy-caching server in a large environment is quantified by using traces that were collected at the Internet connection points for two large corporations, representing significant numbers of references.
Multi-Engine Search and Comparison Using the MetaCrawler
255 Citations1995Erik Selberg, Oren Etzioni
The MetaCrawler is presented, a fielded Web service that represents the next level up in the information "food chain" and is sufficiently lightweight to reside on a user's machine, which facilitates customization, privacy, sophisticated filtering of references, and more.
Analysis of a Very Large AltaVista Query Log
171 Citations1998Craig Silverstein, Monika Henzinger +2 more
Computer Networks and ISDN SystemsInquirus, the NECI meta search engine
126 Citations1998Steve Lawrence, C. Lee Giles
The Inquirus meta search engine makes improvements over existing search engines in a number of areas, e.g.: more useful document summaries incorporating query term context, identification of both pages which no longer exist and pages which have no longer contain the query terms.
ScienceThe World Wide Web as an Instructional Tool
99 Citations1996John M. Barrie, David E. Presti
Three ways in which the WWW can be profitably used in education are discussed: as a giant encyclopedia, as a virtual classroom, and as a supplement to conventional courses.
Towards comprehensive web search
50 Citations1999Erik Selberg, Oren Etzioni
It is concluded that MetaCrawler demonstrates that meta-search can be implemented in a manner such that average Web users will take advantage of the benefits of meta- Search, and that search results can be made stable, but can be improved through Collaborative Index Enhancement, a novel model for enhancing a searchable index based on the experience of previous users.