Summary of WWW characterizations
World Wide WebPublished 1 June 1999
James E. Pitkow
Citations147
SJR quartileQ1
SJR score0.88
SNIP1.17
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper presents a summary of efforts to characterize various aspects of the World Wide Web, highlighting regularities and insights that have been discovered across the variety of access points available for instrumentation.
Abstract
To date there have been a number of efforts that attempt to characterize various aspects of the World Wide Web. This paper presents a summary of these efforts, highlighting regularities and insights that have been discovered across the variety of access points available for instrumentation. Characterizations that are derived from client, proxy, and server instrumentation are reviewed as well as efforts to characterize the entire structure of the WWW. Given the dynamic nature of the Web, it may be surprising for some readers to find that many properties of the Web follow regular and predictable patterns that have not changed in form over the Web's lifetime. Understanding these aspects as well as those that vary is critical to designing a better Web, and as a direct consequence, creating a more enjoyable user experience.
Keywords
Computer Science
Generating representative Web workloads for network and server performance evaluation
1,438 Citations1998Paul Barford, Mark Crovella
This paper applies a number of observations of Web server usage to create a realistic Web workload generation tool which mimics a set of real users accessing a server and addresses the technical challenges to satisfying this large set of simultaneous constraints on the properties of the reference stream.
On the self-similar nature of Ethernet traffic
1,078 Citations1993W.E. Leland, Murad S. Taqqu +2 more
It is demonstrated that Ethernet local area network (LAN) traffic is statistically self-similar, that none of the commonly used traffic models is able to capture this fractal behavior, and that such behavior has serious implications for the design, control, and analysis of high-speed, cell-based networks.
ScienceSearching the World Wide Web
974 Citations1998Steve Lawrence, C. Lee Giles
The coverage and recency of the major World Wide Web search engines was analyzed, yielding some surprising results, including a lower bound on the size of the indexable Web of 320 million pages.
Computer Networks and ISDN SystemsCharacterizing browsing strategies in the World-Wide web
930 Citations1995Lara D. Catledge, James E. Pitkow
A study conducted at Georgia Institute of Technology that captured client-side user events of NCSA's XMosaic supplemented the understanding of user navigation strategies as well as provided real interface usage data.
Minds at UW (University of Wisconsin)Cost-aware WWW proxy caching algorithms
919 Citations1997Pei Cao, Sandy Irani
ScienceStrong Regularities in World Wide Web Surfing
761 Citations1998Bernardo A. Huberman, Peter Pirolli +2 more
A model that assumes that users make a sequence of decisions to proceed to another page, continuing as long as the value of the current page exceeds some threshold, yields the probability distribution for the number of pages that a user visits within a given Web site.
ACM SIGMETRICS Performance Evaluation ReviewWeb server workload characterization
753 Citations1996Martin Arlitt, Carey Williamson
ACM SIGCOMM Computer Communication ReviewUsing predictive prefetching to improve World Wide Web latency
632 Citations1996Venkata N. Padmanabhan, Jeffrey C. Mogul
This work investigates a scheme for reducing the latency perceived by users by predicting and prefetching files that are likely to be requested soon, while the user is browsing through the currently displayed page.
International Journal of Human-Computer StudiesHow people revisit web pages: empirical findings and implications for the design of history systems
563 Citations1997Linda Tauscher, Saul Greenberg
Based on users' revisitation patterns to World Wide Web pages, eight design guidelines for web browser history mechanisms were formulated and explain why some aspects of today's browsers seem to work well, and other's poorly.
Characteristics of WWW Client-based Traces
529 Citations1995Carlos R. Cunha, Azer Bestavros +1 more
This paper presents a descriptive statistical summary of the traces of actual executions of NCSA Mosaic, and shows that many characteristics of WWW use can be modelled using power-law distributions, including the distribution of document sizes, the popularity of documents as a function of size, and the Distribution of user requests for documents.
ACM SIGMETRICS Performance Evaluation ReviewSelf-similarity in World Wide Web traffic
526 Citations1996Mark Crovella, Azer Bestavros
Characterizing reference locality in the WWW
465 Citations2002Virgı́lio Almeida, Azer Bestavros +2 more
The authors propose models for both temporal and spatial locality of reference in streams of requests arriving at Web servers and show that temporal locality can be characterized by the marginal distribution of the stack distance trace, and proposed models for typical distributions and compare their cache performance to the traces.
Caching Proxies: Limitations and Potentials
445 Citations1995Marc Abrams, Charles R. Standridge +3 more
This work assesses the potential of proxy servers to cache documents retrieved with the HTTP, GOPHER, FTP, and WAIS protocols using World Wide Web browsers to find that with workloads over a one-semester period, a proxy has a 30--50% maximum possible hit rate.
Removal policies in network caches for World-Wide Web documents
437 Citations1996Marc Abrams, Charles R. Standridge +3 more
Surprisingly, the criteria used by several proxy-server removal policies are among the worst performing criteria in the authors' simulation; instead, replacing documents based on size maximizes hit rate in each of the studied workloads.
Computer Networks and ISDN SystemsA technique for measuring the relative size and overlap of public Web search engines
393 Citations1998Krishna Bharat, Andrei Broder
A standardized, statistical way of measuring search engine coverage and overlap through random queries is described that can be implemented by third-party evaluators using only public query interfaces and suggests the size of the static, public Web as of November was over 200 million pages.
Heavy-tailed probability distributions in the World Wide Web
328 Citations1998Mark Crovella, Murad S. Taqqu +1 more
Evidence is presented that a number of le size distributions in the Web exhibit heavy tails, including les requested by users, les transmitted through the network, transmission durations of les, and les stored on servers, that are primarily determined by the distribution of les available on the Web.
Rate of change and other metrics: a live study of the world wide web
318 Citations1997Fred Douglis, Anja Feldmann +2 more
The potential benefit of a shared proxy-caching server in a large environment is quantified by using traces that were collected at the Internet connection points for two large corporations, representing significant numbers of references.
IEEE/ACM Transactions on NetworkingReplacement policies for a proxy cache
306 Citations2000Luigi Rizzo, L. Vicisano
This paper proposes a novel replacement policy, called LRV, which selects for replacement the document with the lowest relative value among those in cache, and shows how LRV outperforms least recently used (LRU) and other policies and can significantly improve the performance of the cache, especially for a small one.
Computer Networks and ISDN SystemsA caching relay for the World Wide Web
294 Citations1994S. Glassman
The design and performance of a caching relay for the World Wide Web is described and the data gathered from the relay is used to make some general characterizations about the web.
World-wide web cache consistency
264 Citations1996James Gwertzman, Margo Seltzer
Using trace-driven simulation, it is shown that a weak cache consistency protocol (the one used in the Alex ftp cache) reduces network bandwidth consumption and server load more than either time-to-live fields or an invalidation protocol and can be tuned to return stale data less than 5% of the time.
Self-similarity in World Wide Web traffic
262 Citations1996Mark Crovella, Azer Bestavros
It is shown that the self-similarity in WWW traffic can be explained based on the underlying distributions of WWW document sizes, the effects of caching and user preference in file transfer, the effect of user "think time", and the superimposition of many such transfers in a local area network.
Computer Networks and ISDN SystemsWorld-Wide Web proxies
228 Citations1994Ari Luotonen, Kevin Altis
An overview of proxies is given and a brand new feature is caching performed by the proxy, resulting in shorter response times after the first document fetch, making proxies useful even to the people who have full Internet access and do not really need the proxy just to get out of their local subnet.
System design issues for internet middleware services: deductions from a large client trace
202 Citations1997Steven D. Gribble, Eric Brewer
The heterogeneity of web clients, the existence of a strong and very predictable diurnal cycle in the clients' web activity, the burstiness of clients' requests at small time scales, the presence of locality of reference in the client's requests that is a strong function of the client population size, and the high latency that services encounter when delivering data to clients are demonstrated.
Computer Networks and ISDN SystemsMeasuring the Web
198 Citations1996Tim Bray
This paper presents some difficult qualitative questions concerning the Web, and attempts to provide some partial quantitative answers to them, and uses the numbers in these answers to drive some 3-D visualizations of localities in the Web.
The Measured Access Characteristics of World-Wide-Web Client Proxy Caches
197 Citations1997Bradley M. Duska, David Marwood +1 more
An analysis of access traces collected from seven proxy servers deployed in various locations throughout the Internet shows that a 2- to 10-GB second-level cache yields hit rates between 24% and 45% with 85% of these hits due to sharing among different clients.
Computer Networks and ISDN SystemsProxy caching that estimates page load delays
190 Citations1997Roland Peter Wooster, Marc Abrams
Two new caching algorithms that keep in the cache documents that take the longest to retrieve are explored, and are compared to the best three existing policies—LRU, LFU, and SIZE—using three measures-user response time and ability to minimize Web server loads and network bandwidth consumed.
Explaining World Wide Web Traffic Self-Similarity
177 Citations1995Mark Crovella, Azer Bestavros
It is shown that the self-similarity in traffic due to the World Wide Web can be explained based on the underlying distributions of WWW document sizes, the effects of caching and user preference in file transfer, the effect of user ``think time'''', and the superimposition of many such transfers in a local area network.
Life, death, and lawfulness on the electronic frontier
152 Citations1997James E. Pitkow, Peter Pirolli
To facilitate users’ ability to make sense of large collections of hypertext, two new techniques for inducing clusters of related documents on the World Wide Web are presented.
ACM SIGMETRICS Performance Evaluation ReviewWeb proxy caching
150 Citations1998Ramón Cáceres, Fred Douglis +3 more
Application-level document caching in the Internet
150 Citations2002Azer Bestavros, R.L. Carter +4 more
The results suggest that distinguishing between documents produced locally and those produced remotely can provide useful leverage in designing caching policies, because of differences in the potential for sharing these two document types among multiple users.
Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems
110 Citations2002Azer Bestavros
Two server-initiated protocols to improve the performance of distributed information systems (e.g. WWW) are presented, whereby a request for a document is serviced by sending, in addition to the document requested, a number of other documents that the server speculates will be requested in the near future.
Network Behavior of a Busy Web Server and its Clients
102 Citations1999Jeffrey C. Mogul
The 1994 California Election server, which provided ‘‘live’’ election returns, handled over 1.5 million individual requests, including almost 1 million in a single 24-hour period, which may have been the most intensive single event on the Internet, to date.
Computer Networks and ISDN SystemsA case for delay-conscious caching of Web documents
101 Citations1997Peter Scheuermann, Junho Shim +1 more
This paper presents a new, delay-conscious cache replacement algorithm LNC-R-W3 which maximizes a performance metric called delay-savings-ratio and compares it with other existing cache replacement algorithms, namely LRU and LRU-MIN.
Computer Networks and ISDN SystemsPerformance engineering of the World Wide Web: Application to dimensioning and cache design
101 Citations1996Jean-Chrysostome Bolot, Philipp Hoschka
This paper considers performance issues related to dimensioning and caching, and advocates the use of novel cache replacement algorithms that explicitly take document size and network load into account so as to minimize the retrieval time perceived by clients.
Computer Networks and ISDN SystemsAn investigation of documents from the World Wide Web
88 Citations1996Allison Woodruff, Paul M. Aoki +3 more
Examination of pages from the World Wide Web collected by the Inktomi Web crawler examines many characteristics of these documents, including document size; number and types of tags, attributes, file extensions, protocols, and ports; the number of in-links; and the ratio of document size to thenumber of tags and attributes.
Computer Networks and ISDN SystemsWeb traffic characterization: an assessment of the impact of caching documents from NCSA's web server
88 Citations1995Hans-Werner Braun, kc claffy
This work analyzes two days of queries to the popular NCSA Mosaic server to assess the geographic distribution of transaction requests, and investigates a range of timeouts for flushing documents from the cache, outlining the tradeoff between bandwidth savings and memory/cache management costs.
Determining WWW user's next access and its application to pre-fetching
83 Citations2002Carlos R. Cunha, C.F.B. Jaccoud
Two user models are developed that help determine user's next move and one model uses random walk approximation and the other is based on digital signal processing techniques.
Demand-based document dissemination to reduce traffic and balance load in distributed information systems
81 Citations2002Azer Bestavros
This work proposes a hierarchical demand-based replication strategy that optimally disseminates information from its producer to servers that are closer to its consumers, and shows that by disseminating the most popular documents on servers closer to clients, network traffic could be reduced considerably, while servers are load-balanced.
Web facts and fantasy
80 Citations1997Stephen Manley, Margo Seltzer
A taxonomy of the different types of Web sites is presented and it is shown that, as expected, persistent connections are generally useful, but that dynamic time-out intervals may be unnecessarily complex and that allowing multiple persistent connections per client may actually hinder resource utilization compared to allowing only a single persistent connection.
VTechWorks (Virginia Tech)Analysis and modeling of world wide web traffic
32 Citations1998Ghaleb Abdulla, Edward A. Fox
An approach to model long-range and dependent Web traffic and activities of users accessing a digital library courseware server or Web search tools are described and characterized and an algorithm to characterize users' sessions is developed.
Digital Access to Scholarship at Harvard (DASH) (Harvard University)A Self-Scaling and Self-Configuring Benchmark for Web Servers
30 Citations1997Stephen Manley, Michael Courage +1 more
Computer Networks and ISDN SystemsThe Japan Cache Project: an experiment on domain cache
18 Citations1997M. Nabeshima
This paper proposes the concept of a domain cache, which is dedicated to handling access to a particular domain name, and reports experiments in the Japan Cache Project which operates a public cache server for access to JP (Japan) domain web servers.
Computer Networks and ISDN SystemsWhat can archives offer the World Wide Web?
17 Citations1994Neil Smith
This paper shall demonstrate how, at the UNIX HENSA Archive, the roles of these archive sites are examined with a view to what they can provide the World Wide Web community.
Shared User Behavior on the World Wide Web.
16 Citations1997Ghaleb Abdulla, Edward A. Fox +1 more
A self-scaling and self-configuring benchmark for Web servers (extended abstract)
15 Citations1998Stephen Manley, Margo Seltzer +1 more
This paper presents the new benchmark hBench:Web, a self-configuring, scalable benchmark that generates a server benchmark load based on actual server loads, and demonstrates how it accurately models the load of an actual server.
VTechWorks (Virginia Tech)Optimizing Response Time, Rather than Hit Rates, of WWW Proxy Caches
12 Citations1996Roland Peter Wooster
The experiments show that a removal algorithm that minimizes only the download time yields poor results, but a new algorithm is investigated that does provide improved performance over common removal algorithms using three factors — the speed at which a file is downloaded, the size of the file, and the number of references to the file.