Rate of change and other metrics: a live study of the world wide web
Published 8 December 1997
Fred Douglis, Anja Feldmann, Balachander Krishnamurthy, Jeffrey C. Mogul
Citations318
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The potential benefit of a shared proxy-caching server in a large environment is quantified by using traces that were collected at the Internet connection points for two large corporations, representing significant numbers of references.
Abstract
Caching in the World Wide Web is based on two critical assumptions: that a significant fraction of requests reaccess resources that have already been retrieved; and that those resources do not change between accesses. We tested the validity of these assumptions, and their dependence on characteristics of Web resources, including access rate, age at time of reference, content type, resource size, and Internet top-level domain. We also measured the rate at which resources change, and the prevalence of duplicate copies in the Web. We quantified the potential benefit of a shared proxycaching server in a large environment by using traces that were collected at the Internet connection points for two large corporations, representing significant numbers of references. Only 22% of the resources referenced in the traces we analyzed were accessed more than once, but about half of the references were to those multiplyreferenced resources. Of this half, 13% were to a resource that had been modifi...
Keywords
Computer ScienceEngineering
Auerbach Publications eBooksHypertext Transfer Protocol (HTTP)
2,837 Citations2003Roy T. Fielding, Jim Gettys
Computer Networks and ISDN SystemsSyntactic clustering of the Web
1,347 Citations1997Andrei Broder, S. Glassman +2 more
An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
ACM SIGMETRICS Performance Evaluation ReviewWeb server workload characterization
753 Citations1996Martin Arlitt, Carey Williamson
Characteristics of WWW Client-based Traces
529 Citations1995Carlos R. Cunha, Azer Bestavros +1 more
This paper presents a descriptive statistical summary of the traces of actual executions of NCSA Mosaic, and shows that many characteristics of WWW use can be modelled using power-law distributions, including the distribution of document sizes, the popularity of documents as a function of size, and the Distribution of user requests for documents.
Removal policies in network caches for World-Wide Web documents
437 Citations1996Marc Abrams, Charles R. Standridge +3 more
Surprisingly, the criteria used by several proxy-server removal policies are among the worst performing criteria in the authors' simulation; instead, replacing documents based on size maximizes hit rate in each of the studied workloads.
Exploring the bounds of web latency reduction from caching and prefetching
341 Citations1997Thomas M Kroeger, Darrell D. E. Long +1 more
It is concluded that for the workload studied caching offers moderate assistance in reducing latency and prefetching can offer more than twice the improvement of caching but is still limited in its ability to reduce latency.
Potential benefits of delta encoding and data compression for HTTP
300 Citations1997Jeffrey C. Mogul, Fred Douglis +2 more
It is shown that delta encoding can provide remarkable improvements in response size and response delay for an important subset of HTTP content types, and that the combination of delta encoding and data compression yields the best results.
World-wide web cache consistency
264 Citations1996James Gwertzman, Margo Seltzer
Using trace-driven simulation, it is shown that a weak cache consistency protocol (the one used in the Alex ftp cache) reduces network bandwidth consumption and server load more than either time-to-live fields or an invalidation protocol and can be tuned to return stale data less than 5% of the time.
System design issues for internet middleware services: deductions from a large client trace
202 Citations1997Steven D. Gribble, Eric Brewer
The heterogeneity of web clients, the existence of a strong and very predictable diurnal cycle in the clients' web activity, the burstiness of clients' requests at small time scales, the presence of locality of reference in the client's requests that is a strong function of the client population size, and the high latency that services encounter when delivering data to clients are demonstrated.
Computer Networks and ISDN SystemsMeasuring the Web
198 Citations1996Tim Bray
This paper presents some difficult qualitative questions concerning the Web, and attempts to provide some partial quantitative answers to them, and uses the numbers in these answers to drive some 3-D visualizations of localities in the Web.
WebExpress
133 Citations1996Barron C. Housel, David B. Lindquist
Software technology that makes it possible to support Internet World Wide Web applications in wide area wireless networks by intercepting the HITP data stream and performing various optimizations including: flle caching, forms differencing, protocol reduction, and the elimination of redundant HTTP header transmission is described.
Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems
110 Citations2002Azer Bestavros
Two server-initiated protocols to improve the performance of distributed information systems (e.g. WWW) are presented, whereby a request for a document is serviced by sending, in addition to the document requested, a number of other documents that the server speculates will be requested in the near future.
World Wide WebThe AT&T Internet Difference Engine: Tracking and viewing changes on the web
104 Citations1998Fred Douglis, Thomas Ball +2 more
AIDE is a system that finds and displays changes to pages on the World Wide Web, including a web‐crawler that detects changes, an archive of past versions of pages, a tool to highlight changes between versions of a page, and a graphical interface to view the relationship between pages over time.
Computer Networks and ISDN SystemsAn investigation of documents from the World Wide Web
88 Citations1996Allison Woodruff, Paul M. Aoki +3 more
Examination of pages from the World Wide Web collected by the Inktomi Web crawler examines many characteristics of these documents, including document size; number and types of tags, attributes, file extensions, protocols, and ports; the number of in-links; and the ratio of document size to thenumber of tags and attributes.
Optimistic deltas for WWW latency reduction
77 Citations1997Gaurav Banga, Fred Douglis +1 more
Performance measurements of the optimistic delta system demonstrate that deltas significantly reduce latency when both sides cache the old version, and optimistic delTas can reduce latency, to a lesser degree, when content-provider service times are in the range of seconds or longer.
LibraAvailability and Latency of World Wide Web Information Servers
15 Citations1994Charles L. Viles, James C. French
The magnitude of the latencies suggest the addition of an MGET method to HTTP to help alleviate large TCP set-up times associated with the retrieval of web pages with embedded images.