Web Traffic Logs: An Imperfect Resource for Evaluation

This paper enumerates a number of deficiencies in most common, publicly available request logs and investigates some of the additional insights available from logs containing full content, and how they might apply to content-based prefetching proxies.

Abstract

Replaying or simulation using web traffic logs are common methods of evaluating proxy caches. Request traces are also used to characterize typical web workloads. Unfortunately, most common, publicly available request logs have serious deficiencies because of what is omitted from the logs. In this paper we enumerate a number of these deficiencies and examine the benefits of more complete logs. Additionally, we investigate some of the additional insights available from logs containing full content, and how they might apply to content-based prefetching proxies.