Panel Bias from Attrition and Conditioning: A Case Study of the Knowledge Networks Panel ⁄

Abstract

With the recent rise of Internet based public opinion studies, panel studies have been utilized with much more frequency. Although the benefits of panel methods are well known (e.g., Sharot 1991), using a panel risks bias in two ways. First, since panels rely on re-interviewing panelists, systematic panel attrition can produce a panel that is unrepresentative of the target population. Second, interviewing and reinterviewing panelists may change the opinions/behaviors of the panelists – creating unrepresentative panelists. To investigate the prevalence and impact of these possible biases, I investigate a panel that is particularly suspect to these sources of bias – the panel of Knowledge Networks. Knowledge Networks’ panelists are not only given an interactive TV appliance and Internet access, but they are also surveyed weekly. In this paper I both examine the extent (and effect) of panel attrition in Knowledge Networks’s panel over a 7 month period, as well as report the results of an experiment designed to isolate the possible opinion/behavior changes introduced by panel participation. I find little evidence of either type of bias in the Knowledge Networks panel. The study of panel methodology is important for two reasons. First, panel studies provide public opinion researchers with a valuable source of data that qualitatively differs from that offered by cross-sectional data. As Kish notes, “Only panels permit studies of individual changes; these may be needed not only for counting the frequency of changes, but also for research on the dynamics of causation and relationships”(Kish, 265). Second, given the recent rise of surveying via the internet, many companies’ research methodologies rely on the creation of internet-enabled panels. There are two important methodological issues associated with the utilization of panels: panel attrition and panel conditioning. Panel attrition refers to the possibility that panel members of certain demographics or at-