Impact of the level of aggregation on response accuracy in surveys of behavioral frequency

Abstract

Through observations of AT&T data and the results of two of our own studies, we discovered interesting asymmetrical effects of the level of aggregation of the question on the response accuracy in surveys of behavioral frequency. We find that disaggregating a question to a lower, less comfortable level of aggregation creates greater uncertainty, leading to larger absolute errors in survey responses. However, if a question is asked at a higher less comfortable level, the majority of respondents escape by splitting questions down to the natural level, thereby avoiding greater uncertainty and thus responding more accurately. We argue that for greater accuracy in surveys, one should identify the natural level of aggregation at which a question should be posed. But when in doubt, it is better to ask a question at a higher level of aggregation because of possible escapability downward.