Anchoring and Adjusting in Questionnaire Responses

Abstract

When ordering items on attitude/opinion questionnaires, do survey designers bias respondents’ answers by the mere act of choosing to organize their survey in a particular way? We hypothesize that, under specific frequently-occurring conditions, respondents employ an anchoring and adjusting strategy in which their response to an initial survey item provides a cognitive anchor from which they (insufficiently) adjust in answering the subsequent item. Three experiments indicate that respondents anchor and insufficiently adjust in certain situations, anchoring and adjusting leads to higher inter-item correlations between adjacent items, and these inflated correlations can (spuriously) increase the reliability estimate of the scale that they comprise and affect the resultant correlations with other measures. These effects are not consistently accounted for by a “superior memory search” explanation. In organizing their surveys, researchers may wish to combat this bias by intermixing items designed for different, but related constructs.