The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items

Abstract

The paper argues for an ecological approach to realism of confidence in general knowledge. It is stressed that choice of answer to almanac items and confidence judgments derive from knowledge structures formed by adaptation to a natural environment. People are regarded as well-calibrated to their natural environments and the overconfidence phenomenon is seen as a consequence of the procedures involved in the creation of "traditional" general knowledge items, rather than as the result of a cognitive bias. An experiment is reported showing that when items are informally selected by human "selectors," instructed to select items that differentiate between more and less knowledgeable subjects, we observe poor calibration and the usual overconfidence phenomenon. However, when the item selection is "debiased" and the objects to be compared in the almanac items are selected randomly from a natural environment the overconfidence phenomenon disappears and we observe good calibration, and good resolution. These results support an ecological approach to realism of confidence and suggest that general procedures for debiasing overconfidence may be unwarranted.