Comparing objective and subjective learning curves: Judgments of learning exhibit increased underconfidence with practice.
Journal of Experimental Psychology GeneralPublished 1 January 2002
Asher Koriat, Limor Sheffer, Hilit Ma’ayan
Citations274
SJR quartileQ1
SJR score2.08
SNIP1.83
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The underconfidence-with-practice (UWP) effect was very robust across several experimental manipulations, such as feedback or no feedback regarding the correctness of the answer, self-paced versus fixed-rate presentation, different incentives for correct performance, magnitude and direction of associative relationships, and conditions producing different degrees of knowing.
Abstract
When participants studied a list of paired associates for several study-test cycles, their judgments of learning (JOLs) exhibited relatively good calibration on the 1st cycle, with a slight overconfidence. However, a shift toward marked underconfidence occurred from the 2nd cycle on. This underconfidence-with-practice (UWP) effect was very robust across several experimental manipulations, such as feedback or no feedback regarding the correctness of the answer, self-paced versus fixed-rate presentation, different incentives for correct performance, magnitude and direction of associative relationships, and conditions producing different degrees of knowing. It was also observed both in item-by-item JOLs and in aggregate JOLs. The UWP effect also occurred for list learning and for the memory of action events. Several theoretical explanations for this counterintuitive effect are discussed.
Keywords
PsychologyNeuroscience
Judgment under Uncertainty: Heuristics and Biases
23,088 Citations1975Amos Tversky, Daniel Kahneman
Dual-process theories in social psychology
6,105 Citations1999Shelly Chaiken, Yaacov Trope
Annals of NeurosciencesMemory: A Contribution to Experimental Psychology
2,809 Citations2013Hermann Ebbinghaus
The first scientific text on the psychology of memory, Hermann Ebbinghaus extended the province of systematic, experimental research to the higher mental processes.
Metacognition in Educational Theory and Practice
2,102 Citations1998
Cognitive PsychologyThe weighing of evidence and the determinants of confidence
1,891 Citations1992Dale W. Griffin, Amos Tversky
Cambridge University Press eBooksCalibration of probabilities: The state of the art to 1980
1,804 Citations1982Sarah Lichtenstein, Baruch Fischhoff +1 more
Psychological ReviewProbabilistic mental models: A Brunswikian theory of confidence.
1,448 Citations1991Gerd Gigerenzer, Ulrich Hoffrage +1 more
A comprehensive framework for the theory of probabilistic mental models (PMM theory) is proposed, which explains both the overconfidence effect and the hard-easy effect and predicts conditions under which both effects appear, disappear, or invert.
The Journal of the Abraham Lincoln AssociationMemory; a contribution to experimental psychology,
1,287 CitationsHermann Ebbinghaus
Journal of Experimental Psychology GeneralMonitoring one's own knowledge during study: A cue-utilization approach to judgments of learning.
1,231 Citations1997Asher Koriat
The Macmillan company eBooksAnimal intelligence; experimental studies
1,096 Citations1911Edward L. Thorndike
Organizational Behavior and Human PerformanceDo those who know more also know more about how much they know?
1,086 Citations1977Sarah Lichtenstein, Baruch Fischhoff
Psychological ReviewMonitoring and control processes in the strategic regulation of memory accuracy.
1,027 Citations1996Asher Koriat, Morris Goldsmith
The Journal of Nervous and Mental DiseaseAnimal Intelligence. Experimental Studies
830 Citations1912Edward L. Thorndike, Jelliffe
Organizational Behavior and Human Decision ProcessesOverconfidence: It Depends on How, What, and Whom You Ask
775 Citations1999Joshua Klayman, Jack B. Soll +2 more
Determining why some people, some domains, and some types of judgments are more prone to overconfidence will be important to understanding how confidence judgments are made.
Psychological ScienceWhen People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The “Delayed-JOL Effect”
581 Citations1991Thomas O. Nelson, John Dunlosky
Acta PsychologicaCalibration and probability judgements: Conceptual and methodological issues
550 Citations1991Gideon Keren
Law and Human BehaviorViolence risk assessment and risk communication: The effects of using actual cases, providing instruction, and employing probability versus frequency formats.
499 Citations2000Paul Slovic, John Monahan +1 more
This article describes studies designed to inform policy makers and practitioners about factors influencing the validity of violence risk assessment and risk communication and describes the response-scale effects found by Slovic and Monahan (1995).
Psychology Press eBooksImplicit Memory and Metacognition
488 Citations2014
This chapter discusses Metacognition Does Not Imply Awareness: Strategy Choice is Governed by Implicit Learning and Memory, and Neural Mechanisms For the Control and Monitoring of Memory: A Parallel Distributed Processing Perspective.
Journal of Experimental Psychology GeneralThe mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index.
447 Citations1998Aaron S. Benjamin, Robert A. Bjork +1 more
In each of 3 experiments, participants' predictions of their own future recall performance were examined under conditions in which probability or speed of retrieval at one time or on one task is known to be negatively related to retrieval probability on a later task.
Consciousness and CognitionThe Feeling of Knowing: Some Metatheoretical Implications for Consciousness and Control
432 Citations2000Asher Koriat
Examination of the cues that affect noetic feelings suggest that not only do these feelings inform controlled action, but they are also informed by feedback from the outcome of that action.
Journal of Experimental Psychology Learning Memory and CognitionMetacognitive and control strategies in study-time allocation.
382 Citations2000Lisa K. Son, James C. Metcalfe
How people's metacognitive judgments influence subsequent study-time-allocation strategies is investigated, indicating that people allocate more study time to judged-difficult than to judging-easy items--consistent with extant models of study- time allocation.
Psychological ReviewNaive empiricism and dogmatism in confidence research: A critical examination of the hard–easy effect.
368 Citations2000Peter Juslin, Anders Winman +1 more
A quantitative review of studies with 2-alternative general knowledge items demonstrates that there is very little support for a cognitive-processing bias in these data and near elimination of the hard-easy effect when there is control for scale-end effects and linear dependency.
Journal of Memory and LanguageMemory predictions are based on ease of processing
321 Citations1989Ian Begg, Susanna Duft +3 more
Memory & CognitionImportance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect
304 Citations1992John Dunlosky, Thomas O. Nelson
It is found that the kind of cue that is used for JOLs is critical, and to have the greatest accuracy at predicting eventual recall, a person should make J OLs not immediately after study but, instead, shortly after study (i.e., delayedJOLs).
Journal of Experimental Psychology Learning Memory and CognitionAllocation of self-paced study time and the "labor-in-vain effect."
270 Citations1988Thomas O. Nelson, R. Jacob Leonesio
The major findings were: ease-of-learning judgments and feeling- of-knowing judgments are reliably related to study-time allocation, with more self-paced study time being allocated to the supposedly more difficult items.
Journal of Experimental Psychology Learning Memory and CognitionMetacognitive and control strategies in study-time allocation.
254 Citations2000Lisa K. Son, Janet Metcalfe
Psychological ReviewInterpreting the influence of implicitly activated memories on recall and recognition.
238 Citations1998Douglas L. Nelson, Vanesa M. McKinney +2 more
Experiments designed to test predictions of the model indicate that the associates of a familiar word can exert a powerful effect on its cued recall and recognition.
Journal of Memory and LanguageDoes the Sensitivity of Judgments of Learning (JOLs) to the Effects of Various Study Activities Depend on When the JOLs Occur?
226 Citations1994John Dunlosky, Thomas O. Nelson
Psychological ScienceUtilization of Metacognitive Judgments in the Allocation of Study During Multitrial Learning
215 Citations1994Thomas O. Nelson, John Dunlosky +2 more
Computer-controlled allocation of restudy based on people's JOLs was equivalent to most people's own allocation of Restudy and was more effective than a computer- controlled allocation based on normative performance, indicating that people's metacognitive monitoring of idiosyncratic knowledge has functional utility in causal chains for learning.
The American Journal of PsychologyMemory for Actions
215 Citations2001Marc Jeannerod, Johannes Engelkamp
Theoretical Integration I: Actions as Described by the Multimodal Theory and its Implications for Theories on Episodic Memories, and the Inclusion of Retrieval Processes.
Psychology and AgingAge-related differences in absolute but not relative metamemory accuracy.
212 Citations1997Lisa Tabor Connor, John Dunlosky +1 more
Correlational techniques revealed that older adults monitored learning effectively and showed equally accurate immediate judgments of learning (JOLs) and showed equivalent upgrading in the accuracy of their global prediction from before to after study of test materials.
Journal of Experimental Psychology Learning Memory and CognitionDo different metamemory judgments tap the same underlying aspects of memory?
199 Citations1990R. Jacob Leonesio, Thomas O. Nelson
Acta PsychologicaWhat makes people study more? An evaluation of factors that affect self-paced study
183 Citations1998John Dunlosky, Keith W. Thiede
Under several conditions people's allocation of study time appeared sub-optimal, particularly when points awarded for recalling an item increased and when instructions emphasized mastering each item rather than quickly learning each item.
Training Programs to Improve Learning in Later Adulthood: Helping Older Adults Educate Themselves
180 Citations1998John Dunlosky, Christopher Hertzog
Bulletin of the Psychonomic SocietyWhen you know that you know and when you think that you know but you don’t
173 Citations1980Eugene B. Zechmeister, John J. Shaughnessy
The American Journal of PsychologyJudgments of Knowing: The Influence of Retrieval Practice
166 Citations1980Joseph F. King, Eugene B. Zechmeister +1 more
Memory & CognitionThe rereading effect: Metacomprehension accuracy improves across reading trials
158 Citations2000Katherine A. Rawson, John Dunlosky +1 more
It is discovered that rereading improves metacomprehension accuracy, and correlations between comprehension ratings and test scores were reliably greater for participants who reread texts than for Participants who read texts only once.
Organizational Behavior and Human Decision ProcessesThe Hard–Easy Effect in Subjective Probability Calibration
146 Citations1996Liana Suantak, Fergus Bolger +1 more
Psychological ReviewVisual perception of dynamic properties: Cue heuristics versus direct-perceptual competence.
144 Citations2000Sverker Runeson, Peter Juslin +1 more
Although pretraining results were mixed, the invariant-based model was empirically confirmed after a minimal amount of training and entails the use of advanced kinematic information in a direct-perceptual ("sensory") mode of apprehension, in contrast to beginners' use of simpler cues in an inferential ("cognitive") mode.
Journal of Experimental Psychology GeneralStrategies in study time allocation: Why is study time sometimes not effective?
141 Citations1993Giuliana Mazzoni, Cesare Cornoldi
Journal of Experimental Psychology Learning Memory and CognitionMetamemory: Monitoring future recallability during study.
136 Citations1984Eugene A. Lovelace
Journal of Experimental Psychology Learning Memory and CognitionJudgments of learning are affected by the kind of encoding in ways that cannot be attributed to the level of recall.
124 Citations1995Giuliana Mazzoni, Thomas O. Nelson
Memory & CognitionDo memorability ratings affect study-time allocation?
122 Citations1990Giuliana Mazzoni, Cesare Cornoldi +1 more
The results of three experiments support the main hypotheses: (1) recall increases with increasing JOL, (2) restudy increases JOL accuracy, and (3) study time is related to JOL.
Journal of GerontologyAging and Metamemory: the Roles of Relatedness and Imagery
121 Citations1982Jan C. Rabinowitz, Brian P. Ackerman +2 more
It was concluded that differences in metamemorial skills are not responsible for age differences in memory and both old and young adults were insensitive to the beneficial effects of using interactive imagery at encoding.
Cognitive DevelopmentDevelopmental trends in children's memory monitoring
112 Citations2000Wolfgang Schneider, Mechtild Visé +2 more
The European Journal of Cognitive PsychologyAn explanation of the hard-easy effect in studies of realism of confidence in one's general knowledge
109 Citations1993Peter Juslin
Psychology Press eBooksRetrieval Fluency as a Metacognitive Index
106 Citations2014Aaron S. Benjamin, Robert A. Bjork
Psychology and AgingAge differences in predictions and performance on a cued recall task.
100 Citations1989Raymond J. Shaw, Fergus I. M. Craik
It is concluded that a memory monitoring deficit is not likely to be responsible for age differences in memory and the relative recallability of specific words was well assessed by both age groups.
Organizational Behavior and Human Decision ProcessesItem Difficulty, Discrimination, and the Confidence-Frequency Effect in a Categorical Judgment Task
93 Citations1995Sandra L. Schneider
Organizational Behavior and Human Decision ProcessesAvailability of Information and the Aggregation of Confidence in Prior Decisions
92 Citations1996Jonathan Treadwell, Thomas O. Nelson
Journal of Experimental Psychology Learning Memory and CognitionJudgments of learning are affected by the kind of encoding in ways that cannot be attributed to the level of recall.
90 Citations1995Giuliana Mazzoni, Thomas O. Nelson
The authors investigate the theoretical question of whether different kinds of encoding can affect judgments of learning (JOLs) beyond any indirect effects arising from the differences those kind of encoding produce on the likelihood of recall and find that JOLs are theoretically rich and are based on more than whatever underlies the likelihoodof recall.
Memory & CognitionGenerating makes words memorable, but so does effective reading
87 Citations1991Ian Begg, Ede Vinski +2 more
The experiments did result in benefits for generating as opposed to reading, but only when the readers processed the words poorly—by pronouncing them, and when the Readers processed the items well, by imagining them, generating was no better than reading.
Journal of Verbal Learning and Verbal BehaviorMemory monitoring accuracy and modification of rehearsal strategies
80 Citations1981John J. Shaughnessy
Psychological ResearchThe contextualization of input and output events in memory
71 Citations1991Asher Koriat, Hasida Ben‐Zur +1 more
Acta PsychologicaTradeoff of semantic relatedness and degree of overlearning: Differential effects on metamemory and on long-term retention
60 Citations1997Marie Carroll, Thomas O. Nelson +1 more
Findings imply that the information tracked by metacognitive monitoring judgments is different for JOLs than for FOKs, with the J OLs based more on semantic relatedness and less on the degree of learning during acquisition.
Journal of Applied PsychologyRealism in eyewitness confidence as a function of type of event witnessed and repeated recall.
37 Citations1997Pär Anders Granhag
Psychological ResearchThe subjective organization of input and output events in memory
27 Citations1998Asher Koriat, Shiri Pearlman-Avnion +1 more
The results suggest that enactment may lead to more differentiated memory traces, resulting in more accurate recall, and in all conditions a positive correlation was obtained between recall and subjective organization.
Psychological ReportsOrienting Tasks Affect Recall Performance More Than Subjective Impressions of Ability to Recall
12 Citations1975James E. Cutting