Use of Standardized Patients in Clinical Assessments: Recent Developments and Measurement Findings
Educational ResearcherPublished 1 April 1994Open access
Nu Viet Vu, Howard S. Barrows
Citations183
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results of large-scale standardized patient-based performance assessments are presented and discussed in terms of their feasibility, reliability, validity, and implications for assessing competence in other professions.
Abstract
This article reviews recent developments and measurement findings on the use of live patient simulations or “standardized patients” in performance examinations to assess the competence of medical professionals. The results of large-scale standardized patient-based performance assessments are presented and discussed in terms of their feasibility, reliability, validity, and implications for assessing competence in other professions.
Keywords
Medicine
Educational ResearcherComplex, Performance-Based Assessment: Expectations and Validation Criteria
914 Citations1991Robert L. Linn, Eva L. Baker +1 more
Teaching and Learning in MedicineAssessment of clinical skills with standardized patients: State of the art
574 Citations1990Cees van der Vleuten, David B. Swanson
This article provides a comprehensive review of large‐scale studies of the psychometric characteristics of SP‐based tests and indicates that the major source of measurement error is variation in examinee performance from station to station.
Medical EducationThe effect of assessments and examinations on the learning of medical students
494 Citations1983David Newble, K. Jaeger
A situation where an alteration in the final‐year assessment scheme led to changes in student learning activities which were the exact opposite of those intended is described.
Review of Educational ResearchShifting Conceptions of Validity in Educational Measurement: Implications for Performance Assessment
416 Citations1992Pamela Moss
Applied Measurement in EducationQuality Control in the Development and Use of Performance Assessments
275 Citations1991Stephen B. Dunbar, Daniel Koretz +1 more
Medical EducationPsychometric characteristics of the objective structured clinical examination
232 Citations1988David Newble, David B. Swanson
Detailed psychometric analyses of OSCEs which were administered as part of a well‐established final‐ year examination are reported, showing generalizability theory guided investigation of test reliability.
Annals of Internal MedicineAssessing Clinical Skills of Residents with Standardized Patients
226 Citations1986Paula L. Stillman, DAVID B. SWANSON +16 more
Results indicated that reproducible assessment of the clinical skills could be achieved in approximately 1 day of testing time using standardized patients, and low correlations between standardized-patient-based measures of clinical skills and other evaluation techniques suggested that standardized patients provided unique information.
Educational ResearcherPerformance Assessments: Political Rhetoric and Measurement Reality
184 Citations1992Richard J. Shavelson, Gail P. Baxter +1 more
Educational Measurement Issues and PracticeUsing Performance Assessment for Accountability Purposes
150 Citations1992William A. Mehrens
Academic MedicineA comparison of resident performance on real and simulated patients
130 Citations1982G R Norman, Peter Tugwell +1 more
Four actual patients with chronic stable findings were used in the study; four simulated patients were then programmed to present the same problem, and residents correctly identified 67 percent of the patients as real or simulated against a chance figure of 50 percent.
Academic MedicineAn objective structured clinical examination for the licentiate
124 Citations1992Richard K. Reznick, S Smee +8 more
A pilot examination to test the feasibility, reliability, and validity of running a multisite, two-form, four-hour, 20-station OSCE for national licensure indicates that a full-scale national administration of an OSce for licensure is feasible using the model developed.
The American Journal of MedicineAn objective measure of clinical performance
119 Citations1987Emil Petrusa, Thomas A. Blackwell +4 more
It is suggested that clinical education is inconsistent and that a profile of an examinee's performance is more accurate than a single overall score, and this standardized objective examination of clinical skills is feasible for use in training programs and will provide reliable and valid data on clinical performance not available through typical methods.
Medical EducationThe accuracy of standardized patient presentation
117 Citations1991Robyn Tamblyn, D J Klass +2 more
There was a systematic trend for patients training at Southern Illinois to be more accurate in their presentation than patients trained at the University of Manitoba, and there were significant differences in the accuracy score achieved by patients trained together for the same case in 6 of 35 possible comparisons.
Academic MedicineSix years of comprehensive, clinical, performance-based assessment using standardized patients at the Southern Illinois University School of Medicine
117 Citations1992Nu Viet Vu, Howard S. Barrows +4 more
The results indicate that this kind of clinical performance-based examination can discriminate a wide range of students' clinical performances, and provide evidence for the examinations' test security, content validity, construct validity, and reliability.
Assessment & Evaluation in Higher EducationASSESSMENT OF CLINICAL COMPETENCE: WRITTEN AND COMPUTER‐BASED SIMULATIONS
112 Citations1987David B. Swanson, John J. Norcini +1 more
Written and computer‐based clinical simulations have been used in the health professions to assess aspects of clinical competence for many years, but a review of the dozens of studies of their psychometric characteristics finds little evidence to justify their continued use.
BMJSimulated patients in general practice: a different look at the consultation.
105 Citations1987Jan‐Joost Rethans, C P van Boven
The results of this study show that the use of simulated patients seems to show the efficient performance of general practitioners in practice.
PubMedAssessment of the performance of general practitioners by the use of standardized (simulated) patients.
96 Citations1991Jan‐Joost Rethans, F. Sturmans +2 more
The results show that standardized patients may be the method of choice in the assessment of the quality of actual care of doctors, and that the substandard scores of the doctors do not reflect inadequate competence, but are a result of the difference between competence and performance.
Teaching and Learning in MedicineFactors influencing reproducibility of tests using standardized patients
93 Citations1989David B. Swanson, John J. Norcini
The reasons behind this result are explored, several strategies for reducing testing‐time requirements are suggested, and case‐specificity appears to be the major source of measurement error in SP‐based tests.
Teaching and Learning in MedicineSources of unreliability and bias in standardized‐patient rating
76 Citations1991Robyn Tamblyn, D J Klass +2 more
In tests of clinical competence, standardized patients can be used to present the clinical problem and rate actions taken by the examinee in the patient encounter and have the potential to contribute to unreliability and bias in measurement.
Medical EducationDirect, standardized assessment of clinical competence
75 Citations1987Reed G. Williams, Howard S. Barrows +5 more
Standardized direct measures of clinical competence were developed to control factors and allow direct comparisons of student performance and reliability studies indicated that new cases can be substituted in subsequent years with confidence that scores will maintain similar meaning.
PubMedLarge-scale use of an objective, structured clinical examination for licensing family physicians.
75 Citations1992Paul Grand’Maison, J Lescop +2 more
The integration of such a large-scale OSCE into a licensing examination and the results obtained show that assessment of clinical competence for licensing purposes is feasible and may help other organizations that are developing OSCEs for summative purposes.
JAMAEvaluation of Physical Examination Skills
73 Citations1987Diane L. Elliot
Limits in the current methods of evaluating performance are demonstrated and the need to examine systems of trainee assessment is pointed out.
Medical EducationA comparison of behaviour on simulated patients and patient management problems
73 Citations1981Geoff Norman, John Feightner
The findings of this study are consistent with previous studies comparing performance on PMPs to oral examinations and medical records, and raise some concern about the use of PMPs as a measure of competence in certification and licensure decisions.
Medical EducationObjective measurement of clinical performance
70 Citations1985Geoffrey R. Norman
The paper describes a series of studies which were designed to establish the reliability and concurrent validity of the aggregate score technique.
PubMedA method for introducing standardized (simulated) patients into general practice consultations.
67 Citations1991Jan‐Joost Rethans, R Drop +2 more
It is concluded that the standardized patient method, following the step-by-step procedure described, is feasible in actual practice and ready to cooperate in future studies with standardized patients.
Medical EducationTraining and experience of examiners
65 Citations1989Cees van der Vleuten, Scheltus J. Van Luyk +2 more
An experiment on accuracy of scoring in which provision of training and background of examiners are systematically varied is reported in which experienced teaching staff, medical students and lay subjects were randomly assigned to either training or no‐training groups.
Academic MedicinePerformance on PMPs and performance in practice
48 Citations1980Gordon G. Page, David Fielding
Between PMP performance and practice performance there were major inconsistencies, characterized by substantially greater errors of omission and commission in the practice setting.
The Medical Journal of AustraliaGENERAL PRACTITIONERS AND PSYCHOSOCIAL PROBLEMS AN EVALUATION USING PSEUDOPATIENTS
44 Citations1974Alan Owen, Robin C. Winkler
Recommendations are made for its further use on a more regular basis as a monitoring device for the quality of primary care.
Physical TherapyEvaluating Clinical Competence in the Health Professions
44 Citations1979Lucille Daniels
Archives of Internal MedicineA Diagnostic Fourth-Year Performance Assessment
41 Citations1987Paula L. Stillman
A standardized, performance-based test battery aimed directly at assessing the critical aspects of clinical competence required for graduation from medical school, indicating that a stable, reproducible assessment of clinical skills can be achieved in a one- to two-day test battery.
Academic MedicineAn assessment of the consistency and accuracy of standardized patientsʼ simulations
40 Citations1987Nu Viet Vu, David E. Steward +1 more
Academic MedicineA factor analysis study of performance of first-year residents
38 Citations1986S J Verhulst, Jerry A. Colliver +2 more
Academic MedicineTest security in examinations that use standardized-patient cases at one medical school
37 Citations1991Jerry A. Colliver, Howard S. Barrows +4 more
The results showed no consistent, systematic changes in case means across the five groups tested at different times throughout the examination period, and thus provide no evidence of serious, widespread violations of test security.
Archives of Internal MedicineSharing of Information by Students in an Objective Structured Clinical Examination
37 Citations1991P J Rutala
Six administrations of an Objective Structured Clinical Examination designed to measure skills were studied andalyses were conducted to detect changes in scores over the administrations as well as over the 8.5 hours of each day of testing.
Academic MedicineEvaluation of medical students during a clinical clerkship in internal medicine
34 Citations1978W J OʼDonohue, Jon F. Wergin
Although the individual clinical ratings had low reliability, averages taken over a period of time were more consistent and have importance not only in the assessment of the clinical competence of medical students but also in other areas of granting medical credentials and certification.
Military PsychologyGeneralizability of Job Performance Measurements: Marine Corps Rifleman
33 Citations1990Richard J. Shavelson, Paul W. Mayberry +2 more
Academic MedicinePositive effects of a clinical performance assessment program
32 Citations1991Paula L. Stillman, Heather-Lyn Haley +2 more
Although it is difficult to claim cause and effect, it is clear that since the inception of this exercise, the faculty have made a conscious effort to improve students' clinical skills by providing increased observation and feedback.
Journal of Instructional DevelopmentSimulation in health sciences education
32 Citations1985Geoffrey R. Norman, Linda Muzzin +2 more
Content specificity seems to be a fundamental problem in assessing clinical problem solving ability with simulations, although further research is necessary to determine those characteristics of the simulation which result in effective transfer to the clinical setting.
Academic MedicineFinancing Medical Education, 1989–90
29 Citations1991PAUL JOLLY, Leanne D. Jolin +2 more
Journal of the Association of American Medical CollegesComparing Examination Scores and Personality Types of Students from Three Classes
25 Citations1991Gaynor C. Wild, Betty Skipper
Academic MedicineIs Test Security an Issue in a Multistation Clinical Assessment?—A Preliminary Study
22 Citations1991Beth Dawson‐Saunders, PAULA L. STILLMAN +6 more
Teaching and Learning in MedicineThree studies of the effect of multiple standardized patients on intercase reliability of five standardized‐patient examinations
18 Citations1990Jerry A. Colliver, Linda J. Morrison +5 more
Standardized‐patient (SP) cases have been used in a postclerkship examination to assess clinical competence of five classes of senior medical students at Southern Illinois University School of Medicine to determine the effect of multiple SPs on intercase reliability.
Academic MedicineSequential testing with a performance-based examination using standardized patients
17 Citations1991Jerry A. Colliver, Terrill A. Mast +2 more
Evaluation & the Health ProfessionsStrategies in Comparison of Methods for Scoring Patient Managementproblems
16 Citations1988George D. Webster, Judy A. Shea +3 more
The results demonstrated high correlations between all scoring systems and high correlations with simultaneous multiple-choice (MCQ) scores, and none of the PMPs cores provided significant unique incremental validity not provided by MCQ scores.
Evaluation & the Health ProfessionsMedical Problem-Solving Assessment
15 Citations1979Nu Viet Vu
The purpose of this article is to review and discuss the different methods and instru ments created to assess medical problem- solving.
Academic MedicineEvaluating the clinical skills of foreign medical school graduates participating in an internship preparation program
13 Citations1990A I Rothman, Robert Cohen +1 more
Repeating the same examination permitted investigation of the predictive and construct validities of the clinical skills tests, as well as the stability of the validity and high reliability estimates that emerged from the testing of the first cohort.
Academic MedicineEffects of using two or more standardized patients to simulate the same case on case means and case failure rates
13 Citations1991Jerry A. Colliver, Randall Robbs +1 more
There is a need for caution in the interpretation of scores obtained from a case checklist completed by multiple SPs, particularly in regard to making pass-fail decisions.
Academic MedicineClinical performance-based test sensitivity and specificity in predicting first-year residency performance
13 Citations1993Nu Viet Vu, Linda H. Distlehorst +2 more
The results suggest that the standardized-patient-based PCX is a useful indicator of students' readiness for and performance in residency.
Academic MedicineThe Influences of Student and Standardized Patient Genders on Scoring in an Objective Structured Clinical Examination
12 Citations1991Beth Dawson‐Saunders, PAUL J. RUTALA +3 more
Academic MedicineGeneralizability of standardized patientsʼ satisfaction ratings of their clinical encounter with fourth-year medical students
8 Citations1990Nu Viet Vu, M L Marcy +2 more
Academic MedicineA survey of study methods, preparation time, test-taking strategies, and perceptions of test validity on a clinical performance-based examination
8 Citations1992L E Shirar, Nu Viet Vu +2 more
Teaching and Learning in MedicineA comparison of second‐ and fourth‐year medical students on a standardized‐patient examination of clinical competence: A construct validity study
7 Citations1995Amber Barnhart, M L Marcy +2 more
The usefulness of passing rates and effect‐size measures as a means of enhancing the typically weak evidence for validity provided by group‐differences studies of construct validity is demonstrated.
Teaching and Learning in MedicineFalse‐negative and false‐positive rates resulting from measurement error for a mastery examination using standardized‐patient cases
1 Citations1992Jerry A. Colliver, Nu Viet Vu +4 more
Methods are described for estimating the false‐negative and false‐positive rates resulting from measurement error for a mastery examination and the reliabilities of the pass‐fail decisions based on a recently adopted standard‐setting procedure.