An Evaluation of a Computer Simulation in the Assessment of Physician Competence

Results indicate that all formats are equally valid, although MCQs are the most reliable methods of assessment per unit of testing time, followed by PMPs and CASE, in that order.

Abstract

This study compares physician performance on the Computer-Aided Simulation of the Clinical Encounter (CASE) with peer ratings and performance on multiple choice questions (MCQs) and patient management problems (PMPs). CASEis a simulation of the clinical encounter where the computer plays the role of the patient and the physician elicits information by entering "natural language" questions into a computer terminal. Results indicate that all formats are equally valid, although MCQs are the most reliable methods of assessment per unit of testing time, followed by PMPs and CASE, in that order. All methods measure the same or very highly correlated aspects of competence.