Assessing the Written Communication Skills of Medical School Graduates
Advances in Health Sciences EducationPublished 16 January 2004
John R. Boulet, Thomas A. Rebbecchi, Elizabeth C. Denton, Danette McKinley, Gerald P. Whelan
Citations72
SJR quartileQ1
SJR score1.36
SNIP2.29
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Generalizability analyses suggest that while variability in PN rating scan be attributed to the choice of rater, candidate scores are reproducible over the 10-encounter CSA.
Abstract
The ECFMG Clinical Skills Assessment (CSA) was developed to evaluate whether graduates of international medical schools (IMGs) are ready to enter graduate training programs in the United States. The patient note (PN) exercise, conducted after a 15-minute interview with a standardized patient (SP), is specifically used to assess a candidate's ability to summarize and synthesize the data collected. On a yearly basis, approximately 75,000 patient notes are reviewed and scored by physician raters. Recent changes to the PN scoring rubric, combined with enhancements to quality assurance procedures, mandate that additional evidence be provided to support the intended use of PN scores. The purpose of this study was to further investigate the psychometric adequacy of PN scores. Generalizability analyses suggest that while variability in PN ratings can be attributed to the choice of rater, candidate scores are reproducible over the 10-encounter CSA. The relationship of PN scores with other related ability measures and select candidate characteristics provides additional evidence to support the validity of the written exercise.
Keywords
Medicine
Generalizability Theory
1,100 Citations2001Robert L. Brennan
Psychological BulletinWhen small effects are impressive.
986 Citations1992Deborah A. Prentice, Dale T. Miller
Academic MedicineComparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination
721 Citations1998Glenn Regehr, Helen MacRae +2 more
Global rating scales scored by experts showed higher inter-station reliability, better construct validity, and better concurrent validity than did checklists, suggesting that global rating scales administered by experts are a more appropriate summative measure when assessing candidates on performance-based examinations.
Medical EducationPitfalls in the pursuit of objectivity: issues of reliability
282 Citations1991Cees van der Vleuten, Geoff Norman +1 more
A consistent shift can be noticed in which subjective measures are being replaced by objective measurement methods, and the issue of the presumed superiority of objective measures is addressed in two successive papers.
Medical EducationContext‐bound communication skills training: development of a new method
104 Citations2002Stephen Rollnick, Paul Kinnersley +1 more
To examine how communication skills training might be integrated into everyday clinical practice in a manner that is acceptable to clinicians, a large number of clinicians say it should be included.
Teaching and Learning in MedicineFactors influencing reproducibility of tests using standardized patients
93 Citations1989David B. Swanson, John J. Norcini
The reasons behind this result are explored, several strategies for reducing testing‐time requirements are suggested, and case‐specificity appears to be the major source of measurement error in SP‐based tests.
Advances in Health Sciences EducationAssessing the Comparability of Standardized Patient and Physician Evaluations of Clinical Skills
80 Citations2002John R. Boulet, Danette McKinley +2 more
Data gathering performance based on the number of unweighted history taking and physical examination checklist items adequately predicted the global ratings provided by physician observers, indicating that doctor observers, using mini-CEX rating scales, are able to make realistic assessments of interpersonal skills.
The Journal of Behavioral Health Services & ResearchThe accuracy of medical record documentation in schizophrenia
65 Citations2001Julie A. Cradock, Alexander S. Young +1 more
The accuracy and consistency of medical record documentation should be demonstrated before using it to evaluate care at public mental health clinics, especially for patients who were severely ill, black, or perceived as noncompliant.
Medical EducationEvaluating the spoken English proficiency of graduates of foreign medical schools
59 Citations2001John R. Boulet, Marta van Zanten +2 more
The purpose of this study was to gather additional evidence for the validity and reliability of spoken English proficiency ratings provided by trained standardized patients in high‐stakes clinical skills examination.
Medical EducationSheffield Assessment Instrument for Letters (SAIL): performance assessment using outpatient letters
55 Citations2001Jim Crossley, A.A. Howe +3 more
Well‐designed assessments of performance are urgently required for training doctors, and to provide indicators of the quality of practice.
Academic Emergency MedicineEmergency Medicine Resident Documentation: Results of the 1999 American Board of Emergency Medicine In‐Training Examination Survey
38 Citations2000John M. Howell, Carey D. Chisholm +2 more
How emergency medicine residents perform medical record documentation and coding procedures, and how well they comply with Health Care Financing Administration (HCFA) Medicare charting guidelines are investigated are assessed.
Journal of Osteopathic MedicineStandardized medical record: A new Outpatient Osteopathic SOAP Note Form: Validation of a standardized office form against physician's progress notes
36 Citations1999Sandra L Sleszynski, Thomas Glonek +1 more
The accuracy and efficiency of recording information on a one-page standardized Outpatient Osteopathic SOAP Note Form was compared with that obtained using the physician's progress notes (PPN) and it was demonstrated that the SNF easily and accurately reflected information recorded in the PPN.
Medical EducationThe performance of foreign medical graduates on the National Board of Medical Examiners (NBME) standardized patient examination prototype: a collaborative study of the NBME and the educational Commission for Foreign Medical Graduates (ECFMG)
29 Citations1999Bendavid, Klass +7 more
The purpose of the study was to explore foreign medical graduates’ performance on a clinical skills (SPX) examination.
Teaching and Learning in MedicineEstablishing the Validity of Test Score Inferences: Performance of 4th-Year U.S. Medical Students on the ECFMG Clinical Skills Assessment
28 Citations2001William R. Ayers, John R. Boulet
Investigating the relations between scores obtained on the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment (CSA) prototype and aptitude-ability measures provided additional evidence that inferences based on CSA scores are appropriate and valid.
Medical EducationPredicting communication skills with a paper‐and‐pencil test
26 Citations2002Jan van Dalen, E. Kerkhofs +5 more
This study was conducted to investigate the value of a written knowledge test of communication skills for predicting scores on a performance test of Communication skills.
Advances in Health Sciences EducationPredicting Holistic Ratings of Written Performance Assessments from Analytic Scoring
18 Citations2001Sharon Cadman Slater, John R. Boulet
This study investigates the use of regression-based techniques to predict expert judgments on a written performance task from a combination of analytic scores, and results show that a substantial proportion of variance in expert judgments can be explained by the analytic scores.
Advances in Health Sciences EducationAn Investigation of the Sources of Measurement Error in the Post-Encounter Written Scores from Standardized Patient Examinations
17 Citations1998Jack R. Boulet, Miriam Friedman Ben‐David +4 more
There is little advantage, from a reliability perspective, in using more than one trained physician, or other health professional who is adequately trained to score the patient note, and not the addition of raters.
Academic MedicineIssues of validity and reliability concerning who scores the post-encounter patient-progress note
14 Citations1997Miriam Friedman Ben‐David, John R. Boulet +4 more
Journal of the American Dietetic AssociationAutomating Clinical Dietetics Documentation
12 Citations1995Patricia Grace-Farfaglia, PAMELA ROSOW
A nutrition documentation system was developed by the researchers by reviewing medical chart structure, documentation standards, methods of risk identification, and terminology for clinical documentation style, and the resulting system affected the decision making of physicians.
Family PracticeCorrelates of family-oriented physician communications
10 Citations1999Johanna Shapiro
Findings of this investigation suggest that family orientation in the medical interview is enhanced by having more time and a shared language, as well as a generally probing, clarifying, patient-centered style on the part of the physician.
PubMedCharting a course toward legible medical records. Perfect paperwork can mean financial savings, better patient care.
9 Citations2002David Ollier Weber
Short of going fully electronic, there are various ways to clean up your medical record system to make it more efficient and less time-consuming.
PubMedSOAP to SNOCAMP: improving the medical record format.
8 Citations1995Walter L. Larimore, E V Jordan
A new acronym, "SNOCAMP," for medical record documentation is proposed, which includes subjective, objective, assessment, and plan of treatment, with the addition of nature of the presenting complaint, counseling, and medical decision-making.