ANALYSES/LITERATURE REVIEWS: Evaluating Competence Using a Portfolio: A Literature Review and Web-Based Application to the ACGME Competencies
Teaching and Learning in MedicinePublished 1 October 2004
Carol Carraccio, Robert Englander
Citations130
SJR quartileQ1
SJR score0.95
SNIP1.48
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A Web-based evaluation portfolio for residency training is described here in the hope of facilitating this process for others and has the ability to provide educators with a research infrastructure to practice evidence-based education.
Abstract
Web-based portfolio assessment provides an ideal venue for the evaluation of competence and has the ability to provide educators with a research infrastructure to practice evidence-based education.
Keywords
Social SciencesMedicine
Bibliothèque et Archives nationales du Québec (Québec government)Educating the reflective practitioner
11,390 Citations1987Donald A. Schön
Building on the concepts of professional competence that he introduced in his classic The Reflective Practitioner, Schon offers an approach for educating professional in all areas that will prepare them to handle the complex and unpredictable problems of actual practice with confidence, skill, and care.
Academic MedicineShifting Paradigms
967 Citations2002Carol Carraccio, Susan D. Wolfsthal +3 more
The authors found that in the 1970s and 1980s much attention was given to the need for and the development of professional competencies for many medical disciplines, little attention was devoted to defining the benchmarks of specific competencies, how to attain them, or the evaluation of competence.
Academic MedicineA review of the validity and accuracy of self-assessments in health professions training
532 Citations1991Michael J. Gordon
The validity of self-assessed performance was found to be low to moderate and did not improve with time in conventional health professions training programs, but effective training to improve validity and accuracy is available and feasible.
Annals of Internal MedicineThe Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation
514 Citations1995John J. Norcini, Linda L. Blank +2 more
The logistic and psychometric properties of a variation on the traditional CEX format, called the mini-CEX, are explored, designed around both the skills that residents most often need in actual patient encounters and the educational interactions that attending physicians routinely have with residents during teaching rounds.
Advances in Health Sciences EducationMeasuring Self-assessment: Current State of the Art
403 Citations2002Myléne Ward, Larry D. Gruppen +1 more
Examining the methodological issues that plague the measurement of self-assessment ability and presenting several strategies that address these methodological problems within the current paradigm, an alternative conceptualization ofSelf-Assessment is proposed and its associated methods are described.
Medical TeacherAMEE Medical Education Guide No. 24: Portfolios as a method of student assessment
227 Citations2001M. Friedman Ben David, M H Davis +4 more
Reasons for using portfolios for assessment purposes include the impact that they have in driving student learning and their ability to measure outcomes such as professionalism that are difficult to assess using traditional methods.
Medical EducationClarifying the concepts of confidence and competence to produce appropriate self‐evaluation measurement scales
206 Citations2000Jane A. Stewart, Cath O’Halloran +4 more
The findings of a small‐scale qualitative study which explored the terms ‘confidence’ and ‘competence’ as useful measures in a self‐evaluation scale are discussed.
Medical TeacherThe use of portfolio learning in medical education
199 Citations1998David Snadden
The history of portfolios, how to develop a portfolio to assist learning or for professional development, and difficulties associated with the lack of standardization of the content of portfolios in terms of developing assessment instruments are described.
Archives of Pediatrics and Adolescent MedicineThe Objective Structured Clinical Examination
192 Citations2000Carol Carraccio, Robert Englander
The combination of the OSCE, standardized board examinations, and direct observation in the clinical setting has the potential to become the "gold standard" for measuring physician competence.
Annals of Internal MedicineMethods for Evaluating the Clinical Competence of Residents in Internal Medicine: A Review
184 Citations1998Eric S. Holmboe, Richard E. Hawkins
The major instruments available to educators for the assessment of clinical competence during residency are reviewed: the In-Training Examination, the ABIM Longitudinal Evaluation Form, medical record audits, the clinical evaluation exercise (CEX), and the use of standardized patients (Table 1).
Medical EducationPortfolios in continuing medical education – effective and efficient?
178 Citations1999Nigel Mathers, Maggie Challis +2 more
A cross over comparison between ‘traditional’ continuing medical education activities and portfolio‐based learning in general practice is described.
Medical TeacherPortfolio assessment in medical students' final examinations
168 Citations2001M H Davis, Miriam Friedman Ben‐David +6 more
It is concluded that portfolio assessment is a powerful approach to assessing a range of curriculum outcomes not easily assessed by other methods and is worthy of inclusion in the assessor's toolkit.
Academic MedicineResident-patient interactions
103 Citations1994James O. Woolliscroft, Joy Howell +2 more
Patients, attending physicians, program supervisors, and nurses view differently the humanistic attributes of residents as they interact with patients, and large numbers of patients and attending physicians would be needed to obtain reproducible ratings.
Medical EducationThe use of portfolios for assessment of the competence and performance of doctors in practice
102 Citations2002Tim Wilkinson, Maggie Challis +5 more
The use of portfolios can potentially provide flexibility in the summative assessment of doctors in practice and some of the issues involved in developing such a system are discussed.
Medical EducationEducational portfolios in the assessment of general practice trainers: reliability of assessors
92 Citations1999John Pitts, Colin Coles +1 more
This paper reports a project that assessed a series of portfolios assembled by a cohort of participants attending a course for prospective general practice trainers.
Medical EducationAn Internet‐based learning portfolio in resident education: the KOALA™ multicentre programme
86 Citations2000Michael Fung Kee Fung, Mark Walker +5 more
The Computerized Obstetrics and Gynecology Automated Learning Anaalysis (KOALA) is described, a multicentre, Internet‐based learning portfolio and its effects on residents’ perception of their self‐directed learning abilities are determined.
Journal of General Internal MedicineThe inter-rater reliability and internal consistency of a clinical evaluation exercise
85 Citations1992Frank J. Kroboth, Barbara H. Hanusa +6 more
Although the CEX is a valuable didactic tool, its psychometric properties preclude reliable assessment of clinical skills as a one-time observation.
Medical EducationPortfolio‐based learning: continuing medical education for general practitioners — a mid‐point evaluation
77 Citations1997Maggie Challis, Nigel Mathers +2 more
This paper addresses the process of portfolio development at the mid‐point of a year‐long trial to ascertain the strengths, weaknesses and possible future development of such a process within the context of continuing medical education.
Medical EducationA randomized controlled study of portfolio learning in undergraduate cancer education
75 Citations1998Finlay, Maughan +1 more
The project aimed to provide the student with a holistic understanding of the impact of the disease and its treatment on the patient and family, and the natural history of malignant disease, through long‐term personal experience of a cancer patient.
Journal of General Internal MedicineThe feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents
74 Citations1994Robyn Tamblyn, S. Benaroya +4 more
Journal of General Internal MedicineA new instrument for patients’ ratings of physician performance in the hospital setting
73 Citations1989Dale A. Matthews, Alvan R. Feinstein
When the instrument was administered to 131 randomly chosen medical inpatients, the results showed that technical competence and interpersonal (or humanistic) qualities were equally valued, and that physicians received high ratings for most features of performance.
Journal of Continuing Education in the Health ProfessionsLearning portfolios: Potential to assist health professionals with self-directed learning
71 Citations1996John Parboosingh
The potential of the portfolio to enhance the quality of an individual's learning activities while respecting their desire for control of the education process is explored.
Medical TeacherEnhancing reliability in portfolio assessment: 'shaping' the portfolio
70 Citations2001John Pitts, Colin Coles +1 more
The conclusion remains that while portfolios might be valuable as resources for learning, as assessment tools they should be treated as problematic.
Medical TeacherEnhancing reliability in portfolio assessment: discussions between assessors
64 Citations2002John Pitts, Colin Coles +2 more
Overall reliability of a global pass/refer judgement improved from a kappa of 0.26 (fair) using individual assessment, to 0.5 (moderate) with paired discussants.
Medical EducationPortfolios as a learning tool in obstetrics and gynaecology undergraduate training
63 Citations2001Kirsti Lonka, Virpi Slotte +5 more
A structured portfolio for medical students to use during their obstetrics and gynaecology undergraduate training to support the learning process of the students and to enhance teaching.
Medical TeacherWhat Is… Normative versus Criterion-referenced Assessment
59 Citations1989Jeffrey Turnbull
It is recommended that criterion-referenced testing be the primary method of evaluation, as it best meets the objectives of the medical school by emphasising the achievement of clearly established external standards and, thereby, ensuring a high quality of performance.
Journal of General Internal MedicineA new rating form for use by nurses in assessing residents’ humanistic behavior
56 Citations1991Paula S. Butterfield, Ernest L. Mazzaferri
Academic MedicineComparing peer and faculty evaluations in an internal medicine residency
51 Citations1994G M Van Rosendaal, P. A. Jennett
Comparing in-training evaluations of residents by their peers with evaluations by faculty preceptors in an internal medicine residency suggests that there were differences in the quality of evaluation between the peers and faculty, or Differences in the standards or expectations of the two groups.
Academic MedicineThe triple-jump examination as an assessment tool in the problem-based medical curriculum at the University of Hawaii
47 Citations1993Roger M. Smith
That the administrators' subjective scores for all problems were comparable not only suggests that the administrators were able to adjust for problem variability but also reinforces the appropriateness of using subjective assessments for the TJ examination.
Academic MedicineDevelopment and Implementation of a Web-based Evaluation System for an Internal Medicine Residency Program
42 Citations2001Mark E. Rosenberg, Kathleen Watson +4 more
A World Wide Web-based electronic evaluation system for the internal medicine residency program at the University of Minnesota was easy to use and could quickly and confidentially identify performance problems of residents and faculty from large numbers of evaluations.
Evaluation & the Health ProfessionsPortfolios and Professional Development in the Health Professions
42 Citations1994Gail M. Jensen, Coleen Saylor
Evaluation results obtained through a qualitative content analysis of student portfolios and studentfeedback data suggest that student reflection on professional development was promoted, although the portfolios varied in their structure and complexity.
Academic MedicineStudy of physiciansʼ use of a software program to create a portfolio of their self-directed learning
39 Citations1996Craig M. Campbell, John Parboosingh +5 more
Academic MedicineNurses as evaluators of the humanistic behavior of internal medicine residents
34 Citations1987Paula S. Butterfield, Ernest L. Mazzaferri +1 more
PEDIATRICSDefining Successful Performance Among Pediatric Residents
34 Citations1990Elizabeth M. Altmaier, Gail A. McGuinness +4 more
The results of this study have implications for the evaluation and selection of residents and suggest that pediatric program directors and faculty must give attention to the means by which noncognitive skills are fostered in residents.
Journal of General Internal MedicineThe use of nurses to evaluate houseofficers’ humanistic behavior
30 Citations1990Craig Kaplan, Robert M. Centor
Nurses can provide information about humanistic behavior that will allow program directors to discriminate among different levels of houseofficer behavior, and these ratings are affected by gender and by the type of unit from which they are obtained.
Academic MedicineUse of nonphysician staff to evaluate humanistic behavior of internal medicine residents and faculty members
24 Citations1986Lawrence S. Linn, Robert K. Oye +2 more
Academic MedicineMEOW
15 Citations2001Christopher B. White, T. Andrew Albritton +1 more
The purpose of MEOW is to provide on the Internet information that third-year medical students can easily access from any clerkship site, and has been very helpful in accomplishing two tasks: maintaining contact and communication with offcampus students, and maintaining consistency within and among the core clinical clerkships.