Best practices for assessing competence and performance of the behavioral Health workforce
Administration and Policy in Mental Health and Mental Health Services ResearchPublished 1 May 2005
Philip G. Bashook
Citations48
SJR quartileQ1
SJR score1.26
SNIP1.40
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Strategies used in general medicine and other disciplines for assessing trainees and practitioners for competency assessment in behavioral health are reviewed.
Abstract
The need for mechanisms to assess the competence and performance of the behavioral health workforce has received increasing attention. This article reviews strategies used in general medicine and other disciplines for assessing trainees and practitioners. The possibilities and limitations of various approaches are reviewed, and the implications for behavioral health are addressed. A conceptual model of competence is presented, and practical applications of this model are reviewed. Finally, guidelines are proposed for building competency assessment protocols for behavioral health.
Keywords
MedicineHealth Professions
National Academies Press eBooksTo Err Is Human
14,096 Citations2000Institute of Medicine
A process for the preparation of prostaglandins which are known pharmaceutically active compounds and then R alpha , R omega -cyclopentenoids is described.
BMJCrossing the Quality Chasm: A New Health System for the 21st Century
10,744 Citations2001Alexi Baker
Analyzing health care organizations as complex systems, Crossing the Quality Chasm also documents the causes of the quality gap, identifies current practices that impede quality care, and explores how systems approaches can be used to implement change.
Academic MedicineThe assessment of clinical skills/competence/performance
4,764 Citations1990George E. Miller
Revista Española de PedagogíaStandards for Educational and Psychological Testing.
2,225 Citations2017Eva Jiménez-García
Rasch Models for Measurement
1,324 Citations1988David Andrich
Academic MedicineComparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination
721 Citations1998Glenn Regehr, Helen MacRae +2 more
Global rating scales scored by experts showed higher inter-station reliability, better construct validity, and better concurrent validity than did checklists, suggesting that global rating scales administered by experts are a more appropriate summative measure when assessing candidates on performance-based examinations.
AnesthesiologyAssessment of Clinical Performance during Simulated Crises Using Both Technical and Behavioral Ratings
525 Citations1998David M. Gaba, Steven K. Howard +4 more
The rating system needs more refinement before it can be used to assess clinical competence for residency graduation or board certification, and behavioral performance can be assessed from videotapes of simulations.
Computers & GraphicsMedicine Meets Virtual Reality 14
462 Citations2005
In conclusion, virtual reality surgical skills training may produce a significant learning effect that can transfer to actual robot-assisted laparoscopic procedures.
Medical EducationThe relationship between competence and performance: implications for assessing practice performance
410 Citations2002J-J Rethans, John J. Norcini +7 more
Current views of the relationship between competence and performance are described and some of the implications of the distinctions between the two areas are delineated for the purpose of assessing doctors in practice.
JAMAUse of Peer Ratings to Evaluate Physician Performance
351 Citations1993Paul G. Ramsey
Using a shorter version of the questionnaire used in this study, peer ratings provide a practical method to assess clinical performance in areas such as humanistic qualities and communication skills that are difficult to assess with other measures.
Educational ResearcherPerformance-Based Assessment: Lessons From the Health Professions
281 Citations1995David B. Swanson, Geoffrey R. Norman +1 more
Four performance-based assessment methods are used as examples: written clinical simulations (more commonly termed patient management problems), computer-based clinical simulations, oral examinations, and standardized patients (“live” simulations).
Studies in health technology and informaticsValidation Of Virtual Reality To Teach And Assess Psychomotor Skills In Laparoscopic Surgery: Results From Randomised Controlled Studies Using The MIST VR Laparoscopic Simulator
264 Citations1998N Taffinder, Chris Sutton +3 more
The MIST VR simulator can objectively assess a number of desirable qualities in laparoscopic surgery, and can distinguish between experienced and novice surgeons, and quantified the beneficial effect of a structured training course on psychomotor skill acquisition.
The American Journal of SurgeryReliability and construct validity of a Structured Technical Skills Assessment Form
257 Citations1994Christopher P. Winckel, Richard K. Reznick +2 more
The incorporation of structured guidelines to the assessment of technical skill leads to high inter-rater reliability and construct validity, which ultimately may result in improved and reproducible evaluations of surgical trainees.
JAMAPhysician Clinical Performance Assessment
236 Citations2003Bruce E. Landon, Sharon‐Lise T. Normand +2 more
It is concluded that important technical barriers stand in the way of using physician clinical performance assessment for evaluating the competency of individual physicians and that overcoming these barriers will require considerable additional research and development.
Academic MedicineDeveloping key-feature problems and examinations to assess clinical decision-making skills
214 Citations1995Gordon Page, Georges Bordage +1 more
This problem format, with its focus on only the critical steps in problem resolution, and with its flexibility in question format and scoring keys, effectively addresses the psychometric considerations of content validity and test score reliability, and accommodates the complexity and configurations of actions often required in the resolution of clinical problems.
PubMedAssessment of physician performance in Alberta: the physician achievement review.
194 Citations1999William C. Hall, Claudio Violato +7 more
The Physician Achievement Review provides a multidimensional view of performance through structured feedback to physicians, and two-thirds of the physicians indicated that they were considering or had implemented changes to their medical practice on the basis of their PAR data.
Journal of Educational MeasurementAn Essay on the History and Future of Reliability from the Perspective of Replications
171 Citations2001Robert L. Brennan
Educational ResearcherPerformance Assessment: Policy Promises and Technical Measurement Standards
156 Citations1994Robert L. Linn
Major changes in the nature and context of assessment are underway that have potential implications for technical measurement standards, and these changes provide the context for the recently launched effort to revise the Standards for Educational and Psychological Testing.
Academic MedicineExploring the Gap Between Knowledge and Behavior: A Qualitative Study of Clinician Action Following an Educational Intervention
137 Citations2004Tara J T Kennedy, Glenn Regehr +3 more
The knowledge–behavior gap was exposed and factors reported to influence clinicians’ decisions about whether to implement new knowledge were elicited, and two conditions that promote clinical action based on knowledge (level of certainty and sense of urgency) were identified.
Medical EducationInternational standards in medical education: assessment and accreditation of medical schools' – educational programmes. A WFME position paper
127 Citations1998The Executive Council
WFME has recently decided to extend its `International Collaborative Programme for the Reorientation of Medical Education', aiming at the implementation of its educational policy at the institutional level, to establish a system to assure minimum quality standards for medical school programmes.
JAMACompetency Assessment of Primary Care Physicians as Part of a Peer Review Program
124 Citations1993Geoffrey R. Norman
The study demonstrated the feasibility, reliability, and validity of a multicomponent examination in the peer review process and to design and test a program that assesses clinical competence as a second stage in apeer review process.
Administration and Policy in Mental Health and Mental Health Services ResearchThe fundamentals of workforce competency: implications for behavioral health
123 Citations2005Michael A. Hoge, Janis Tondora +1 more
The forces that drive the current focus on competency of the healthcare workforce are outlined and recommendations are offered to guide future work on competencies in behavioral health.
BMJThe Certification and Recertification of Doctors: Issues in the Assessment of Clinical Competence
113 Citations1994David Newble, Brian Jolly +1 more
Assessment of both postgraduate and undergraduate medical students should be valid, reliable, and fair, but unfortunately it is rarely any of these and never all.
Teaching and Learning in MedicineGuidelines for assessing clinical competence
100 Citations1994David Newble, Beth Dawson +7 more
A set of guidelines that examining bodies should follow in developing and implementing assessment procedures that are a valid reflection of examinees’ current level of competence and of their ability to perform satisfactorily at the next stage of training or practice are presented.
Medical EducationWhen enough is enough: a conceptual basis for fair and defensible practice performance assessment
96 Citations2002Lambert Schuwirth, Lesley Southgate +6 more
Because of the severe consequences an assessment of practice performance may have, it is essential that the procedure is both defensible to the stakeholders and fair in that it distinguishes well between good performers and underperformers.
BMJContinuing medical education: Recertification and the maintenance of competence
78 Citations1998P G Bashook, John Parboosingh
Recertification should assess real performance in practice and competence to continue to learn and should evaluate documented participation in continuing education as evidence of continuing competence as a specialist.
Journal of General Internal MedicineA questionnaire for patients’ evaluations of their physicians’ humanistic behaviors
76 Citations1993Michael J. Weaver, Cathy L. Ow +2 more
There is good correlation between the PHBQ and patient satisfaction, which supports the validity of thePHBQ, and the relative lack of agreement between patients and attending physicians suggests different observations, criteria, or standards.
American Journal of RoentgenologyMaintenance of Certification
75 Citations2005John E. Madewell, Robert R. Hattery +5 more
The Trustees of the American Board of Urology (ABU) took note of this condemnation of the process of Maintenance of Certification (MOC), and sought to address the misconceptions and presumptions shared by Diplomates from a variety of Boards about MOC.
Evaluation & the Health ProfessionsValidation of Professional Licensure Examinations
71 Citations1994Anthony LaDuca
Methods for modifying approaches to validation of professional licensing tests requires modifying a theory of professions incorporated into the test development process so as to acknowledge the social character of professions.
Teaching and Learning in MedicineConceptual and methodological issues in studies comparing assessment formats
55 Citations1996Geoffrey R. Norman, David B. Swanson +1 more
Critically examine methodological issues that arise in studies comparing alternate written assessment methods, and identify conditions that must be met before it can be concluded that different formats assess different aptitudes or traits.
Academic PsychiatryPortfolios as a Novel Approach for Residency Evaluation
47 Citations2002Patricia O’Sullivan
The authors describe the development process for a portfolio that programs may use for evaluating psychiatry residents that requires orientation of both residents and faculty, attention to time demands, and availability of examples and resources.
Medical EducationThe General Medical Council's Performance Procedures: peer review of performance in the workplace
46 Citations2001Lesley Southgate, Jim Cox +9 more
The General Medical Council procedures to assess the performance of doctors who may be seriously deficient include peer review of the doctor’s practice at the workplace and tests of competence and skills.
Academic MedicineAn assessment of an oral examination format for evaluating clinical competence in emergency medicine
46 Citations1990David Solomon, Mary Ann Reinhart +3 more
Medical EducationProcedures for establishing defensible programmes for assessing practice performance
25 Citations2002Stephen R Lew, Gordon G. Page +6 more
Issues to consider in the articulation of the purposes and outcomes of the assessment, planning the programme, the administrative processes involved, including communication and preparation of assessees are described.
Educational and Psychological MeasurementInterjudge Reliability and Decision Reproducibility
25 Citations1994Mary E. Lunz, John A. Stahl +1 more
American Journal of PsychiatryReliability of the Part II Board Certification Examination in Psychiatry: interexaminer consistency
21 Citations1991John Francis McDermott, Peter E. Tanguay +6 more
The Part II ABPN examination demonstrates fair to good reliability as measured by interexaminer consistency, and development of more explicit grading criteria should further improve examiner agreement in future examinations.
Academic MedicineNew Medical Licensing Examination Using Computer-based Case Simulations and Standardized Patients
19 Citations2002Maria Teresa Guagnano, D. Merlitti +3 more
The Multimedia Integrated Pilot Project (MIPP), a single two-step examination that combines computer-based case simulations and clinical encounters using standardized patients, is a good tool for assessing clinical competence.
Teaching and Learning in MedicineEffects of examiner training on open‐ended, higher taxonomic level questioning in oral certification examinations
11 Citations1993J E Des Marchais, P Jean
In order to improve the taxonomic level of questions, groups of examiners were given a 3‐hr training session before the examination, which resulted in a twofold increase in the number of open‐ended questions of problem‐solving level.
Springer international handbooks of educationRelicensure, Recertification and Practice-Based Assessment
6 Citations2002John Cunnington, Lesley Southgate
The maintenance of competence of physicians in practice has emerged as a central issue in health care and there have been increasing calls for the recertification or the relicensure of physicians.