Observational Teamwork Assessment for Surgery

It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period and OTAS is an accessible tool that can be used robustly by assessors from both clinical and nonclinical backgrounds.

Abstract

In Brief Objectives: To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool. Background: Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated. Methods: Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session. Results: All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert. Conclusions: It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds. This study assesses whether clinical and nonclinical novice observers can be trained to use the observational teamwork assessment for surgery (OTAS) after short-term, structured training. All novices became calibrated with an expert at rating OTAS behaviors to an acceptable degree of reliability by the end of training, and calibration improved over time for 4 of the 5 OTAS behaviors. OTAS is an accessible tool for training assessors with different backgrounds to assess nontechnical skills in the operating room.