Superior inter-rater reliability for checklist scores versus global rating scale, pass/fail measurements
FRIDAY, July 8, 2016 (HealthDay News) — Objective Structured Assessment of Technical Skills (OSATS) checklists are reliable for assessing technical skills among orthopedic residents performing open surgery on the shoulder, according to a study published in the Journal of Surgical Education.
Johnathan A. Bernard, M.D., M.P.H., from Johns Hopkins University in Baltimore, and colleagues compared the reliability and validity of three tools for measuring surgical skill among orthopedic residents. Twenty-three residents at different stages of training were assessed for technical skills pertaining to three shoulder open surgical approaches using the OSATS checklists, the Global Rating Scale (GRS), and a final pass/fail assessment determined by surgeons.
The researchers found that for the three open shoulder approaches, there was criterion validity between the OSATS checklists and GRS. Superior inter-rater reliability was seen for checklist scores versus GRS and subjective pass/fail measurements. Across training years, there was a positive correlation for GRS scores. Postgraduate year 1 and postgraduate year 2 residents had higher incidence of adverse events than more experienced residents.
“OSATS checklists are a valid and reliable assessment of technical skills across three surgical shoulder approaches. However, checklist scores do not measure quality of technique,” the authors write. “Multiple methods of assessing surgical skill should be considered when evaluating orthopedic resident surgical performance.”
Copyright © 2016 HealthDay. All rights reserved.
