Adaptation of “Objective Structured Assessment of Technical Skills” for Adenotonsillectomy into Turkish: A Validity and Reliability Study

Özet

Objective: To adapt an objective surgical assessmenttool (OSATS) for adenotonsillectomy/tonsillectomyto Turkish.Methods: A prospective longitudinal validation studywas planned. Adenoidectomy items were added to thetonsillectomy OSATS. Data were collected from 80real-time observed surgical performances. To examineinterrater reliability, 25 performances were simultaneously scored by two evaluators. In total, 105 test formswere filled, and reliability and validity studies wereconducted.Results: Regarding reliability, Cronbach's alphavalues were found to be >0.95, indicating very highinternal consistency. For interrater reliability, a statistically significant positive correlation was foundbetween the two evaluators’ task-specific checklist (TSC) (p=0.0001; r=0.969) and global ratingchecklist (GRC) (p=0.0001; r=0.957). Betweenthe TSC and GRC, there was a statistically significant positive correlation (p=0.0001; r=0.951).A model was developed using the generalizability theory, and the highest variances belonged tosurgical processes. Regarding validity, the trainers’scores were significantly higher than the trainees’scores (p=0.0001). As surgical experience increased,both the TSC and the GRC scores significantly increased (p=0.0001). A trainee’s first-ever 34adenotonsillectomy performances were scored. Ascatter plot of the obtained data showed that thetrainee had gained more scores as the number ofoperations increased.Conclusion: The results indicate that adenotonsillectomy OSATS is a reliable and valid tool for theassessment of residents’ surgical skills. This is the firstOSATS developed for adenotonsillectomy in theTurkish literature. It is ready to be used in the operating room to provide constructive feedback andmonitor the development of trainees’ surgical skills inadenotonsillectomy.