How is peer surgical coach performance evaluated?
In this study, the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool for assessing peer surgical coach performance, was developed and assessed. Evidence to support the validity of WiSCoR includes systematic content development, consistent rater training, and high interrater reliability.
WiSCoR reliably assesses the performance of a surgical coach and can provide formative feedback to surgical coaches or assess the fidelity of coaching interventions to coaching principles.
Surgical coaching continues to gain momentum as an innovative method for continuous professional development. A tool to measure the performance of a surgical coach is needed to provide formative feedback to coaches for continued skill development and to assess the fidelity of a coaching intervention for future research and dissemination.
To evaluate the validity of the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool to assess the performance of a peer surgical coach.
Design, Setting, and Participants
Surgical coaching sessions from November 2014 through February 2018 conducted by 2 statewide peer surgical coaching programs were audio recorded and transcribed. Twelve raters used WiSCoR to rate the performance of the surgical coach for each session. The study included peer surgical coaches in the Wisconsin Surgical Coaching Program (n = 8) and the Michigan Bariatric Surgery Collaborative coaching program (n = 15). The data were analyzed in 2019.
Interventions or Exposures
Use of WiSCoR to rate peer surgical coaching sessions.
Main Outcomes and Measures
There were 282 WiSCoR ratings from the 106 coaching sessions included in the study. WiSCoR was evaluated using a framework, including inter-rater reliability assessed with Gwet weighted agreement coefficent. Descriptive statistics of WiSCoR were calculated.
Eight coaches (35%) and 11 coachees (29%) were from the Wisconsin Surgical Program and 15 coaches (65%) and 27 coachees (71%) were from the Michigan Bariatric Surgery Collaborative. The validity of WiSCoR is supported by high interrater reliability (Gwet weighted agreement coefficient, 0.87) as well as a weakly positive correlation of WiSCoR to coachee ratings of coaches (r = 0.22; P = .04), rigorous content development, consistent rater training, and the association of WiSCoR with coach and coaching program development. The mean (SD) overall coach performance rating using WiSCoR was 3.23 (0.82; range, 1-5).
Conclusions and Relevance
WiSCoR is a reliable measure that can assess the performance of a surgical coach, inform fidelity to coaching principles, and provide formative feedback to surgical coaches. While coachee ratings may reflect coachee satisfaction, they are not able to determine the quality of a coach.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Vande Walle KA, Quamme SRP, Beasley HL, et al. Development and Assessment of the Wisconsin Surgical Coaching Rubric. JAMA Surg. Published online April 22, 2020. doi:10.1001/jamasurg.2020.0424
Customize your JAMA Network experience by selecting one or more topics from the list below.
Create a personal account or sign in to: