[Skip to Content]
[Skip to Content Landing]
Views 1,848
Citations 0
Original Investigation
April 22, 2020

Development and Assessment of the Wisconsin Surgical Coaching Rubric

Author Affiliations
  • 1Wisconsin Surgical Outcomes Research Program, Department of Surgery, University of Wisconsin–Madison, Madison
  • 2Department of Curriculum and Instruction, University of Wisconsin School of Education, Madison
  • 3JCD Advisors LLC, Ann Arbor, Michigan
  • 4Center for Healthcare Outcomes & Policy, Department of Surgery, University of Michigan, Ann Arbor
  • 5Surgical Innovation Editor, JAMA Surgery
  • 6Department of Industrial and Systems Engineering, University of Wisconsin–Madison, Madison
JAMA Surg. Published online April 22, 2020. doi:10.1001/jamasurg.2020.0424
Key Points

Question  How is peer surgical coach performance evaluated?

Findings  In this study, the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool for assessing peer surgical coach performance, was developed and assessed. Evidence to support the validity of WiSCoR includes systematic content development, consistent rater training, and high interrater reliability.

Meaning  WiSCoR reliably assesses the performance of a surgical coach and can provide formative feedback to surgical coaches or assess the fidelity of coaching interventions to coaching principles.

Abstract

Importance  Surgical coaching continues to gain momentum as an innovative method for continuous professional development. A tool to measure the performance of a surgical coach is needed to provide formative feedback to coaches for continued skill development and to assess the fidelity of a coaching intervention for future research and dissemination.

Objective  To evaluate the validity of the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool to assess the performance of a peer surgical coach.

Design, Setting, and Participants  Surgical coaching sessions from November 2014 through February 2018 conducted by 2 statewide peer surgical coaching programs were audio recorded and transcribed. Twelve raters used WiSCoR to rate the performance of the surgical coach for each session. The study included peer surgical coaches in the Wisconsin Surgical Coaching Program (n = 8) and the Michigan Bariatric Surgery Collaborative coaching program (n = 15). The data were analyzed in 2019.

Interventions or Exposures  Use of WiSCoR to rate peer surgical coaching sessions.

Main Outcomes and Measures  There were 282 WiSCoR ratings from the 106 coaching sessions included in the study. WiSCoR was evaluated using a framework, including inter-rater reliability assessed with Gwet weighted agreement coefficent. Descriptive statistics of WiSCoR were calculated.

Results  Eight coaches (35%) and 11 coachees (29%) were from the Wisconsin Surgical Program and 15 coaches (65%) and 27 coachees (71%) were from the Michigan Bariatric Surgery Collaborative. The validity of WiSCoR is supported by high interrater reliability (Gwet weighted agreement coefficient, 0.87) as well as a weakly positive correlation of WiSCoR to coachee ratings of coaches (r = 0.22; P = .04), rigorous content development, consistent rater training, and the association of WiSCoR with coach and coaching program development. The mean (SD) overall coach performance rating using WiSCoR was 3.23 (0.82; range, 1-5).

Conclusions and Relevance  WiSCoR is a reliable measure that can assess the performance of a surgical coach, inform fidelity to coaching principles, and provide formative feedback to surgical coaches. While coachee ratings may reflect coachee satisfaction, they are not able to determine the quality of a coach.

Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    ×