[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Table 1.  
Cases
Cases
Table 2.  
Coding Schema
Coding Schema
Table 3.  
Teaching Points per Hour in the Operating Room (OR) vs the Video-Based Coaching Sessions
Teaching Points per Hour in the Operating Room (OR) vs the Video-Based Coaching Sessions
1.
Ericsson  KA.  Deliberate practice and acquisition of expert performance: a general overview.  Acad Emerg Med. 2008;15(11):988-994.PubMedGoogle ScholarCrossref
2.
Bohnen  JD, Chang  DC, George  BC.  Changing trends in operating room times between teaching and non-teaching cases: less time for learning?  J Am Coll Surg. 2015;221(4):S49.Google ScholarCrossref
3.
Teman  NR, Gauger  PG, Mullan  PB, Tarpley  JL, Minter  RM.  Entrustment of general surgery residents in the operating room: factors contributing to provision of resident autonomy.  J Am Coll Surg. 2014;219(4):778-787.PubMedGoogle ScholarCrossref
4.
Fairfax  LM, Christmas  AB, Green  JM, Miles  WS, Sing  RF.  Operative experience in the era of duty hour restrictions: is broad-based general surgery training coming to an end?  Am Surg. 2010;76(6):578-582.PubMedGoogle Scholar
5.
Kairys  JC, McGuire  K, Crawford  AG, Yeo  CJ.  Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees?  J Am Coll Surg. 2008;206(5):804-813. PubMedGoogle ScholarCrossref
6.
Schwartz  SI, Galante  J, Kaji  A,  et al.  Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.  JAMA Surg. 2013;148(9):829-833.PubMedGoogle ScholarCrossref
7.
Snyder  RA, Phillips  SE, Terhune  KP.  Impact of implementation of a pediatric surgery fellowship on general surgery resident operative volume.  J Surg Educ. 2012;69(6):753-758.PubMedGoogle ScholarCrossref
8.
Hanks  JB, Ashley  SW, Mahvi  DM,  et al.  Feast or famine? the variable impact of coexisting fellowships on general surgery resident operative volumes.  Ann Surg. 2011;254(3):476-485. PubMedGoogle ScholarCrossref
9.
Bell  RH  Jr.  Why Johnny cannot operate.  Surgery. 2009;146(4):533-542.PubMedGoogle ScholarCrossref
10.
Podnos  YD, Wilson  SE.  Threats to the surgical residency in the academic medical center.  Arch Surg. 2001;136(2):161-164.PubMedGoogle ScholarCrossref
11.
Sandhu  G, Teman  NR, Minter  RM.  Training autonomous surgeons: more time or faculty development?  Ann Surg. 2015;261(5):843-845.PubMedGoogle ScholarCrossref
12.
Nauta  RJ.  The surgical residency and Accreditation Council for Graduate Medical Education reform: steep learning or sleep learning?  Am J Surg. 2011;201(6):715-718.PubMedGoogle ScholarCrossref
13.
Mattar  SG, Alseidi  AA, Jones  DB,  et al.  General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors.  Ann Surg. 2013;258(3):440-449.PubMedGoogle ScholarCrossref
14.
Davis  D, O’Brien  MA, Freemantle  N, Wolf  FM, Mazmanian  P, Taylor-Vaisey  A.  Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?  JAMA. 1999;282(9):867-874.PubMedGoogle ScholarCrossref
15.
Hu  YY, Peyre  SE, Arriaga  AF,  et al.  Postgame analysis: using video-based coaching for continuous professional development.  J Am Coll Surg. 2012;214(1):115-124.PubMedGoogle ScholarCrossref
16.
Glarner  CE, Hu  YY, Chen  CH,  et al.  Quantifying technical skills during open operations using video-based motion analysis.  Surgery. 2014;156(3):729-734.PubMedGoogle ScholarCrossref
17.
KU Center for Research on Learning.  What Coaches Do. 3rd iteration. Lawrence: University of Kansas; June 2010.
18.
Newman  LR, Roberts  DH, Schwartzstein  RM. Peer Observation of Teaching Handbook. www.mededportal.org/publication/9150. Published by MedEdPORTAL in 2012. Accessed November 4, 2016.
19.
Yule  S.  Checklist for Trainers Facilitating a NOTSS Debrief. Aberdeen, Scotland: University of Aberdeen; 2010.
20.
Guerlain  S, Adams  RB, Turrentine  FB,  et al.  Assessing team performance in the operating room: development and use of a “black-box” recorder and other tools for the intraoperative environment.  J Am Coll Surg. 2005;200(1):29-37.PubMedGoogle ScholarCrossref
21.
Hauge  LS, Wanzek  JA, Godellas  C.  The reliability of an instrument for identifying and quantifying surgeons’ teaching in the operating room.  Am J Surg. 2001;181(4):333-337.PubMedGoogle ScholarCrossref
22.
Cope  AC, Mavroveli  S, Bezemer  J, Hanna  GB, Kneebone  R.  Making meaning from sensory cues: a qualitative investigation of postgraduate learning in the operating room.  Acad Med. 2015;90(8):1125-1131.PubMedGoogle ScholarCrossref
23.
Roberts  NK, Brenner  MJ, Williams  RG, Kim  MJ, Dunnington  GL.  Capturing the teachable moment: a grounded theory study of verbal teaching interactions in the operating room.  Surgery. 2012;151(5):643-650.PubMedGoogle ScholarCrossref
24.
Hu  YY, Arriaga  AF, Roth  EM,  et al.  Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care.  Ann Surg. 2012;256(2):203-210.PubMedGoogle ScholarCrossref
25.
International Coach Federation. What is professional coaching? http://coachfederation.org/need/landing.cfm?ItemNumber=978&navItemNumber=567. Accessed March 8, 2016.
26.
Singh  P, Aggarwal  R, Tahir  M, Pucher  PH, Darzi  A.  A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills.  Ann Surg. 2015;261(5):862-869.PubMedGoogle ScholarCrossref
27.
Karam  MD, Thomas  GW, Koehler  DM,  et al.  Surgical coaching from head-mounted video in the training of fluoroscopically guided articular fracture surgery.  J Bone Joint Surg Am. 2015;97(12):1031-1039.PubMedGoogle ScholarCrossref
28.
Yule  S, Parker  SH, Wilkinson  J,  et al.  Coaching non-technical skills improves surgical residents’ performance in a simulated operating room.  J Surg Educ. 2015;72(6):1124-1130.PubMedGoogle ScholarCrossref
29.
Bonrath  EM, Dedy  NJ, Gordon  LE, Grantcharov  TP.  Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial.  Ann Surg. 2015;262(2):205-212.PubMedGoogle ScholarCrossref
30.
Grant  J.  Learning needs assessment: assessing the need.  BMJ. 2002;324(7330):156-159.PubMedGoogle ScholarCrossref
31.
Kaufman  DM.  Applying educational theory in practice.  BMJ. 2003;326(7382):213-216.PubMedGoogle ScholarCrossref
32.
Cox  SS, Swanson  MS.  Identification of teaching excellence in operating room and clinic settings.  Am J Surg. 2002;183(3):251-255.PubMedGoogle ScholarCrossref
33.
Snyder  RA, Tarpley  MJ, Tarpley  JL, Davidson  M, Brophy  C, Dattilo  JB.  Teaching in the operating room: results of a national survey.  J Surg Educ. 2012;69(5):643-649.PubMedGoogle ScholarCrossref
34.
Pugh  CM, DaRosa  DA, Glenn  D, Bell  RH  Jr.  A comparison of faculty and resident perception of resident learning needs in the operating room.  J Surg Educ. 2007;64(5):250-255.PubMedGoogle ScholarCrossref
35.
Mutabdzic  D, Mylopoulos  M, Murnaghan  ML,  et al.  Coaching surgeons: is culture limiting our ability to improve?  Ann Surg. 2015;262(2):213-216.PubMedGoogle ScholarCrossref
36.
Greenberg  CC, Klingensmith  ME.  The continuum of coaching: opportunities for surgical improvement at all levels.  Ann Surg. 2015;262(2):217-219.PubMedGoogle ScholarCrossref
37.
Butvidas  LD, Anderson  CI, Balogh  D, Basson  MD.  Disparities between resident and attending surgeon perceptions of intraoperative teaching.  Am J Surg. 2011;201(3):385-389.PubMedGoogle ScholarCrossref
38.
Chen  XP, Williams  RG, Sanfey  HA, Smink  DS.  A taxonomy of surgeons’ guiding behaviors in the operating room.  Am J Surg. 2015;209(1):15-20.PubMedGoogle ScholarCrossref
39.
Pernar  LI, Peyre  SE, Hasson  RM,  et al.  Exploring the content of intraoperative teaching.  J Surg Educ. 2016;73(1):79-84.PubMedGoogle ScholarCrossref
40.
Lio  CH, Carswell  CM, Strup  SE, Roth  JS, Grant  R. The operating room as classroom: understanding cognitive challenges facing surgical trainees. In: Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting. Thousand Oaks, CA: SAGE Publications; 2010:1571-1575.
41.
Claridge  JA, Calland  JF, Chandrasekhara  V, Young  JS, Sanfey  H, Schirmer  BD.  Comparing resident measurements to attending surgeon self-perceptions of surgical educators.  Am J Surg. 2003;185(4):323-327.PubMedGoogle ScholarCrossref
42.
Vollmer  CM  Jr, Newman  LR, Huang  G, Irish  J, Hurst  J, Horvath  K.  Perspectives on intraoperative teaching: divergence and convergence between learner and teacher.  J Surg Educ. 2011;68(6):485-494.PubMedGoogle ScholarCrossref
43.
Rose  JS, Waibel  BH, Schenarts  PJ.  Disparity between resident and faculty surgeons’ perceptions of preoperative preparation, intraoperative teaching, and postoperative feedback.  J Surg Educ. 2011;68(6):459-464.PubMedGoogle ScholarCrossref
44.
Pernar  LI, Breen  E, Ashley  SW, Peyre  SE.  Preoperative learning goals set by surgical residents and faculty.  J Surg Res. 2011;170(1):1-5.PubMedGoogle ScholarCrossref
Original Investigation
April 2017

Complementing Operating Room Teaching With Video-Based Coaching

Author Affiliations
  • 1Center for Surgery and Public Health, Brigham and Women’s Hospital, Boston, Massachusetts
  • 2Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
  • 3currently with the Department of Surgery, Connecticut Children’s Medical Center, Hartford
  • 4Goodman Surgical Education Center, Department of Surgery, Stanford University, Palo Alto, California
  • 5STRATUS (Simulation, Training, Research and Technology Utilization System) Center for Medical Simulation, Brigham and Women’s Hospital, Boston, Massachusetts
  • 6Department of Surgery, Brigham and Women’s Hospital, Boston, Massachusetts
  • 7Department of Anesthesiology, Brigham and Women’s Hospital, Boston, Massachusetts
  • 8currently with the Department of Anesthesiology and Critical Care, University of Pennsylvania Health System, Philadelphia
  • 9Wisconsin Surgical Outcomes Research Program, Department of Surgery, University of Wisconsin Hospitals and Clinics, Madison
  • 10Department of Health Policy and Management, Harvard T. H. Chan School of Public Health, Boston, Massachusetts
 

Copyright 2016 American Medical Association. All Rights Reserved.

JAMA Surg. 2017;152(4):318-325. doi:10.1001/jamasurg.2016.4619
Key Points

Question  Is post hoc video-based coaching an effective modality for teaching residents to operate?

Findings  This mixed methods analysis compared teaching in a video-based coaching session with teaching during the corresponding operating room case. During video-based coaching, instruction was more individualized to the resident, and higher-level concepts, such as decision making, were discussed.

Meaning  In this era of reduced resident operative time and autonomy, intraoperative education may be supplemented by video-based coaching.

Abstract

Importance  Surgical expertise demands technical and nontechnical skills. Traditionally, surgical trainees acquired these skills in the operating room; however, operative time for residents has decreased with duty hour restrictions. As in other professions, video analysis may help maximize the learning experience.

Objective  To develop and evaluate a postoperative video-based coaching intervention for residents.

Design, Setting, and Participants  In this mixed methods analysis, 10 senior (postgraduate year 4 and 5) residents were videorecorded operating with an attending surgeon at an academic tertiary care hospital. Each video formed the basis of a 1-hour one-on-one coaching session conducted by the operative attending; although a coaching framework was provided, participants determined the specific content collaboratively. Teaching points were identified in the operating room and the video-based coaching sessions; iterative inductive coding, followed by thematic analysis, was performed.

Main Outcomes and Measures  Teaching points made in the operating room were compared with those in the video-based coaching sessions with respect to initiator, content, and teaching technique, adjusting for time.

Results  Among 10 cases, surgeons made more teaching points per unit time (63.0 vs 102.7 per hour) while coaching. Teaching in the video-based coaching sessions was more resident centered; attendings were more inquisitive about residents’ learning needs (3.30 vs 0.28, P = .04), and residents took more initiative to direct their education (27% [198 of 729 teaching points] vs 17% [331 of 1977 teaching points], P < .001). Surgeons also more frequently validated residents’ experiences (8.40 vs 1.81, P < .01), and they tended to ask more questions to promote critical thinking (9.30 vs 3.32, P = .07) and set more learning goals (2.90 vs 0.28, P = .11). More complex topics, including intraoperative decision making (mean, 9.70 vs 2.77 instances per hour, P = .03) and failure to progress (mean, 1.20 vs 0.13 instances per hour, P = .04) were addressed, and they were more thoroughly developed and explored. Excerpts of dialogue are presented to illustrate these findings.

Conclusions and Relevance  Video-based coaching is a novel and feasible modality for supplementing intraoperative learning. Objective evaluation demonstrates that video-based coaching may be particularly useful for teaching higher-level concepts, such as decision making, and for individualizing instruction and feedback to each resident.

Introduction

Acquiring expertise in a complex skill, such as operating, requires “deliberate practice” (ie, the facilitation of specific task improvement through immediate feedback, problem solving, and performance refinement).1 Traditionally, surgical trainees acquired deliberate practice in the operating room (OR), with graduated independence throughout training. However, as resident operative experience becomes increasingly limited2-4 by duty hour restrictions,5,6 the implementation of fellowships,7,8 and productivity, quality, and safety pressures on attendings,3,9-11 concerns about the competence of new graduates have arisen.9-13

While surgical training programs supplement their curricula with lectures, web-based learning, and simulation, nothing yet approximates the rich experience of operating. Many efforts neglect adult learning theory principles; they are teacher driven rather than learner driven, didactic rather than interactive, and amassed rather than distributed over time.14 Few evidence-based methods exist to help trainees evaluate or improve their operative performance.

In other professions requiring accelerated skill acquisition, coaches provide learners with perspective and immediate, targeted feedback.1 Athletes routinely review performance videos with coaches, who guide them through an individualized self-improvement process. With such directed feedback, deliberate practice is achieved, accelerating the learning curve. Our group previously piloted a video-based coaching intervention for practicing surgeons, demonstrating feasibility, acceptability, and a high perception of value.15 Qualitative analysis of this continuing professional development module revealed a natural predominance of critical thinking topics, an aspect we suspect is underemphasized in intraoperative teaching. We sought to expand the scope of investigation to residents, hypothesizing that video-based coaching would fill these gaps in their operative education.

Methods
Participant Recruitment and Data Collection

We recruited postgraduate year 4 and 5 general surgery residents (to maximize the chance of capturing complex discussions) and attending surgeons at Brigham and Women’s Hospital, an academic tertiary care center. A convenience sample of 10 residents was offered a choice of coaches from colorectal, minimally invasive, oncologic, acute care, and transplant surgery.

We videorecorded and audiorecorded 1 operation for each resident-coach dyad (Table 1). Videos captured the surgical field and entire OR, as previously described.16 Each video formed the basis of a 1-hour one-on-one coaching session conducted by the operative attending, scheduled 0 to 43 days after surgery (mean of 13.6 and median of 5.5 days elapsed). To prepare, coaches were given a “Coaching Residents” pamphlet, adapted from handbooks on coaching or giving feedback in other settings.17-19 The content of the video-based coaching sessions was determined entirely by the participants; we neither mandated nor suggested discussion topics. The video-based coaching sessions were audiorecorded. Written informed consent was obtained from surgeons and patients. The study was approved by the Partners Institutional Review Board.

Coding

Operating room videorecordings20 and coaching audiorecordings were transcribed with patient and surgeon identifiers removed. A preliminary coding schema was adapted from prior surgical education work15,21-23 and then refined through an iterative inductive coding process. Two surgical research fellows (Y.-Y.H. and L.M.M.) independently reviewed each transcript to identify teaching points, coding them by initiator, content, teaching technique, and tone. When the initiator was the resident, the attending response was coded as open, closed, or reflective. Two deidentified transcripts of OR videos from a prior study24 served as training cases for intercoder calibration. Interrater discrepancies were resolved by discussion until consensus was achieved. Transcripts were recoded iteratively until thematic saturation was reached. Table 2 lists the final coding schema, including definitions and examples. Modeling was not included as a teaching technique; teaching had to be explicitly verbalized and intentionally directed toward the resident to be counted.

Statistical Analysis

χ2 Tests were used to compare the OR and the video-based coaching sessions by the percentage of teaching points initiated by the attending vs the resident and the response of the attending to resident-initiated points. Paired t tests were used to compare the mean counts of teaching points per hour, based on content and technique, between the OR and the video-based coaching sessions. For the coaching sessions lasting less than 1 hour, the time was rounded up to 1 hour rather than extrapolated. We performed a sensitivity analysis in which we limited OR time to the attending presence; results were similar, so only the original analysis is reported herein. Significance was set at 2-sided P < .05. All analyses were performed using statistical software (SAS, version 9.4; SAS Institute Inc).

Results

A comparison of teaching points was made between the OR and the video-based coaching sessions with respect to initiator, content, teaching technique, and tone, adjusted for time. While coaching, surgeons made more teaching points per unit time. Quiz Ref IDAmong 10 OR cases, we observed 1977 teaching points during 31.4 hours of video (63.0 per hour); in the coaching sessions, we counted 719 teaching points during 7.0 hours of video (102.7 per hour).

Focus on Resident Education

While coaching, surgeons were more focused on resident education; they performed educational needs assessments almost 10 times more often (3.30 vs 0.28, P = .04). Quiz Ref IDResidents were better able to direct teaching to accommodate their own learning needs: 17% (331 of 1977) of teaching points were initiated by residents in the OR compared with 27% (198 of 729) in the videobased coaching sessions (P < .001).

While coaching, surgeons asked almost 3 times more questions to prompt reflection or critical thinking (9.30 vs 3.32, P = .07) and set 10 times more goals to facilitate future practice (2.90 vs 0.28, P = .11). They also validated residents’ experiences (eg, regarding the challenging nature of a case) more frequently (8.40 vs 1.81, P < .01), thus softening critical feedback and increasing resident engagement (Table 2).

Teaching Content

Anatomy (8.48 vs 3.10, P = .01) and progress through the steps of the procedure (20.30 vs 7.50, P < .01) were more commonly discussed in the OR. Quiz Ref IDIn the video-based coaching sessions, higher-level concepts—specifically, intraoperative decision making (mean, 9.70 vs 2.77 instances per hour, P = .03) and failure to progress (mean, 1.20 vs 0.13 instances per hour, P = .04)—were more frequently discussed (Table 2).

Increased Depth of Teaching

Teaching was less directive and more explanatory during video-based coaching; attendings preferentially used instructive techniques in the OR (24.48 vs 0.10 teaching points, P < .01) and informative techniques while coaching (44.80 vs 31.05 teaching points, P = .20) (Table 3). Quiz Ref IDAttending responses to resident-initiated teaching points tended to be more thoughtful and more thorough during the coaching sessions. In the OR, most attending responses were open (95.7% [310 of 324]), few were reflective (2.5% [8 of 324]), and fewer were closed (1.9% [6 of 324]). In the coaching sessions, while open responses predominated (82.8% [149 of 180]), reflective responses were common (16.7% [30 of 180]), and closed responses were nonexistent (0% [0 of 180]) (P < .01) (Table 2).

Even when the same teaching point was made in both venues, it was generally more thoroughly articulated in the video-based coaching sessions. During a biopsy of a retroperitoneal mass, the resident asked about the approach and received a response that was informative but minimally so: Resident: “So you think we should go through [the] lesser sac here?” Attending: “Yeah, I think you need to to get to that….” In the corresponding coaching session, the coach used several techniques—questioning, informing, and validating—to deepen the resident’s understanding about intraoperative decision making surrounding the exposure:

Coach: Where are your windows in the upper abdomen to…get back where that thing was? Do you remember what you were thinking about the different options for getting back there…?

Resident: …We talked about going through the gastrohepatic ligament, which is more or less what we did in the end, right? Or we could have gone posterior to the stomach [be]cause we were going into the lesser sac; we could’ve kocherized the duodenum….

Coach: I think in this case it was easier to go through the gastrohepatic ligament because it was a direct route down to the thing. It was more to the right of the aorta than it was to the left…. Remember, we had this big discussion: where’s the cava…? …Two reasons why there’s complex [anatomy]. (1) …You just don’t have good exposure because you’re trying to do it through a tiny hole…. (2) You [have] just got tumor socked in there, and you just can’t tell. So…you think about what things are going to be in that neighborhood and what you need to know. And you get a good sense of strategy and reassurance when you’ve clearly identified all the key structures. So we knew, “…Where is the cava?” because there were 2 options…. If the thing had pushed the cava posteriorly…we could have been coming down on the cava, which was not what we wanted to do. We thought the cava was anterior, right? …Triangulate the scans. We thought we were pulling the cava up….

Reconciliation of Missed Learning Opportunities
Correction of Misconceptions

Often during the video-based coaching sessions, previously unrecognized gaps in residents’ comprehension of a case came to light. One resident admitted confusion with the attending’s instructions during the case: Resident: “I remember at one point you said to…kocherize the duodenum by feel, and I was like, ‘No idea how to do that.’” Throughout the session, the coach set goals for this technique: choosing a relevant case for further practice, enumerating the steps, and stopping to further clarify points as requested by the resident (eAppendix in the Supplement, example 1).

Similarly, during a complex retroperitoneal sarcoma resection, the attending directed the resident through steps of the dissection without extensive explanation of the attendant decision making. The resident did not interject but asked for clarification during the coaching session: Resident: “I’m just looking at…taking down the transverse colon. Did we do that just for…retraction at the point?” The coach then used a mixture of questioning and informing to explain the relevant anatomy and intraoperative decision making (eAppendix in the Supplement, example 2).

Without coaching, these residents would have missed nuances of the critical decision making required for the cases. Learning opportunities would have been lost.

Retrospection

Post hoc coaching allows for retrospective insight. With knowledge of pathology report results, an attending was able to reflect that the open approach had given them tactile feedback that changed the operation and hence the patient’s prognosis (eAppendix in the Supplement, example 3). Similarly, in reviewing a jejunostomy tube placement (a procedurally straightforward case), the resident and coach reflected on the postoperative death, initiating a novel discussion about decision making in the setting of terminal disease (eAppendix in the Supplement, example 4), a topic often neglected in surgery.

Hypothetical Scenarios

Coaching also allowed for the discussion of theoretical complications. In the case of a splenectomy, which “turned out to be the perfect, ideal situation,” the attending used the coaching session to review alternate scenarios “if it [had not been] so safe and clean.” The attending questioned the resident about anatomic and pathophysiological issues, technique, and potential intraoperative pitfalls (eAppendix in the Supplement, examples 5, 6, and 7). Such one-on-one personalized discussion of intraoperative management remains rare in today’s training paradigm despite being critical to everyday practice.

Technical Learning

Increased depth was seen during coaching not just when teaching higher-level concepts, such as decision making. It was also seen when reinforcing basic psychomotor principles, such as posture or knot tying (eAppendix in the Supplement, examples 8 and 9).

Educational Value

Quiz Ref IDAttendings and residents expressed sentiments that coaching was educationally valuable in a way that complemented but was not redundant with the intraoperative experience. Both referenced the pressure they perceive to avoid consuming OR time with teaching:

Resident: You get different feedback, and it’s nice…for me to sit here and watch you go to different things than you might otherwise mention in the OR.

Coach: And you get more time to ask questions…. You don’t feel like, “Well, we have to finish this operation, and I’ll learn about this later.” Well, now it’s later, and we can go more in detail…. Video reviews are fantastic teaching tools…. It would be great to make the videos available for the residents to review whenever…. [The resident could] look at this and say, “Here’s how I did it last time. I’m going to do another one of these next week…. How do I want to get ready?” …If another resident is going to do one of these next week…[the resident] could say, “Hey, you should look at this video…that [the attending] did with me. He’ll probably do it the same way. You’ll be ready.” I think there’s a lot of learning opportunity here.

Discussion

The International Coach Federation defines coaching as “partnering with clients in a thought-provoking and creative process that inspires them to maximize their personal and professional potential.”25 A novel concept in surgical education, coaching has been studied in limited settings, most of them simulated and all centered around a discrete skill set. Small randomized controlled trials have shown that coaching improves technical assessment scores in medical students26 and junior orthopedic residents,27 as well as nontechnical skill scores in senior surgical residents during simulations.28 Bonrath et al29 conducted a randomized controlled trial of video-based coaching on laparoscopic jejunostomies; coached residents scored higher on technical assessments. Case volume, the classic proxy for technical competence, correlated with scores only in coached residents; without a coach’s interpretation and feedback, additional operative experience seemed to make no contribution to residents’ technical learning curve.

In our application, video-based coaching demonstrated usefulness in teaching a variety of topics, technical and nontechnical. It was efficient, requiring only 1 hour (often less) of surgeons’ time, with minimal preparation, and delivering greater than 50% more teaching points per unit time than operating. While one might anticipate that more teaching would occur merely because of the absence of concurrent clinical responsibility, the intervention’s primary value lies in the individualization, depth, and quality of instruction.

While coaching, attending surgeons were more likely to perform educational needs assessments, and residents were more likely to initiate teaching points than they were in the OR. According to adult learning theory, education that is centered around the learner is more likely to motivate practice change.30,31 Cox and Swanson found that one of the largest differences between superior and mediocre surgical teaching was “awareness and sensitivity to resident learning needs.”32(p252) Yet, only 18% of general surgery residents nationally report that faculty help them identify their personal educational operative goals.33 Residents’ learning needs cannot be assumed given the disparity between residents’ and attendings’ OR educational priorities.34 In adherence with these principles, we did not dictate content for the video-based coaching sessions; therefore, our intervention contrasts starkly with other studies of coaching for surgical trainees, in which learning goals were predetermined. Indeed, in their interviews of surgeons, Mutabdzic et al35 found that a major barrier to coaching was surgeons’ fear of losing control over their learning agenda. As pointed out in the editorial36 accompanying that article, such self-determination is actually the goal of coaching; the importance of residents’ input in defining their learning goals cannot be underestimated.

Residents across the country describe communication as a key attribute of a great teacher.37 However, by our count, instructing—directing without explanation—comprises a large proportion of the teaching in the OR. This finding is consistent with a study by Chen et al38 of surgical videos, in which most intraoperative guidance was in the form of directing. After analyzing videorecorded operations, Roberts et al concluded that such “instrumental” interactions, in which the attending directs the resident through various actions without explanation or guidance, “may be lost opportunities for deliberate education.”23(p649) We found that surgeons less frequently used instructing and more frequently used informing while coaching; hence, video-based coaching enabled surgeons to reconcile the “lost opportunities” in the OR described by Roberts et al.

Prior observational data have shown that technical steps are more frequently taught in the OR than decision-making ones.39 Similarly, we found that anatomic principles and operative steps were more commonly taught in the OR. After interviewing surgical attendings and residents about intraoperative learning, Cope et al concluded that such factual knowledge is “suitable for the junior learner.”22(p1126) The tendency to simplify the teaching in the OR may reflect surgeons’ (necessary) preoccupation with operating while in the OR. In their human factors analysis of intraoperative teaching, Lio et al40 hypothesize that trainees expend so much mental effort on the primary task of operating that there is no remaining processing ability to continually assess and plan upcoming steps. Our coaches shifted focus to more advanced topics—intraoperative decision making and failure to progress—during video review, again demonstrating that teaching opportunities missed in the OR may be recaptured with coaching. Moreover, coaches continued to teach technical skills, implying usefulness of coaching at all levels.

Cope et al described a novel educational theme of “sensory semiosis,” or “learning to interpret visual and haptic cues.”22(p1127) They noted that such cues “can seldom be adequately simulated”22(p1129) and advocated intraoperative video review for visual training. Of surgical residents with access to videorecording, 94% reported that it was a helpful educational tool.33 Video is a known trigger for “reflection on action.”30 However, we warn that video review alone is not enough; the expertise of a coach in facilitating reflection is critical. While residents increased the rate at which they initiated teaching points during the coaching sessions, most were initiated by the coaches. In the study by Bonrath et al,29 only coached residents’ self-assessments correlated with those of experts, indicating that coaching improves the accuracy of self-reflection, a skill vital to lifelong learning.

Given well-established disparities between attendings and residents in their perception of teaching,37,41-43 the objective measurement of teaching points by third-party observers was a methodological strength of our study. The use of videorecording and audiorecording further increased the accuracy and reproducibility of coding. Our mixed-methods approach combined statistical significance with a level of granularity only achievable through qualitative analysis. Our data set of 10 cases, representing more than 31 hours of intraoperative time, is considerable in qualitative research. In dealing with the coaching sessions that lasted less than 1 hour, we used actual counts and did not extrapolate them over the unused time; as such, our results are biased against coaching.

Limitations

There are several limitations to our study. It was conducted at a single academic tertiary care hospital, which may limit generalizability. Operating room videos may have failed to capture all preoperative discussions between attendings and residents. Because we most frequently started recording before the arrival of the surgeon, we captured several such conversations. Pernar et al44 observed that attendings and residents were simultaneously present before surgery in only 18.5% of cases, reassuring us that we missed few such teaching moments. The Hawthorne effect is always a concern in observational research, but it applied to both the OR teaching and the video-based coaching sessions. We do not have baseline data on intraoperative teaching at our institution; however, we did not notice a qualitative difference between the teaching in these cases and in those that we previously recorded to answer other research questions. In all except 1 session, attendings were coaching for the first time; because they were unaware of the structure or content of the coaching session at the time of the operation, they could not have anticipatorily altered their intraoperative teaching.

Conclusions

Video-based coaching is a novel and feasible modality for supplementing intraoperative learning. It is particularly useful for individualizing instruction and feedback to each resident, increasing the depth of what is taught, and teaching higher-level concepts, such as decision making. Requiring only videos, which are easily obtainable with standard laparoscopic equipment, and minimal preparation from surgeons, we believe that video-based coaching is an easily replicable and effective educational intervention.

Back to top
Article Information

Accepted for Publication: September 3, 2016.

Corresponding Author: Yue-Yung Hu, MD, MPH, Department of Surgery, Connecticut Children’s Medical Center, 282 Washington St, Hartford, CT 06106 (yhu@connecticutchildrens.org).

Published Online: December 14, 2016. doi:10.1001/jamasurg.2016.4619

Author Contributions: Drs Hu and Smink had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Hu, Yule, Arriaga, Greenberg, Lipsitz, Gawande, Smink.

Acquisition, analysis, or interpretation of data: Hu, Mazer, Yule, Arriaga, Lipsitz, Smink.

Drafting of the manuscript: Hu, Mazer, Greenberg, Lipsitz.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hu, Mazer, Greenberg, Lipsitz.

Administrative, technical, or material support: Hu, Greenberg, Gawande, Smink.

Study supervision: Yule, Arriaga, Greenberg, Smink.

Conflict of Interest Disclosures: Dr Greenberg reported being a consultant for Johnson & Johnson and reported receiving a grant funded by Covidien/Medtronic, both outside of the scope of this work. No other disclosures were reported.

Funding/Support: This work was supported by grants L30 RR031458-01 (Dr Hu) and 2T32 DK00754-12 from the National Institutes of Health, by the Rx Foundation (Hadley, Massachusetts) (Dr Greenberg), and by the Controlled Risk Insurance Company [CRICO]/Risk Management Foundation (Boston, Massachusetts) (Dr Greenberg).

Role of the Funder/Sponsor: None of the funding organizations had a role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Meeting Presentation: A preliminary analysis of this work (using a more limited data set) was presented at the 2015 American College of Surgeons Clinical Congress Surgical Forum; October 5, 2015; Chicago, Illinois (Surgical Education I: “Post Game Analysis: Postoperative Video-Based Coaching for Residents”). It won the Excellence in Research: Education Award and was published in the corresponding Journal of the American College of Surgeons15 special issue.

References
1.
Ericsson  KA.  Deliberate practice and acquisition of expert performance: a general overview.  Acad Emerg Med. 2008;15(11):988-994.PubMedGoogle ScholarCrossref
2.
Bohnen  JD, Chang  DC, George  BC.  Changing trends in operating room times between teaching and non-teaching cases: less time for learning?  J Am Coll Surg. 2015;221(4):S49.Google ScholarCrossref
3.
Teman  NR, Gauger  PG, Mullan  PB, Tarpley  JL, Minter  RM.  Entrustment of general surgery residents in the operating room: factors contributing to provision of resident autonomy.  J Am Coll Surg. 2014;219(4):778-787.PubMedGoogle ScholarCrossref
4.
Fairfax  LM, Christmas  AB, Green  JM, Miles  WS, Sing  RF.  Operative experience in the era of duty hour restrictions: is broad-based general surgery training coming to an end?  Am Surg. 2010;76(6):578-582.PubMedGoogle Scholar
5.
Kairys  JC, McGuire  K, Crawford  AG, Yeo  CJ.  Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees?  J Am Coll Surg. 2008;206(5):804-813. PubMedGoogle ScholarCrossref
6.
Schwartz  SI, Galante  J, Kaji  A,  et al.  Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.  JAMA Surg. 2013;148(9):829-833.PubMedGoogle ScholarCrossref
7.
Snyder  RA, Phillips  SE, Terhune  KP.  Impact of implementation of a pediatric surgery fellowship on general surgery resident operative volume.  J Surg Educ. 2012;69(6):753-758.PubMedGoogle ScholarCrossref
8.
Hanks  JB, Ashley  SW, Mahvi  DM,  et al.  Feast or famine? the variable impact of coexisting fellowships on general surgery resident operative volumes.  Ann Surg. 2011;254(3):476-485. PubMedGoogle ScholarCrossref
9.
Bell  RH  Jr.  Why Johnny cannot operate.  Surgery. 2009;146(4):533-542.PubMedGoogle ScholarCrossref
10.
Podnos  YD, Wilson  SE.  Threats to the surgical residency in the academic medical center.  Arch Surg. 2001;136(2):161-164.PubMedGoogle ScholarCrossref
11.
Sandhu  G, Teman  NR, Minter  RM.  Training autonomous surgeons: more time or faculty development?  Ann Surg. 2015;261(5):843-845.PubMedGoogle ScholarCrossref
12.
Nauta  RJ.  The surgical residency and Accreditation Council for Graduate Medical Education reform: steep learning or sleep learning?  Am J Surg. 2011;201(6):715-718.PubMedGoogle ScholarCrossref
13.
Mattar  SG, Alseidi  AA, Jones  DB,  et al.  General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors.  Ann Surg. 2013;258(3):440-449.PubMedGoogle ScholarCrossref
14.
Davis  D, O’Brien  MA, Freemantle  N, Wolf  FM, Mazmanian  P, Taylor-Vaisey  A.  Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?  JAMA. 1999;282(9):867-874.PubMedGoogle ScholarCrossref
15.
Hu  YY, Peyre  SE, Arriaga  AF,  et al.  Postgame analysis: using video-based coaching for continuous professional development.  J Am Coll Surg. 2012;214(1):115-124.PubMedGoogle ScholarCrossref
16.
Glarner  CE, Hu  YY, Chen  CH,  et al.  Quantifying technical skills during open operations using video-based motion analysis.  Surgery. 2014;156(3):729-734.PubMedGoogle ScholarCrossref
17.
KU Center for Research on Learning.  What Coaches Do. 3rd iteration. Lawrence: University of Kansas; June 2010.
18.
Newman  LR, Roberts  DH, Schwartzstein  RM. Peer Observation of Teaching Handbook. www.mededportal.org/publication/9150. Published by MedEdPORTAL in 2012. Accessed November 4, 2016.
19.
Yule  S.  Checklist for Trainers Facilitating a NOTSS Debrief. Aberdeen, Scotland: University of Aberdeen; 2010.
20.
Guerlain  S, Adams  RB, Turrentine  FB,  et al.  Assessing team performance in the operating room: development and use of a “black-box” recorder and other tools for the intraoperative environment.  J Am Coll Surg. 2005;200(1):29-37.PubMedGoogle ScholarCrossref
21.
Hauge  LS, Wanzek  JA, Godellas  C.  The reliability of an instrument for identifying and quantifying surgeons’ teaching in the operating room.  Am J Surg. 2001;181(4):333-337.PubMedGoogle ScholarCrossref
22.
Cope  AC, Mavroveli  S, Bezemer  J, Hanna  GB, Kneebone  R.  Making meaning from sensory cues: a qualitative investigation of postgraduate learning in the operating room.  Acad Med. 2015;90(8):1125-1131.PubMedGoogle ScholarCrossref
23.
Roberts  NK, Brenner  MJ, Williams  RG, Kim  MJ, Dunnington  GL.  Capturing the teachable moment: a grounded theory study of verbal teaching interactions in the operating room.  Surgery. 2012;151(5):643-650.PubMedGoogle ScholarCrossref
24.
Hu  YY, Arriaga  AF, Roth  EM,  et al.  Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care.  Ann Surg. 2012;256(2):203-210.PubMedGoogle ScholarCrossref
25.
International Coach Federation. What is professional coaching? http://coachfederation.org/need/landing.cfm?ItemNumber=978&navItemNumber=567. Accessed March 8, 2016.
26.
Singh  P, Aggarwal  R, Tahir  M, Pucher  PH, Darzi  A.  A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills.  Ann Surg. 2015;261(5):862-869.PubMedGoogle ScholarCrossref
27.
Karam  MD, Thomas  GW, Koehler  DM,  et al.  Surgical coaching from head-mounted video in the training of fluoroscopically guided articular fracture surgery.  J Bone Joint Surg Am. 2015;97(12):1031-1039.PubMedGoogle ScholarCrossref
28.
Yule  S, Parker  SH, Wilkinson  J,  et al.  Coaching non-technical skills improves surgical residents’ performance in a simulated operating room.  J Surg Educ. 2015;72(6):1124-1130.PubMedGoogle ScholarCrossref
29.
Bonrath  EM, Dedy  NJ, Gordon  LE, Grantcharov  TP.  Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial.  Ann Surg. 2015;262(2):205-212.PubMedGoogle ScholarCrossref
30.
Grant  J.  Learning needs assessment: assessing the need.  BMJ. 2002;324(7330):156-159.PubMedGoogle ScholarCrossref
31.
Kaufman  DM.  Applying educational theory in practice.  BMJ. 2003;326(7382):213-216.PubMedGoogle ScholarCrossref
32.
Cox  SS, Swanson  MS.  Identification of teaching excellence in operating room and clinic settings.  Am J Surg. 2002;183(3):251-255.PubMedGoogle ScholarCrossref
33.
Snyder  RA, Tarpley  MJ, Tarpley  JL, Davidson  M, Brophy  C, Dattilo  JB.  Teaching in the operating room: results of a national survey.  J Surg Educ. 2012;69(5):643-649.PubMedGoogle ScholarCrossref
34.
Pugh  CM, DaRosa  DA, Glenn  D, Bell  RH  Jr.  A comparison of faculty and resident perception of resident learning needs in the operating room.  J Surg Educ. 2007;64(5):250-255.PubMedGoogle ScholarCrossref
35.
Mutabdzic  D, Mylopoulos  M, Murnaghan  ML,  et al.  Coaching surgeons: is culture limiting our ability to improve?  Ann Surg. 2015;262(2):213-216.PubMedGoogle ScholarCrossref
36.
Greenberg  CC, Klingensmith  ME.  The continuum of coaching: opportunities for surgical improvement at all levels.  Ann Surg. 2015;262(2):217-219.PubMedGoogle ScholarCrossref
37.
Butvidas  LD, Anderson  CI, Balogh  D, Basson  MD.  Disparities between resident and attending surgeon perceptions of intraoperative teaching.  Am J Surg. 2011;201(3):385-389.PubMedGoogle ScholarCrossref
38.
Chen  XP, Williams  RG, Sanfey  HA, Smink  DS.  A taxonomy of surgeons’ guiding behaviors in the operating room.  Am J Surg. 2015;209(1):15-20.PubMedGoogle ScholarCrossref
39.
Pernar  LI, Peyre  SE, Hasson  RM,  et al.  Exploring the content of intraoperative teaching.  J Surg Educ. 2016;73(1):79-84.PubMedGoogle ScholarCrossref
40.
Lio  CH, Carswell  CM, Strup  SE, Roth  JS, Grant  R. The operating room as classroom: understanding cognitive challenges facing surgical trainees. In: Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting. Thousand Oaks, CA: SAGE Publications; 2010:1571-1575.
41.
Claridge  JA, Calland  JF, Chandrasekhara  V, Young  JS, Sanfey  H, Schirmer  BD.  Comparing resident measurements to attending surgeon self-perceptions of surgical educators.  Am J Surg. 2003;185(4):323-327.PubMedGoogle ScholarCrossref
42.
Vollmer  CM  Jr, Newman  LR, Huang  G, Irish  J, Hurst  J, Horvath  K.  Perspectives on intraoperative teaching: divergence and convergence between learner and teacher.  J Surg Educ. 2011;68(6):485-494.PubMedGoogle ScholarCrossref
43.
Rose  JS, Waibel  BH, Schenarts  PJ.  Disparity between resident and faculty surgeons’ perceptions of preoperative preparation, intraoperative teaching, and postoperative feedback.  J Surg Educ. 2011;68(6):459-464.PubMedGoogle ScholarCrossref
44.
Pernar  LI, Breen  E, Ashley  SW, Peyre  SE.  Preoperative learning goals set by surgical residents and faculty.  J Surg Res. 2011;170(1):1-5.PubMedGoogle ScholarCrossref
×