eAppendix. Full OR and Coaching Session Dialogue
Customize your JAMA Network experience by selecting one or more topics from the list below.
Hu Y, Mazer LM, Yule SJ, et al. Complementing Operating Room Teaching With Video-Based Coaching. JAMA Surg. 2017;152(4):318–325. doi:10.1001/jamasurg.2016.4619
Copyright 2016 American Medical Association. All Rights Reserved.
Is post hoc video-based coaching an effective modality for teaching residents to operate?
This mixed methods analysis compared teaching in a video-based coaching session with teaching during the corresponding operating room case. During video-based coaching, instruction was more individualized to the resident, and higher-level concepts, such as decision making, were discussed.
In this era of reduced resident operative time and autonomy, intraoperative education may be supplemented by video-based coaching.
Surgical expertise demands technical and nontechnical skills. Traditionally, surgical trainees acquired these skills in the operating room; however, operative time for residents has decreased with duty hour restrictions. As in other professions, video analysis may help maximize the learning experience.
To develop and evaluate a postoperative video-based coaching intervention for residents.
Design, Setting, and Participants
In this mixed methods analysis, 10 senior (postgraduate year 4 and 5) residents were videorecorded operating with an attending surgeon at an academic tertiary care hospital. Each video formed the basis of a 1-hour one-on-one coaching session conducted by the operative attending; although a coaching framework was provided, participants determined the specific content collaboratively. Teaching points were identified in the operating room and the video-based coaching sessions; iterative inductive coding, followed by thematic analysis, was performed.
Main Outcomes and Measures
Teaching points made in the operating room were compared with those in the video-based coaching sessions with respect to initiator, content, and teaching technique, adjusting for time.
Among 10 cases, surgeons made more teaching points per unit time (63.0 vs 102.7 per hour) while coaching. Teaching in the video-based coaching sessions was more resident centered; attendings were more inquisitive about residents’ learning needs (3.30 vs 0.28, P = .04), and residents took more initiative to direct their education (27% [198 of 729 teaching points] vs 17% [331 of 1977 teaching points], P < .001). Surgeons also more frequently validated residents’ experiences (8.40 vs 1.81, P < .01), and they tended to ask more questions to promote critical thinking (9.30 vs 3.32, P = .07) and set more learning goals (2.90 vs 0.28, P = .11). More complex topics, including intraoperative decision making (mean, 9.70 vs 2.77 instances per hour, P = .03) and failure to progress (mean, 1.20 vs 0.13 instances per hour, P = .04) were addressed, and they were more thoroughly developed and explored. Excerpts of dialogue are presented to illustrate these findings.
Conclusions and Relevance
Video-based coaching is a novel and feasible modality for supplementing intraoperative learning. Objective evaluation demonstrates that video-based coaching may be particularly useful for teaching higher-level concepts, such as decision making, and for individualizing instruction and feedback to each resident.
Acquiring expertise in a complex skill, such as operating, requires “deliberate practice” (ie, the facilitation of specific task improvement through immediate feedback, problem solving, and performance refinement).1 Traditionally, surgical trainees acquired deliberate practice in the operating room (OR), with graduated independence throughout training. However, as resident operative experience becomes increasingly limited2-4 by duty hour restrictions,5,6 the implementation of fellowships,7,8 and productivity, quality, and safety pressures on attendings,3,9-11 concerns about the competence of new graduates have arisen.9-13
While surgical training programs supplement their curricula with lectures, web-based learning, and simulation, nothing yet approximates the rich experience of operating. Many efforts neglect adult learning theory principles; they are teacher driven rather than learner driven, didactic rather than interactive, and amassed rather than distributed over time.14 Few evidence-based methods exist to help trainees evaluate or improve their operative performance.
In other professions requiring accelerated skill acquisition, coaches provide learners with perspective and immediate, targeted feedback.1 Athletes routinely review performance videos with coaches, who guide them through an individualized self-improvement process. With such directed feedback, deliberate practice is achieved, accelerating the learning curve. Our group previously piloted a video-based coaching intervention for practicing surgeons, demonstrating feasibility, acceptability, and a high perception of value.15 Qualitative analysis of this continuing professional development module revealed a natural predominance of critical thinking topics, an aspect we suspect is underemphasized in intraoperative teaching. We sought to expand the scope of investigation to residents, hypothesizing that video-based coaching would fill these gaps in their operative education.
We recruited postgraduate year 4 and 5 general surgery residents (to maximize the chance of capturing complex discussions) and attending surgeons at Brigham and Women’s Hospital, an academic tertiary care center. A convenience sample of 10 residents was offered a choice of coaches from colorectal, minimally invasive, oncologic, acute care, and transplant surgery.
We videorecorded and audiorecorded 1 operation for each resident-coach dyad (Table 1). Videos captured the surgical field and entire OR, as previously described.16 Each video formed the basis of a 1-hour one-on-one coaching session conducted by the operative attending, scheduled 0 to 43 days after surgery (mean of 13.6 and median of 5.5 days elapsed). To prepare, coaches were given a “Coaching Residents” pamphlet, adapted from handbooks on coaching or giving feedback in other settings.17-19 The content of the video-based coaching sessions was determined entirely by the participants; we neither mandated nor suggested discussion topics. The video-based coaching sessions were audiorecorded. Written informed consent was obtained from surgeons and patients. The study was approved by the Partners Institutional Review Board.
Operating room videorecordings20 and coaching audiorecordings were transcribed with patient and surgeon identifiers removed. A preliminary coding schema was adapted from prior surgical education work15,21-23 and then refined through an iterative inductive coding process. Two surgical research fellows (Y.-Y.H. and L.M.M.) independently reviewed each transcript to identify teaching points, coding them by initiator, content, teaching technique, and tone. When the initiator was the resident, the attending response was coded as open, closed, or reflective. Two deidentified transcripts of OR videos from a prior study24 served as training cases for intercoder calibration. Interrater discrepancies were resolved by discussion until consensus was achieved. Transcripts were recoded iteratively until thematic saturation was reached. Table 2 lists the final coding schema, including definitions and examples. Modeling was not included as a teaching technique; teaching had to be explicitly verbalized and intentionally directed toward the resident to be counted.
χ2 Tests were used to compare the OR and the video-based coaching sessions by the percentage of teaching points initiated by the attending vs the resident and the response of the attending to resident-initiated points. Paired t tests were used to compare the mean counts of teaching points per hour, based on content and technique, between the OR and the video-based coaching sessions. For the coaching sessions lasting less than 1 hour, the time was rounded up to 1 hour rather than extrapolated. We performed a sensitivity analysis in which we limited OR time to the attending presence; results were similar, so only the original analysis is reported herein. Significance was set at 2-sided P < .05. All analyses were performed using statistical software (SAS, version 9.4; SAS Institute Inc).
A comparison of teaching points was made between the OR and the video-based coaching sessions with respect to initiator, content, teaching technique, and tone, adjusted for time. While coaching, surgeons made more teaching points per unit time. Quiz Ref IDAmong 10 OR cases, we observed 1977 teaching points during 31.4 hours of video (63.0 per hour); in the coaching sessions, we counted 719 teaching points during 7.0 hours of video (102.7 per hour).
While coaching, surgeons were more focused on resident education; they performed educational needs assessments almost 10 times more often (3.30 vs 0.28, P = .04). Quiz Ref IDResidents were better able to direct teaching to accommodate their own learning needs: 17% (331 of 1977) of teaching points were initiated by residents in the OR compared with 27% (198 of 729) in the videobased coaching sessions (P < .001).
While coaching, surgeons asked almost 3 times more questions to prompt reflection or critical thinking (9.30 vs 3.32, P = .07) and set 10 times more goals to facilitate future practice (2.90 vs 0.28, P = .11). They also validated residents’ experiences (eg, regarding the challenging nature of a case) more frequently (8.40 vs 1.81, P < .01), thus softening critical feedback and increasing resident engagement (Table 2).
Anatomy (8.48 vs 3.10, P = .01) and progress through the steps of the procedure (20.30 vs 7.50, P < .01) were more commonly discussed in the OR. Quiz Ref IDIn the video-based coaching sessions, higher-level concepts—specifically, intraoperative decision making (mean, 9.70 vs 2.77 instances per hour, P = .03) and failure to progress (mean, 1.20 vs 0.13 instances per hour, P = .04)—were more frequently discussed (Table 2).
Teaching was less directive and more explanatory during video-based coaching; attendings preferentially used instructive techniques in the OR (24.48 vs 0.10 teaching points, P < .01) and informative techniques while coaching (44.80 vs 31.05 teaching points, P = .20) (Table 3). Quiz Ref IDAttending responses to resident-initiated teaching points tended to be more thoughtful and more thorough during the coaching sessions. In the OR, most attending responses were open (95.7% [310 of 324]), few were reflective (2.5% [8 of 324]), and fewer were closed (1.9% [6 of 324]). In the coaching sessions, while open responses predominated (82.8% [149 of 180]), reflective responses were common (16.7% [30 of 180]), and closed responses were nonexistent (0% [0 of 180]) (P < .01) (Table 2).
Even when the same teaching point was made in both venues, it was generally more thoroughly articulated in the video-based coaching sessions. During a biopsy of a retroperitoneal mass, the resident asked about the approach and received a response that was informative but minimally so: Resident: “So you think we should go through [the] lesser sac here?” Attending: “Yeah, I think you need to to get to that….” In the corresponding coaching session, the coach used several techniques—questioning, informing, and validating—to deepen the resident’s understanding about intraoperative decision making surrounding the exposure:
Coach: Where are your windows in the upper abdomen to…get back where that thing was? Do you remember what you were thinking about the different options for getting back there…?
Resident: …We talked about going through the gastrohepatic ligament, which is more or less what we did in the end, right? Or we could have gone posterior to the stomach [be]cause we were going into the lesser sac; we could’ve kocherized the duodenum….
Coach: I think in this case it was easier to go through the gastrohepatic ligament because it was a direct route down to the thing. It was more to the right of the aorta than it was to the left…. Remember, we had this big discussion: where’s the cava…? …Two reasons why there’s complex [anatomy]. (1) …You just don’t have good exposure because you’re trying to do it through a tiny hole…. (2) You [have] just got tumor socked in there, and you just can’t tell. So…you think about what things are going to be in that neighborhood and what you need to know. And you get a good sense of strategy and reassurance when you’ve clearly identified all the key structures. So we knew, “…Where is the cava?” because there were 2 options…. If the thing had pushed the cava posteriorly…we could have been coming down on the cava, which was not what we wanted to do. We thought the cava was anterior, right? …Triangulate the scans. We thought we were pulling the cava up….
Often during the video-based coaching sessions, previously unrecognized gaps in residents’ comprehension of a case came to light. One resident admitted confusion with the attending’s instructions during the case: Resident: “I remember at one point you said to…kocherize the duodenum by feel, and I was like, ‘No idea how to do that.’” Throughout the session, the coach set goals for this technique: choosing a relevant case for further practice, enumerating the steps, and stopping to further clarify points as requested by the resident (eAppendix in the Supplement, example 1).
Similarly, during a complex retroperitoneal sarcoma resection, the attending directed the resident through steps of the dissection without extensive explanation of the attendant decision making. The resident did not interject but asked for clarification during the coaching session: Resident: “I’m just looking at…taking down the transverse colon. Did we do that just for…retraction at the point?” The coach then used a mixture of questioning and informing to explain the relevant anatomy and intraoperative decision making (eAppendix in the Supplement, example 2).
Without coaching, these residents would have missed nuances of the critical decision making required for the cases. Learning opportunities would have been lost.
Post hoc coaching allows for retrospective insight. With knowledge of pathology report results, an attending was able to reflect that the open approach had given them tactile feedback that changed the operation and hence the patient’s prognosis (eAppendix in the Supplement, example 3). Similarly, in reviewing a jejunostomy tube placement (a procedurally straightforward case), the resident and coach reflected on the postoperative death, initiating a novel discussion about decision making in the setting of terminal disease (eAppendix in the Supplement, example 4), a topic often neglected in surgery.
Coaching also allowed for the discussion of theoretical complications. In the case of a splenectomy, which “turned out to be the perfect, ideal situation,” the attending used the coaching session to review alternate scenarios “if it [had not been] so safe and clean.” The attending questioned the resident about anatomic and pathophysiological issues, technique, and potential intraoperative pitfalls (eAppendix in the Supplement, examples 5, 6, and 7). Such one-on-one personalized discussion of intraoperative management remains rare in today’s training paradigm despite being critical to everyday practice.
Increased depth was seen during coaching not just when teaching higher-level concepts, such as decision making. It was also seen when reinforcing basic psychomotor principles, such as posture or knot tying (eAppendix in the Supplement, examples 8 and 9).
Quiz Ref IDAttendings and residents expressed sentiments that coaching was educationally valuable in a way that complemented but was not redundant with the intraoperative experience. Both referenced the pressure they perceive to avoid consuming OR time with teaching:
Resident: You get different feedback, and it’s nice…for me to sit here and watch you go to different things than you might otherwise mention in the OR.
Coach: And you get more time to ask questions…. You don’t feel like, “Well, we have to finish this operation, and I’ll learn about this later.” Well, now it’s later, and we can go more in detail…. Video reviews are fantastic teaching tools…. It would be great to make the videos available for the residents to review whenever…. [The resident could] look at this and say, “Here’s how I did it last time. I’m going to do another one of these next week…. How do I want to get ready?” …If another resident is going to do one of these next week…[the resident] could say, “Hey, you should look at this video…that [the attending] did with me. He’ll probably do it the same way. You’ll be ready.” I think there’s a lot of learning opportunity here.
The International Coach Federation defines coaching as “partnering with clients in a thought-provoking and creative process that inspires them to maximize their personal and professional potential.”25 A novel concept in surgical education, coaching has been studied in limited settings, most of them simulated and all centered around a discrete skill set. Small randomized controlled trials have shown that coaching improves technical assessment scores in medical students26 and junior orthopedic residents,27 as well as nontechnical skill scores in senior surgical residents during simulations.28 Bonrath et al29 conducted a randomized controlled trial of video-based coaching on laparoscopic jejunostomies; coached residents scored higher on technical assessments. Case volume, the classic proxy for technical competence, correlated with scores only in coached residents; without a coach’s interpretation and feedback, additional operative experience seemed to make no contribution to residents’ technical learning curve.
In our application, video-based coaching demonstrated usefulness in teaching a variety of topics, technical and nontechnical. It was efficient, requiring only 1 hour (often less) of surgeons’ time, with minimal preparation, and delivering greater than 50% more teaching points per unit time than operating. While one might anticipate that more teaching would occur merely because of the absence of concurrent clinical responsibility, the intervention’s primary value lies in the individualization, depth, and quality of instruction.
While coaching, attending surgeons were more likely to perform educational needs assessments, and residents were more likely to initiate teaching points than they were in the OR. According to adult learning theory, education that is centered around the learner is more likely to motivate practice change.30,31 Cox and Swanson found that one of the largest differences between superior and mediocre surgical teaching was “awareness and sensitivity to resident learning needs.”32(p252) Yet, only 18% of general surgery residents nationally report that faculty help them identify their personal educational operative goals.33 Residents’ learning needs cannot be assumed given the disparity between residents’ and attendings’ OR educational priorities.34 In adherence with these principles, we did not dictate content for the video-based coaching sessions; therefore, our intervention contrasts starkly with other studies of coaching for surgical trainees, in which learning goals were predetermined. Indeed, in their interviews of surgeons, Mutabdzic et al35 found that a major barrier to coaching was surgeons’ fear of losing control over their learning agenda. As pointed out in the editorial36 accompanying that article, such self-determination is actually the goal of coaching; the importance of residents’ input in defining their learning goals cannot be underestimated.
Residents across the country describe communication as a key attribute of a great teacher.37 However, by our count, instructing—directing without explanation—comprises a large proportion of the teaching in the OR. This finding is consistent with a study by Chen et al38 of surgical videos, in which most intraoperative guidance was in the form of directing. After analyzing videorecorded operations, Roberts et al concluded that such “instrumental” interactions, in which the attending directs the resident through various actions without explanation or guidance, “may be lost opportunities for deliberate education.”23(p649) We found that surgeons less frequently used instructing and more frequently used informing while coaching; hence, video-based coaching enabled surgeons to reconcile the “lost opportunities” in the OR described by Roberts et al.
Prior observational data have shown that technical steps are more frequently taught in the OR than decision-making ones.39 Similarly, we found that anatomic principles and operative steps were more commonly taught in the OR. After interviewing surgical attendings and residents about intraoperative learning, Cope et al concluded that such factual knowledge is “suitable for the junior learner.”22(p1126) The tendency to simplify the teaching in the OR may reflect surgeons’ (necessary) preoccupation with operating while in the OR. In their human factors analysis of intraoperative teaching, Lio et al40 hypothesize that trainees expend so much mental effort on the primary task of operating that there is no remaining processing ability to continually assess and plan upcoming steps. Our coaches shifted focus to more advanced topics—intraoperative decision making and failure to progress—during video review, again demonstrating that teaching opportunities missed in the OR may be recaptured with coaching. Moreover, coaches continued to teach technical skills, implying usefulness of coaching at all levels.
Cope et al described a novel educational theme of “sensory semiosis,” or “learning to interpret visual and haptic cues.”22(p1127) They noted that such cues “can seldom be adequately simulated”22(p1129) and advocated intraoperative video review for visual training. Of surgical residents with access to videorecording, 94% reported that it was a helpful educational tool.33 Video is a known trigger for “reflection on action.”30 However, we warn that video review alone is not enough; the expertise of a coach in facilitating reflection is critical. While residents increased the rate at which they initiated teaching points during the coaching sessions, most were initiated by the coaches. In the study by Bonrath et al,29 only coached residents’ self-assessments correlated with those of experts, indicating that coaching improves the accuracy of self-reflection, a skill vital to lifelong learning.
Given well-established disparities between attendings and residents in their perception of teaching,37,41-43 the objective measurement of teaching points by third-party observers was a methodological strength of our study. The use of videorecording and audiorecording further increased the accuracy and reproducibility of coding. Our mixed-methods approach combined statistical significance with a level of granularity only achievable through qualitative analysis. Our data set of 10 cases, representing more than 31 hours of intraoperative time, is considerable in qualitative research. In dealing with the coaching sessions that lasted less than 1 hour, we used actual counts and did not extrapolate them over the unused time; as such, our results are biased against coaching.
There are several limitations to our study. It was conducted at a single academic tertiary care hospital, which may limit generalizability. Operating room videos may have failed to capture all preoperative discussions between attendings and residents. Because we most frequently started recording before the arrival of the surgeon, we captured several such conversations. Pernar et al44 observed that attendings and residents were simultaneously present before surgery in only 18.5% of cases, reassuring us that we missed few such teaching moments. The Hawthorne effect is always a concern in observational research, but it applied to both the OR teaching and the video-based coaching sessions. We do not have baseline data on intraoperative teaching at our institution; however, we did not notice a qualitative difference between the teaching in these cases and in those that we previously recorded to answer other research questions. In all except 1 session, attendings were coaching for the first time; because they were unaware of the structure or content of the coaching session at the time of the operation, they could not have anticipatorily altered their intraoperative teaching.
Video-based coaching is a novel and feasible modality for supplementing intraoperative learning. It is particularly useful for individualizing instruction and feedback to each resident, increasing the depth of what is taught, and teaching higher-level concepts, such as decision making. Requiring only videos, which are easily obtainable with standard laparoscopic equipment, and minimal preparation from surgeons, we believe that video-based coaching is an easily replicable and effective educational intervention.
Accepted for Publication: September 3, 2016.
Corresponding Author: Yue-Yung Hu, MD, MPH, Department of Surgery, Connecticut Children’s Medical Center, 282 Washington St, Hartford, CT 06106 (email@example.com).
Published Online: December 14, 2016. doi:10.1001/jamasurg.2016.4619
Author Contributions: Drs Hu and Smink had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Hu, Yule, Arriaga, Greenberg, Lipsitz, Gawande, Smink.
Acquisition, analysis, or interpretation of data: Hu, Mazer, Yule, Arriaga, Lipsitz, Smink.
Drafting of the manuscript: Hu, Mazer, Greenberg, Lipsitz.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Hu, Mazer, Greenberg, Lipsitz.
Administrative, technical, or material support: Hu, Greenberg, Gawande, Smink.
Study supervision: Yule, Arriaga, Greenberg, Smink.
Conflict of Interest Disclosures: Dr Greenberg reported being a consultant for Johnson & Johnson and reported receiving a grant funded by Covidien/Medtronic, both outside of the scope of this work. No other disclosures were reported.
Funding/Support: This work was supported by grants L30 RR031458-01 (Dr Hu) and 2T32 DK00754-12 from the National Institutes of Health, by the Rx Foundation (Hadley, Massachusetts) (Dr Greenberg), and by the Controlled Risk Insurance Company [CRICO]/Risk Management Foundation (Boston, Massachusetts) (Dr Greenberg).
Role of the Funder/Sponsor: None of the funding organizations had a role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.
Meeting Presentation: A preliminary analysis of this work (using a more limited data set) was presented at the 2015 American College of Surgeons Clinical Congress Surgical Forum; October 5, 2015; Chicago, Illinois (Surgical Education I: “Post Game Analysis: Postoperative Video-Based Coaching for Residents”). It won the Excellence in Research: Education Award and was published in the corresponding Journal of the American College of Surgeons15 special issue.
Create a personal account or sign in to: