[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.
Risk-Adjusted Excess Body Weight Loss at 1 Year, According to Quartile of Surgical Skill
Risk-Adjusted Excess Body Weight Loss at 1 Year, According to Quartile of Surgical Skill
Figure 2.
Risk-Adjusted Rate of Resolution of Medical Comorbidities, According to Quartile of Surgical Skill
Risk-Adjusted Rate of Resolution of Medical Comorbidities, According to Quartile of Surgical Skill
Table 1.  
Characteristics of Patients Undergoing Laparoscopic Roux-en-Y Gastric Bypass, According to Peer Rating of Surgical Skill
Characteristics of Patients Undergoing Laparoscopic Roux-en-Y Gastric Bypass, According to Peer Rating of Surgical Skill
Table 2.  
Risk-Adjusted Rates of Long-term Outcomes After Laparoscopic Roux-en-Y Gastric Bypass, According to Peer Ratings of Surgical Skill
Risk-Adjusted Rates of Long-term Outcomes After Laparoscopic Roux-en-Y Gastric Bypass, According to Peer Ratings of Surgical Skill
1.
Hawn  MT, Vick  CC, Richman  J,  et al.  Surgical site infection prevention: time to move beyond the surgical care improvement program.  Ann Surg. 2011;254(3):494-499.PubMedArticle
2.
Haynes  AB, Weiser  TG, Berry  WR,  et al; Safe Surgery Saves Lives Study Group.  A surgical safety checklist to reduce morbidity and mortality in a global population.  N Engl J Med. 2009;360(5):491-499.PubMedArticle
3.
Stulberg  JJ, Delaney  CP, Neuhauser  DV, Aron  DC, Fu  P, Koroukian  SM.  Adherence to surgical care improvement project measures and the association with postoperative infections.  JAMA. 2010;303(24):2479-2485.PubMedArticle
4.
Martin  JA, Regehr  G, Reznick  R,  et al.  Objective Structured Assessment of Technical Skill (OSATS) for surgical residents.  Br J Surg. 1997;84(2):273-278.PubMedArticle
5.
Vassiliou  MC, Feldman  LS, Andrew  CG,  et al.  A global assessment tool for evaluation of intraoperative laparoscopic skills.  Am J Surg. 2005;190(1):107-113.PubMedArticle
6.
Zevin  B, Bonrath  EM, Aggarwal  R, Dedy  NJ, Ahmed  N, Grantcharov  TP; ATLAS group.  Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery.  J Am Coll Surg. 2013;216(5):955-965.e8.PubMedArticle
7.
Birkmeyer  JD, Finks  JF, O’Reilly  A,  et al; Michigan Bariatric Surgery Collaborative.  Surgical skill and complication rates after bariatric surgery.  N Engl J Med. 2013;369(15):1434-1442.PubMedArticle
8.
Birkmeyer  NJ, Share  D, Campbell  DA  Jr, Prager  RL, Moscucci  M, Birkmeyer  JD.  Partnering with payers to improve surgical quality: the Michigan plan.  Surgery. 2005;138(5):815-820.PubMedArticle
9.
Share  DA, Campbell  DA, Birkmeyer  N,  et al.  How a regional collaborative of hospitals and physicians in Michigan cut costs and improved the quality of care.  Health Aff (Millwood). 2011;30(4):636-645.PubMedArticle
10.
Birkmeyer  NJ, Dimick  JB, Share  D,  et al; Michigan Bariatric Surgery Collaborative.  Hospital complication rates with bariatric surgery in Michigan.  JAMA. 2010;304(4):435-442.PubMedArticle
11.
Erickson  P.  Evaluation of a population-based measure of quality of life: the Health and Activity Limitation Index (HALex).  Qual Life Res. 1998;7(2):101-114.PubMedArticle
12.
Livingston  EH, Ko  CY.  Use of the health and activities limitation index as a measure of quality of life in obesity.  Obes Res. 2002;10(8):824-832.PubMedArticle
13.
Finks  JF, Kole  KL, Yenumula  PR,  et al; Michigan Bariatric Surgery Collaborative, from the Center for Healthcare Outcomes and Policy.  Predicting risk for serious complications with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative.  Ann Surg. 2011;254(4):633-640.PubMedArticle
14.
Reznick  RK, MacRae  H.  Teaching surgical skills: changes in the wind.  N Engl J Med. 2006;355(25):2664-2669.PubMedArticle
15.
Livhits  M, Mercado  C, Yermilov  I,  et al.  Exercise following bariatric surgery: systematic review.  Obes Surg. 2010;20(5):657-665.PubMedArticle
16.
Livhits  M, Mercado  C, Yermilov  I,  et al.  Is social support associated with greater weight loss after bariatric surgery? a systematic review.  Obes Rev. 2011;12(2):142-148.PubMedArticle
17.
McMahon  MM, Sarr  MG, Clark  MM,  et al.  Clinical management after bariatric surgery: value of a multidisciplinary approach.  Mayo Clin Proc. 2006;81(10)(suppl):S34-S45.PubMedArticle
18.
American College of Surgeons. Resources for optimal care of the metabolic and Bariatric Surgery patients 2014. https://www.facs.org/quality%20programs/mbsaqip/standards. Published 2014. Accessed March 12, 2016.
19.
Ryan  KK, Tremaroli  V, Clemmensen  C,  et al.  FXR is a molecular target for the effects of vertical sleeve gastrectomy.  Nature. 2014;509(7499):183-188.PubMedArticle
20.
Greenberg  CC, Ghousseini  HN, Pavuluri Quamme  SR, Beasley  HL, Wiegmann  DA.  Surgical coaching for individual performance improvement.  Ann Surg. 2015;261(1):32-34.PubMedArticle
Original Investigation
June 15, 2016

Video Ratings of Surgical Skill and Late Outcomes of Bariatric Surgery

Author Affiliations
  • 1Center for Healthcare Outcomes and Policy, Department of Surgery, University of Michigan, Ann Arbor
  • 2Department of Surgery, Henry Ford Hospital, Detroit, Michigan
  • 3Dartmouth Hitchcock Medical Center, Hanover, New Hampshire
JAMA Surg. 2016;151(6):e160428. doi:10.1001/jamasurg.2016.0428
Abstract

Importance  Measures of surgeons’ skills have been associated with variations in short-term outcomes after laparoscopic gastric bypass. However, the effect of surgical skill on long-term outcomes after bariatric surgery is unknown.

Objective  To study the association between surgical skill and long-term outcomes of bariatric surgery.

Design, Setting, and Participants  In this retrospective observational study, 20 surgeons performing bariatric surgery submitted videos; surgeons were ranked on their skill level through blinded peer video review and sorted into quartiles of skill. Outcomes of bariatric surgery were then examined at the patient level across skill levels. The patients (N = 3631) undergoing surgery with these surgeons had 1-year postoperative follow-up data available between 2006 and 2012. The study was conducted using the Michigan Bariatric Surgery Collaborative, a prospective clinical registry of 40 hospitals performing bariatric surgery in the state of Michigan.

Exposure  Surgeon skill level.

Main Outcomes and Measures  Excess body weight loss at 1 year; resolution of medical comorbidities (hypertension, sleep apnea, diabetes, and hyperlipidemia), functional status, and patient satisfaction.

Results  Surgeons in the top and bottom quartiles had each been practicing for a mean of 11 years. Peer ratings of surgical skill varied from 2.6 to 4.8 on a 5-point scale. There was no difference between the best (top 25%) and worst (bottom 25%) performance quartiles when comparing excess body weight loss (67.2% vs 68.5%; P = .86) at 1 year. There were no differences in resolution of sleep apnea (62.6% vs 62.0%; P = .77), hypertension (47.1% vs 45.4%; P = .73), or hyperlipidemia (52.3% vs 63.4%; P = .45). Surgeons with the lowest skill rating had patients with higher rates of diabetes resolution (78.8%) when compared with the high-skill group (72.8%) (P = .01).

Conclusions and Relevance  In contrast to its effect on early complications, surgical skill did not affect postoperative weight loss or resolution of medical comorbidities at 1 year after laparoscopic gastric bypass. These findings suggest that long-term outcomes after bariatric surgery may be less dependent on a surgeon’s operative skill and instead be driven by other factors. Operative technique was not assessed in this analysis and should be considered in future studies.

Introduction

After a decade of focus on optimizing perioperative care, there has been a paradigm shift toward optimizing the quality of the operation itself. Until recently, surgical quality improvement has largely focused on improving systems of care, such as improving compliance with perioperative checklists or adherence to evidence-based guidelines for antibiotic administration.13 Efforts to improve perioperative care have contributed to significant gains in patient safety but have failed to provide insight on the effect of the surgeon on the operation itself. Studies from the surgical education literature have demonstrated that it is possible to directly assess surgical skill with high levels of reliability and reproducibility.46 Recently, these ratings of surgical skill were shown to be strongly associated with variation in postoperative outcomes.7

However, understanding how surgical skill relates to outcomes is in its nascent stages. While existing research has shown a link between surgeon skill level and short-term outcomes (eg, 30-day complication rates),7 to our knowledge, there have been no data evaluating the link between longer-term outcomes and surgeon performance. Long-term outcomes are important to consider in a number of operations, particularly after bariatric surgery when weight loss and comorbidity resolution occur typically beyond the first month after surgery. There are multiple ways that long-term outcomes could be affected by surgical skill. For example, short-term outcomes (eg, complications) could mediate suboptimal longer-term outcomes. For instance, a postoperative complication, such as a leak or a stricture, that resulted from an intraoperative technical error may result in a prolonged hospitalization and slow recovery. In turn, this may result in a patient’s limited ability to participate in postoperative nutritional counseling and exercise programs long-term. To our knowledge, there are no studies to date that evaluate the association between surgeon skill and these late operative outcomes. In addition, the current metrics of skill level have largely been validated in the educational realm and used to rate novices such as surgical residents. These current skill measures may have limitations in their applicability, which merit further investigation.

In this context, we conducted a study evaluating the association between surgeon skill and late surgical outcomes. To do this, we focused on laparoscopic Roux-en-y gastric bypass (LRYGB) because these patients have high rates of postoperative follow-up and multiple quantitative metrics on which to measure their late outcomes of surgery. Quiz Ref IDWe used existing surgical skill ratings of 20 practicing bariatric surgeons in the state of Michigan, as judged anonymously by their peers who evaluated the video of a typical case using a validated instrument. We then assessed the association between these skill ratings and long-term outcomes of bypass surgery, chiefly weight loss and resolution of medical comorbidities.

Box Section Ref ID

Key Points

  • Question What effect does a bariatric surgeon’s skill level have on long-term patient outcomes?

  • Findings In this observational study of nearly 4000 patients undergoing bariatric surgery, there was no linkage between the skill level of the practicing surgeon performing the operation and long-term patient outcomes such as weight loss or resolution of diabetes.

  • Meaning While great attention has been paid to the association between surgeons’ skills and their postoperative outcomes, not all outcomes appear to be directly associated with skill, at least as it is currently measured.

Methods
Data Source and Study Population

This study used data collected from the Michigan Bariatric Surgery Collaborative (MBSC), a prospectively collected clinical registry that includes 40 hospitals performing bariatric surgery within the state of Michigan.8,9 The collaborative collects data for both primary and revisional bariatric surgery that includes approximately 6000 patients annually.

The MBSC registry includes basic patient demographics and comorbidities, short-term outcomes, and 30-day complication rates; it additionally captures long-term weight loss and health status. All variable definitions are standardized across the 40 centers, and data collection is conducted by centrally trained data abstractors.10 The registry also has a high degree of intrahospital communication and collaboration, with quarterly meetings between surgeons and data abstractors. Each hospital is visited annually and audited to ensure accurate and complete data collection.

Our patient population included all patients who underwent primary LRYGB by 1 of the 20 surgeons who participated in the video surgical skills study. Patient data were obtained from cases that occurred between 2006 (the start of the MBSC registry) and 2012 (when the video skill ratings were performed), for which 1-year patient follow-up data were available.

This study was approved by the University of Michigan institutional review board. Oral consent was obtained from the participating surgeons. All patients included in the Michigan Bariatric Surgery Collaborative provide written consent for participation in ongoing research projects through the collaborative, which includes video recording.

Participating Surgeons and Rating of Surgical Skill

In a previous study within our research group, 20 surgeons from the state were recruited to participate in this analysis. Participation was entirely voluntary. Each surgeon submitted a video of himself or herself performing a laparoscopic gastric bypass. Surgeons were instructed to submit a representative case (ie, a “typical” case, but the choice of what video to submit was left to the individuals); all patient identifiers were removed from these videos.

Each video was professionally edited down to 3 critical portions of the operation: (1) creation of the gastric pouch, (2) gastrojejunostomy, and (3) jejunojejunostomy. Afterwards, the videos were distributed electronically for peer review. The specifics of this video peer review process have been published in detail elsewhere.7 Each video had at least 10 peer reviews, and the rating instrument used was a modified version of the Objective Structured Assessment of Technical Skills (OSATS). The OSATS instrument is the most commonly used for assessing surgical skill and has been externally validated, demonstrating construct and content validity for use in surgical trainees and practicing bariatric surgeons.46 Specifically for laparoscopic gastric bypass surgery, the procedure used in this analysis, the OSATS has been demonstrated to show wide variability among practicing bariatric surgeons. Moreover, the OSATS ratings were strongly linked to short-term outcomes, once again demonstrating construct validity.

Surgeons were rated using a 1-to-5 anchored Likert-type scale on overall skill and then on the 5 OSATS domains of skill: (1) ease of tissue handling, (2) tissue exposure, (3) instrument handling, (4) efficiency of time and motion, and (5) flow of the operation. Anchors were used, with a skill rating of 1 indicating the level of a general surgery chief resident; 3, that of an average bariatric surgeon; and 5, a master surgeon. No training was given to the peer raters and no attempt was made to establish rating norms.

Outcome Measures

The primary outcome of interest was excess body weight loss (EBWL) at 1 year after the operation. Secondary outcome measures were resolution of medical comorbidities at 1 year (diabetes, sleep apnea, hypertension, and hypercholesterolemia). We also assessed overall health and function using the Health and Activity Limitation Index (HALEX),11 which has been validated for use in evaluating the quality of life in patients with obesity,12 and patient satisfaction.

Statistical Analysis

In our conceptual model, skill measured at the surgeon level served as the exposure variable against which each outcome measure was assessed. Because each surgeon treated multiple patients, all analyses were adjusted for surgeon-level clustering.

As a measure of surgeon skill, the average peer rating across all 6 metrics of the rating instrument was used (the global rating and the 5 specific domains). The ratings across these domains correlated very closely with one another (correlation coefficients, 0.85-0.97). This close correlation was the rationale for using a simple average, as discussed in our prior work.7 Surgeons were categorized into 3 groups, sorted by quartiles of skill level (the bottom group representing the lowest quartile; the middle group, the second and third quartiles; and the top group, the fourth quartile). Outcome measures were assessed across these 3 groups using multivariate logistic regression models for each of our outcome measures. These models were risk adjusted for age, male sex, previous venous thromboembolism, mobility limitations, coronary artery disease, and pulmonary disease; the methods and justification for this risk adjustment have been described by our group in previous works.13 A post hoc power calculation demonstrated that our analysis had greater than 99% power to detect a 5% difference for all outcomes metrics (α = .05, 2-sided). All analyses were conducted in SAS version 9.4 (SAS Institute Inc).

Results
Surgeon Skill Ratings and Patient and Surgeon Characteristics

As previously described by our group, aggregate skill ratings for the 20 surgeons ranged from 2.6 to 4.8.7Quiz Ref ID The surgeons within the lowest skill quartile had mean skill ratings of 2.9; the highest quartile had mean ratings of 4.4 by comparison. The 5 individual domains of surgical skill correlated closely to the global rating; therefore, we used an overall average of all 6 ratings for the skill measures.

Surgeons’ skill level did not correlate with years in practice, fellowship completion, or practice location (ie, teaching vs nonteaching hospital). Surgeons in the top and bottom quartiles had each been practicing for a mean of 11 years (P = .44); 20% of surgeons in each of the top and bottom quartiles, respectively, had completed fellowship training in minimally invasive or bariatric surgery (P = .56). We found that surgical skill level was positively associated with annual procedural volume. Our group previously reported additional details on the demographics of the surgeons themselves.7

The total patient cohort included 3631 patients for whom follow-up data at 1 year for body weight, HALEX survey, and patient satisfaction were available. Table 1 demonstrates the characteristics of these patients across the quartiles of surgical skill. Overall, the cohorts were similar in age, sex, insurance status, and preoperative body mass index (calculated as weight in kilograms divided by height in meters squared). There were several differences in patient comorbidities that were statistically significant (eg, cardiovascular disease and musculoskeletal disease). However, there was no clear association between these differences and the surgeon skill levels, and they are likely attributable to the large sample size rather than clinical significance.

Excess Body Weight Loss

Body weight at 1 year was available for 3631 patients undergoing LRYGB among the 20 participating surgeons. There were no significant differences in EBWL at 1 year between the groups of surgical skill. Figure 1 shows the EBWL subdivided by quartile of surgeon skill. Quiz Ref IDPatients undergoing operations by surgeons in the top quartile of skill had a 67.2% EBWL compared with 68.5% in the bottom quartile (P = .86). Average preoperative body mass index was similar for each quartile as seen in Table 1.

Resolution of Medical Comorbidities

One-year follow-up data were available for 1041 patients with diabetes. The preoperative incidence of diabetes was similar across quartiles of surgical skill. Quiz Ref IDOf these patients, 72.8% in the highest skill quartile had resolution of diabetes (as defined by cessation of oral medication and/or insulin therapy) compared with 78.8% in the lowest skill quartile (P = .01). For patients with hypertension, follow-up data were available for 1618 patients. A total of 47.1% of patients in the highest skill quartile had resolution of hypertension compared with 45.4% in the lowest skill quartile (P = .73). For the 1191 patients with sleep apnea, there was no association between skill and resolution of sleep apnea (62.6% vs 62.0%; P = .77) and similarly for the 1106 patients with hyperlipidemia, no association was seen (52.3% vs 63.4%; P = .45). Figure 2 shows the relative rate of comorbidity resolution across surgical skill levels.

For each comorbidity, the number of patients is smaller compared with our overall patient cohort because the incidence rate of each comorbidity varies. In addition 1-year follow-up data were not available for all comorbidity measures in some patients.

Patient Function and Satisfaction

The HALEX survey contains questions regarding perceived health and activity limitation; scoring of the survey ranges from 0.1 (most dysfunctional) to 1.0 (healthiest state). Among the 3631 patients who completed the HALEX survey at 1 year, no association between surgeon skill and patient quality of life was seen, as patients in the lowest skill quartile had a mean HALEX score of 0.83 compared with 0.84 in the highest skill group (P = .57) (Table 2).

Quiz Ref IDWhen patients were asked how satisfied they were with their surgeon and surgery at 1 year, we did observe a positive association between surgeon skill and patient satisfaction. A total of 90.3% of patients in the highest skill quartile responded with “very satisfied” at 1 year compared with 87.1% in the lowest skill quartile (P = .05). However, while this small difference achieved statistical significance, it likely lacks clinical significance as all quartiles had high overall satisfaction. Table 2 summarizes all outcome measures according to surgeon skill level.

Discussion

To our knowledge, this is the first study to examine the association between operative skill and long-term outcomes after surgery among practicing surgeons. Across multiple patient-level measures, we found no association between levels of surgical skill and long-term outcomes after laparoscopic gastric bypass. At 1 year, there was no difference in EBWL when comparing patients who underwent surgery by a surgeon with a lower skill rating and patients who underwent surgery by a surgeon with a higher skill rating. Patients also experienced resolution of diabetes, hypertension, and hyperlipidemia at similar rates clinically, regardless of surgeon skill. Finally, there were no differences in patient-reported quality of life associated with surgical skill ratings. These findings are somewhat counterintuitive because there is a well-known association between other proxies of surgical skill (eg, surgeon volume) and most measured outcomes. In our present study, our higher skilled surgeons were, on average, higher volume but we found no association between OSATS peer ratings, a more direct measure of surgical skill, and long-term outcomes.

Prior studies that examined surgical skill were targeted mainly to novices and trainees and had no means of determining the long-term effect of their skill on outcomes.4,14 For bariatric surgery, variation in long-term outcomes have largely been assessed by comparing numerous patient factors, clinical and social support structure, and variables such as physical activity level, nutrition, and counseling.1517 It stands to reason that a surgeon’s operative abilities also have an effect on outcomes but prior metrics have been limited. Proxies for a surgeon’s skill, such as hospital volume and advanced fellowship training, have been found to be correlated with outcomes.18 However, these metrics fail to address many attributes associated with a surgeon’s skill, such as dexterity, precision, tissue handling, flow of operation, tissue exposure, and intraoperative decision-making skills. By analyzing peer-reviewed ratings of surgical videos, we had a unique opportunity to investigate the effect of these metrics on surgical outcomes.

Our study indicates that surgical skill does not have a profound effect on long-term outcomes after LRYGB, which stands in contrast to our prior study demonstrating a strong positive association between surgeon skill level and 30-day complication rates.7 Why would one expect surgical skill to be associated with long-term outcomes? There are multiple ways in which surgical skill could lead to poor long-term outcomes. First, skill could affect long-term outcomes as delayed sequelae of the initial complication (eg, a perioperative sleeve leak turning into a stricture or reflux). Second, intraoperative skill deficiencies could lead directly to long-term problems. For example, cork-screwing or kinking of the sleeve may not cause immediate problems but could ultimately manifest as stricture or gastroesophageal reflux. Finally, both of these short- and long-term complications may lead to additional hospitalizations or contribute to the patient’s inability to adhere to the multidisciplinary weight loss or exercise programs.

One possible reason for our finding is that the videos were rated for surgical skill alone and did not take into account technique-specific differences between each operation. That is to say, we measured how well each surgeon executed the various steps of the procedure (surgical skill) but did not analyze the steps themselves (operative technique). Given the likely variation in surgeons’ judgment about operative technique (ie, pouch size, length of roux limb, length of biliopancreatic limb, type of anastomosis, and management of hiatal hernias), there may be specific technical measures of quality affecting long-term outcomes that were not measured in this study. Another plausible explanation for the difference in short- and long-term outcomes between surgeons with high and low skill ratings is that long-term outcomes may be more dependent on program-specific measures, such as dietary education, patient engagement, and resources that promote healthy lifestyle choices. Because surgeons within the MBSC meet quarterly to discuss and participate in a wide variety of quality-improvement projects, it is possible that similar practice patterns have led to similar outcomes among all patients at 1 year, regardless of the surgeon’s operative ability. Finally, despite performing risk-adjusted statistical models to compare patients across various hospitals, there remains numerous genetic markers that may have a significant effect on weight loss and comorbidity resolution after bariatric surgery.19 Because these types of genetic markers have not been measured and many have yet to be determined, this study lacked the ability to identify them as potential significant predictors of weight loss.

Our study did have several limitations. First, this was a retrospective observational study and, as such, a number of factors outside of our measured variables may influence outcomes. However, we controlled for patient-level factors using the detailed patient variables from a clinical registry and the best available methods of risk adjustment. Second, there may have been selection bias among participating surgeons. Specifically, it is conceivable that surgeons with the best skill might be the only ones willing to submit videos. As a result, surgeons participating in our study may not be representative of the full range of skill. However, we did have a wide range of surgical skill ratings based on the OSATS instrument. Moreover, in a prior study,7 these variations in skill were tightly correlated with short-term outcomes. Thus, it is very likely that the surgeon skill levels included in this study span a clinically meaningful range. Furthermore, skill ratings were obtained from a single, self-selected video that was deemed to be typical. It is possible that the videos submitted demonstrated their very best single performance and not their overall practice. However, we previously performed a sensitivity analysis using a second video sample for each of the high- and low-performing surgeons, and there was a high level of consistency between videos.7

Third, there was no attempt to standardize or train the video raters and, owing to the large number of reviewers, no 2 surgeons were evaluated by the same set of raters. Nevertheless, there remained considerable differences among the skill ratings between each video and we noted high interrater reliability (0.85 to 0.97 across the skill-rating domains). Fourth, our skill ratings were obtained at a single point; however, our patient outcomes were measured over a several-year period. Surgeon skill may not have been static over this period as practicing surgeons refined their technique; an improved level of skill over time would potentially bias our findings toward a null result. However, all surgeons in our cohort had been in practice for similar durations (mean of 11 years in both the top and bottom quartiles), so a significant change in performance over time would have been expected in either group.

Finally, our ratings only measured surgeons’ skill for discrete portions of the operation: pouch creation, gastrojejunostomy, and jejunojejunostomy. These 3 videos were selected because we believed that they represented the most technically challenging portions of the operation, but they may not reflect all of the events that occurred in the operating room. Another aspect of surgical quality that was not measured in this study was the judgment and variation in surgical technique from surgeon to surgeon. Although surgical skill may account for differences in short-term complication rates, it is possible that operative technique (as discussed here) may account for long-term outcomes.

Our study has several important implications for practice. Recently, attention has turned to trying to better evaluate and improve the conduct of an operation as a whole through video and live coaching initiatives.20 Laparoscopic surgery is a prime target of such work because videos are simple to obtain using existing equipment as part of routine practice and therefore represent an untapped data source for studying both operative technique and surgical skill. These efforts have identified the importance of measuring both operative technique and surgical skill as well as better understanding the complex interplay between the two. We believe this present study is an important negative result because it informs this ongoing work by providing information on which outcomes we should expect to be affected by improving surgeon skill and which will not be affected. Further research is needed to understand the effect of surgical technique (how the operation is done) on long-term outcomes after bariatric surgery. Thus far, there has been great enthusiasm for improving surgical skill directly through initiatives such as these and through regional collaboratives that allow dissemination of best practices.9 Our findings highlight the importance of ongoing scientific work analyzing surgical videos for a broader set of procedures and a diverse set of outcomes so we can better understand which outcomes are most dependent on surgical skill.

Conclusions

Peer-review ratings of surgical skill did not affect postoperative weight loss or resolution of medical comorbidities at 1 year after laparoscopic gastric bypass. Although surgical skill may influence short-term complication rates and patient satisfaction ratings, these findings suggest that long-term outcomes after bariatric surgery may be more dependent on other factors not yet measured among patients, hospitals, or surgeons. Future studies should take advantage of video analysis by measuring both operative technique and surgical skill as a means of understanding a surgeon’s effect on surgical quality.

Back to top
Article Information

Corresponding Author: Christopher P. Scally, MD, Center for Healthcare Outcomes and Policy, Department of Surgery, University of Michigan, 2800 Plymouth Rd, Bldg 16, Ann Arbor, MI 48109 (cscally@med.umich.edu).

Accepted for Publication: February 11, 2016.

Published Online: April 13, 2016. doi:10.1001/jamasurg.2016.0428.

Author Contributions: Dr Scally had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Scally, Varban, Birkmeyer, Dimick.

Acquisition, analysis, or interpretation of data: Scally, Carlin, Birkmeyer, Dimick.

Drafting of the manuscript: Scally, Dimick.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Scally, Birkmeyer, Dimick.

Obtained funding: Birkmeyer, Dimick.

Administrative, technical, or material support: Scally, Dimick.

Study supervision: Varban, Carlin, Dimick.

Conflict of Interest Disclosures: Dr Dimick is a consultant and has an equity interest in ArborMetrix Inc, which provides software and analytics for measuring hospital quality and efficiency. The company had no role in the study herein. No other disclosures were reported.

Funding/Support: Dr Scally is supported by grant 5T32CA009672-23 from the National Cancer Institute. This study was supported by grants RO1 HS023597 from the Agency for Healthcare Research and Quality and RO1 DK101423 from the National Institute of Diabetes and Digestive and Kidney Diseases to Dr Dimick.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: The Michigan Bariatric Surgery Collaborative members include James Pilkington, MD (Gratiot Medical Center), Marek Lutrzykowski, MD (Harper University Hospital), Mark Pleatman, MD (Crittenton Hospital and Medical Center, Harper University Hospital, St Joseph Oakland), Mohamed Gazayerli, MD (Crittenton Hospital and Medical Center, Harper University Hospital, St Joseph Oakland), Steve Poplawski, MD (Forest Health Medical Center), David Kam, MD (Munson Medical Center), John Schram, MD (Spectrum Health Zeeland, Spectrum Health), Ernest Cudjoe, MD (Gratiot, MidMichigan Health–Midland), Jeffrey Smith,MD (Gratiot, MidMichigan Health–Midland), John Webber, MD (Harper University Hospital, Crittenton Hospital and Medical Center), Scott Laker, MD (Henry Ford Health System, Huron Valley Sinai), Carl Pesta, DO (St John Oakland, McLaren Macomb), Keith Marshall, DO (St John Oakland), Jeffrey Genaw, MD (Henry Ford Health System), Arthur Carlin, MD (Henry Ford Health System), Steven Hendrick, MD (Harper University Hospital), James Wagner, MD (Beaumont Oakwood), Jamal Farhan, MD (Hurley Medical Center), Michael Nizzi, DO (Munson Medical Center), Roche Featherstone, MD (Munson Medical Center), Steven Slikkers, MD (Munson Medical Center), Zubin Bhesania, MD (McLarenPort Huron, Lake Huron Medical Center), Anthony Boutt, MD (McLaren Port Huron, Lake Huron Medical Center), Michael Schuhknecht, DO (Lakeland Community Hospital), Gary Katz, DO (St John Oakland, St John Providence), Abdelkader Hawasli, MD (St John Hospital and Medical Center, Harper University Hospital, St John Oakland, Beumont Grosse Pointe), Tallal Zeni, MD (St Mary Mercy, St Joseph Mercy Oakland), Jonathan Finks, MD (University of Michigan Health System), Kevin Krause, MD (Beaumont Royal Oak, Beaumont Troy), David Chengelis, MD (Beaumont Royal Oak, Beaumont Troy), Kerry Kole, DO (St John Oakland, St John Providence, Beaumont Troy), Ahmed Meguid, MD (St John Hospital and Medical Center, Harper, St John Oakland), Michael Wood, MD (Harper University Hospital), Jamokay Taylor, MD (Harper University Hospital, Forest Health Medical Center), Andre Nunn, MD (Harper University Hospital, Forest Health Medical Center), Mubashir Sabir, MD (St John Oakland), Matthew Weiner, MD (Harper University Hospital, Huron Valley Sinai), Kurt Kralovich, MD (Beaumont Oakwood), Harris Dabideen, MD (McLaren Flint), Michael Kia, DO (McLaren Flint), Brian Gluck, DO (Mercy Health Partners), Stuart Verseman, MD (Borgess Medical Center, Bronson Methodist Hospital), Randal Baker, MD (Spectrum Health System, Mercy Health Saint Mary’s and McLaren Northern Michigan), Paul Kemmeter, MD (Spectrum Health System, Mercy Health Saint Mary’s and McLaren Northern Michigan), James Foote, MD (Spectrum Health System, North Ottawa Community Hospital, Mercy Health Saint Mary’s), Jacob Roberts, DO (St Mary’s Livonia, St Joseph Mercy Oakland), David Edelman, MD (Harper University Hospital), Daniel Bacal, MD (Beaumont Oakwood), Edward Mavashev, MD (Beaumont Oakwood), Derek Nagle, MD (Spectrum Health System, North Ottawa Community Hospital, Mercy Health Saint Mary’s, Northern Michigan), Jeffrey Bonacci, MD (Gratiot Medical Center, MidMichigan Health–Midland), Mindy Lane, DO (Sparrow Health System), Justin Dimick, MD (University of Michigan Health System), Bilal Kharbutli, MD (Henry Ford Wyandotte), Kibwe Weaver, MD (Borgess Medical Center, Bronson Methodist Hospital), Chad Ringley, MD (Covenant HealthCare), Oliver Varban, MD (University of Michigan Health System), Taghreed Almahmeed, MD (St John Hospital and Medical Center), Karen McFarlane, MD (McLaren Port Huron Hospital), Eyad Wohaibi, MD (Covenant HealthCare), Kosisochi M. Obinwanne, MD (Sparrow Health System), Kerianne Holman, MD (Spectrum Health System, Spectrum Health Zeeland), Erina Kansaker, MD (McLaren Port Huron), Vasanth Stalin, MD (St Mary’s Saginaw), James Adair, MD (War Memorial Hospital), Ashley N. Sachtleben DO (Munson Medical Center), Kathryn Ziegler, MD (Beaumont Royal Oak), Mark Jonker, MD (St Mary’s Livonia), and Alain Elian, MD (Borgess Medical Center).

Disclaimer: The views expressed herein do not necessarily represent the views of the US government. Dr Dimick is the Surgical Innovation Editor of JAMA Surgery but was not involved in the editorial review or the decision to accept the manuscript for publication.

Additional Contributions: Jyothi Thumma, MPH, provided assistance with data management and statistical analysis. She is a salaried employee of the Michigan Center for Healthcare Outcomes and Policy.

References
1.
Hawn  MT, Vick  CC, Richman  J,  et al.  Surgical site infection prevention: time to move beyond the surgical care improvement program.  Ann Surg. 2011;254(3):494-499.PubMedArticle
2.
Haynes  AB, Weiser  TG, Berry  WR,  et al; Safe Surgery Saves Lives Study Group.  A surgical safety checklist to reduce morbidity and mortality in a global population.  N Engl J Med. 2009;360(5):491-499.PubMedArticle
3.
Stulberg  JJ, Delaney  CP, Neuhauser  DV, Aron  DC, Fu  P, Koroukian  SM.  Adherence to surgical care improvement project measures and the association with postoperative infections.  JAMA. 2010;303(24):2479-2485.PubMedArticle
4.
Martin  JA, Regehr  G, Reznick  R,  et al.  Objective Structured Assessment of Technical Skill (OSATS) for surgical residents.  Br J Surg. 1997;84(2):273-278.PubMedArticle
5.
Vassiliou  MC, Feldman  LS, Andrew  CG,  et al.  A global assessment tool for evaluation of intraoperative laparoscopic skills.  Am J Surg. 2005;190(1):107-113.PubMedArticle
6.
Zevin  B, Bonrath  EM, Aggarwal  R, Dedy  NJ, Ahmed  N, Grantcharov  TP; ATLAS group.  Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery.  J Am Coll Surg. 2013;216(5):955-965.e8.PubMedArticle
7.
Birkmeyer  JD, Finks  JF, O’Reilly  A,  et al; Michigan Bariatric Surgery Collaborative.  Surgical skill and complication rates after bariatric surgery.  N Engl J Med. 2013;369(15):1434-1442.PubMedArticle
8.
Birkmeyer  NJ, Share  D, Campbell  DA  Jr, Prager  RL, Moscucci  M, Birkmeyer  JD.  Partnering with payers to improve surgical quality: the Michigan plan.  Surgery. 2005;138(5):815-820.PubMedArticle
9.
Share  DA, Campbell  DA, Birkmeyer  N,  et al.  How a regional collaborative of hospitals and physicians in Michigan cut costs and improved the quality of care.  Health Aff (Millwood). 2011;30(4):636-645.PubMedArticle
10.
Birkmeyer  NJ, Dimick  JB, Share  D,  et al; Michigan Bariatric Surgery Collaborative.  Hospital complication rates with bariatric surgery in Michigan.  JAMA. 2010;304(4):435-442.PubMedArticle
11.
Erickson  P.  Evaluation of a population-based measure of quality of life: the Health and Activity Limitation Index (HALex).  Qual Life Res. 1998;7(2):101-114.PubMedArticle
12.
Livingston  EH, Ko  CY.  Use of the health and activities limitation index as a measure of quality of life in obesity.  Obes Res. 2002;10(8):824-832.PubMedArticle
13.
Finks  JF, Kole  KL, Yenumula  PR,  et al; Michigan Bariatric Surgery Collaborative, from the Center for Healthcare Outcomes and Policy.  Predicting risk for serious complications with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative.  Ann Surg. 2011;254(4):633-640.PubMedArticle
14.
Reznick  RK, MacRae  H.  Teaching surgical skills: changes in the wind.  N Engl J Med. 2006;355(25):2664-2669.PubMedArticle
15.
Livhits  M, Mercado  C, Yermilov  I,  et al.  Exercise following bariatric surgery: systematic review.  Obes Surg. 2010;20(5):657-665.PubMedArticle
16.
Livhits  M, Mercado  C, Yermilov  I,  et al.  Is social support associated with greater weight loss after bariatric surgery? a systematic review.  Obes Rev. 2011;12(2):142-148.PubMedArticle
17.
McMahon  MM, Sarr  MG, Clark  MM,  et al.  Clinical management after bariatric surgery: value of a multidisciplinary approach.  Mayo Clin Proc. 2006;81(10)(suppl):S34-S45.PubMedArticle
18.
American College of Surgeons. Resources for optimal care of the metabolic and Bariatric Surgery patients 2014. https://www.facs.org/quality%20programs/mbsaqip/standards. Published 2014. Accessed March 12, 2016.
19.
Ryan  KK, Tremaroli  V, Clemmensen  C,  et al.  FXR is a molecular target for the effects of vertical sleeve gastrectomy.  Nature. 2014;509(7499):183-188.PubMedArticle
20.
Greenberg  CC, Ghousseini  HN, Pavuluri Quamme  SR, Beasley  HL, Wiegmann  DA.  Surgical coaching for individual performance improvement.  Ann Surg. 2015;261(1):32-34.PubMedArticle
×