The figures show trend lines for risk-adjusted mortality rates for hospitals that did and did not meet the Leapfrog volume standard for each procedure. A, Pancreatic resection. B, Esophageal resection. C, Rectal resection. D, Lung resection. The shaded area about the line represents the 95% CI for the estimates.
A, Proportion of patients undergoing surgery in a US hospital that met the Leapfrog volume standard by year, 2005-2016. B, Proportion of US hospitals performing high-risk cancer surgery that met the standard by year, 2005-2016.
eFigure 1. Overall Risk-Adjusted Outcome Rates by Procedure for all Patients in the Study Cohort.
eFigure 2. Risk-Adjusted Complication Rates by Procedure for Hospitals Meeting and Not Meeting the Volume Standards.
eFigure 3. Regional Availability of Any Hospital Meeting the Leapfrog Volume Standards.
eTable. Risk-Adjusted Outcomes for the Lowest and Highest Volume Hospitals, 2013-2016.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Sheetz KH, Chhabra KR, Smith ME, Dimick JB, Nathan H. Association of Discretionary Hospital Volume Standards for High-risk Cancer Surgery With Patient Outcomes and Access, 2005-2016. JAMA Surg. 2019;154(11):1005–1012. doi:10.1001/jamasurg.2019.3017
What is the association of hospitals’ enforcement of existing discretionary minimum volume standards for high-risk surgical procedures and patient access and outcomes?
This longitudinal population-based study of Medicare claims found that the Leapfrog group’s volume standards did not differentiate outcomes for 3 of 4 high-risk cancer operations. Although an increasing majority of patients had surgery in hospitals meeting the Leapfrog volume standards over time, the overall proportion of hospitals meeting them remains low.
These findings highlight potentially important tradeoffs between setting effective volume thresholds and practical expectations for hospital adherence and patient access to centers that meet those standards.
Various clinical societies and patient advocacy organizations continue to encourage minimum volume standards at hospitals that perform certain high-risk operations. Although many clinicians and quality and safety experts believe this can improve outcomes, the extent to which hospitals have responded to these discretionary standards remains unclear.
To evaluate the association between short-term clinical outcomes and hospitals’ adherence to the Leapfrog Group’s minimum volume standards for high-risk cancer surgery.
Design, Setting, and Participants
Longitudinal cohort study using 100% of the Medicare claims for 516 392 patients undergoing pancreatic, esophageal, rectal, or lung resection for cancer between January 1, 2005, and December 31, 2016. Data were accessed between December 1, 2018, and April 30, 2019.
High-risk cancer surgery in hospitals meeting and not meeting the minimum volume standards.
Main Outcomes and Measures
Patients having surgery in hospitals meeting the volume standard and 30-day and in-hospital mortality and complication rates.
Overall, a total of 516 392 procedures (47 318 pancreatic resections, 29 812 esophageal resections, 116 383 rectal resections, and 322 879 lung resections) were included in the study, and patient mean (SD) age was 73.1 (7.5) years. Outcomes improved over time in both hospitals meeting and not meeting the minimum volume standards. Mortality after pancreatic resection decreased from 5.5% in 2005 to 4.8% in 2016 (P for trend <.001). Mortality after esophageal resection decreased from in 6.7% 2005 to 5.0% in 2016 (P for trend <.001). Mortality after rectal resection decreased from 3.6% in 2005 to 2.7 % in 2016 (P for trend <.001). Mortality after lung resection decreased from 4.2% in 2005 to 2.7 % in 2016 (P for trend <.001). Throughout the study period, there were no statistically significant differences in risk-adjusted mortality between hospitals meeting and not meeting the volume standards for esophageal, lung, and rectal cancer resections. Mortality rates after pancreatic resection were consistently lower at hospitals meeting the volume standard, although mortality at all hospitals decreased over the study period. For example, in 2016, risk-adjusted mortality rates for hospitals meeting the volume standard were 3.8% (95% CI, 3.3%-4.3%) compared with 5.7% (95% CI, 5.1%-6.5%) for hospitals that did not. Although an increasing majority of patients underwent surgery in hospitals meeting the Leapfrog volume standards over time, the overall proportion of hospitals meeting the standards in 2016 ranged from 5.6% for esophageal resection to 23.3% for pancreatic resection.
Conclusions and Relevance
Although volume remains an important factor for patient safety, the Leapfrog Group’s minimum volume standards did not differentiate hospitals based on mortality for 3 of the 4 high-risk cancer operations assessed, and few hospitals were able to meet these standards. These findings highlight important tradeoffs between setting effective volume thresholds and practical expectations for hospital adherence and patient access to centers that meet those standards.
Various clinical societies and patient advocacy organizations, such as the Leapfrog Group, have encouraged minimum-volume standards at hospitals that perform certain high-risk operations for decades. These standards build on an extensive body of research that asserts that patients are safer and have better short- and long-term survival when their surgery is performed by hospitals and surgeons with sufficient experience.1,2 For example, greater hospital volume is associated both with fewer postoperative complications and an 11% increase in long-term survival after resection for pancreatic cancer.3 In recent years, certain health systems have gone a step further by taking a “volume pledge” to eliminate low-volume surgery in their hospitals.4
Although many clinicians and quality and safety experts believe that reducing low-volume surgery will improve outcomes, hospitals in the United States have no legal or financial imperative to do so. Therefore, adherence to existing standards remains discretionary. Hospitals may voluntarily act to reduce low-volume surgery to optimize short-term and long-term outcomes, but there are substantial disincentives that may prevent them from doing so. The decision to restrict surgeons’ practice or even eliminate an entire service line may be detrimental to hospital finances, referral patterns, training, and patient access.5,6 How hospitals manage these competing interests has implications for evaluating the feasibility and effectiveness of policies that enforce volume-based credentialing standards. These standards represent the most common mechanism by which clinical societies, for example, may advocate for greater centralization of complex cancer operations.7 High-risk cancer surgery has become generally safer in the past decade, even in lower-volume centers.8 It is unknown whether historical volume standards are still relevant to patient outcomes or whether a reappraisal of their usefulness is warranted.
In this study, we used 100% of Medicare claims for the study years to evaluate national trends in adherence to the Leapfrog volume standards for 4 high-risk cancer operations. We estimated risk-adjusted short-term outcomes for hospitals meeting and not meeting the minimum volume standards over time. We then compared outcomes across the hospitals with the highest and lowest volumes to evaluate whether differences in volume continue to differentiate hospitals based on perioperative mortality.
This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. We used data from the 100% capture Medicare Provider Analysis and Review (MedPAR) files for January 1, 2005, to December 31, 2016. Data were accessed from December 1, 2018, to April 30, 2019. We collected data on patient age, demographic features, geographic location, and comorbidities. We included only patients aged 65 to 99 years. We identified patients undergoing proctectomy, esophagectomy, pancreatectomy, and lung resection for cancer using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and International Classifications of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes. These procedures are specifically identified in the Leapfrog Group’s high-risk surgery report, which specifies minimal-volume standards for hospitals and surgeons.7 We used the American Hospital Association annual survey to obtain additional hospital characteristics such as bed capacity, nurse to patient ratios, business model, and teaching status. This study was deemed exempt from approval by the institutional review board of the University of Michigan because data examined were deidentified.
The primary outcomes of interest were the proportion of patients undergoing surgery in a hospital meeting the minimum-volume standard, the proportion of hospitals meeting this standard in a given year, and the proportion of hospital referral regions with at least 1 hospital offering each procedure.
Because Medicare covers only a proportion of patients undergoing these operations at US hospitals, we used the National Inpatient Sample to derive more complete estimates of hospital volume. For each procedure and year, we calculated proportions for each payer using the National Inpatient Sample. We then divided the Medicare-only volume from our primary analytic files by the proportion of patients covered by Medicare in the National Inpatient Survey. The mean Medicare proportion was 53% for pancreatic, 49% for rectal, 44% for esophageal, and 56% for lung resection. This calculation should result in a more accurate estimate of hospitals’ procedural volume for a given year and has been previously implemented in similar studies.1
We assessed several secondary outcomes, including postoperative mortality, complications, and total episode spending. We used ICD-9-CM and ICD-10-CM codes to identify 30-day postoperative complications such as pulmonary failure, pneumonia, myocardial infarction, deep venous thrombosis, pulmonary embolism, renal failure, surgical site infection, gastrointestinal bleeding, and postoperative hemorrhage. These complications represent a subset of codes from administrative claims with the greatest sensitivity and specificity.9 We identified mortality as death occurring within 30 days of the index operation. Total episode spending included actual Medicare payments for the index hospitalization, readmissions, and postacute care. To address differences in Medicare spending related to known geographic variation in payments, we price standardized payments using methods previously described.10
We calculated each primary outcome annually across the entire study period. We derived volume standards from the Leapfrog Group’s high-risk surgery report, which specifies annual hospital volumes for esophagectomy (20), lung resection (40), pancreatectomy (20), and proctectomy (16) (Table 1). All proportions reflect the unadjusted number of patients, hospitals, or regions meeting the specified criteria divided by the total number of each in a given year.
We used multilevel mixed-effects logistic regression models to adjust our secondary outcomes for differences in patient and hospital characteristics. For each model, we included 27 Elixhauser comorbidities, age, race, and hospital characteristics (bed capacity, nurse to patient ratio, teaching status, business model) as fixed effects. We included the hospital identifier as a random-effects parameter in each model. Estimates of each binary outcome were generated using marginal means.
We also grouped hospitals into quintiles based on their total annual volume to perform secondary analysis on the most recent years of data (Table 1). Here the goal was to assess the volume-outcome relationship across hospitals that varied more widely in annual volume than those stratified by the Leapfrog standards alone. All models for this analysis were identical to those above, but also included a categorical variable for hospital volume quintiles.
All statistical analyses were performed using Stata statistical software version 14 (StataCorp). We used a 2-sided approach at the 5% significance level for all hypothesis testing.
Of the 516 392 total patients, mean (SD) patient age was 73.1 (7.5) years. Of 47 318 patients undergoing pancreatic resection during the study period, mean (SD) age was 73.1 (7.5) years. Demographic and comorbidity characteristics of patients for each of the 4 procedures are listed in Table 2. Patient age and demographics remained similar across the study period. For all procedures, however, there were significant trends toward higher coded illness severity. The mean (SD) number of comorbidities for patients undergoing pancreatic resection increased from 2.2 (1.2) in 2005 through 2008 to 3.4 (1.8) in 2013 through 2016 (P for trend = .004). The mean (SD) number of comorbidities for patients undergoing esophageal resection increased from 2.0 (1.2) in 2005 through 2008 to 3.2 (1.8) in 2013 through 2016 (P for trend < .001). The mean (SD) number of comorbidities for patients undergoing rectal resection increased from 2.1 (1.3) in 2005 through 2008 to 2.7 (1.9) in 2013 through 2016 (P for trend < .001). The mean (SD) number of comorbidities for patients undergoing lung resection increased from 2.2 (1.3) in 2005 through 2008 to 2.9 (1.7) in 2013 through 2016 (P for trend = .008).
There were trends toward better outcomes for all procedures across the study period (Figure 1 and eFigure 1 A-D in the Supplement). Mortality after pancreatic resection decreased from 5.5% in 2005 to 4.8% in 2016 (P for trend < .001). Mortality after esophageal resection decreased from in 6.7% 2005 to 5.0% in 2016 (P for trend < .001). Mortality after rectal resection decreased from 3.6% in 2005 to 2.7 % in 2016 (P for trend < .001). Mortality after lung resection decreased from 4.2% in 2005 to 2.7 % in 2016 (P for trend < .001). Despite these trends, there were no statistically significant differences in risk-adjusted mortality between hospitals that met the Leapfrog volume standard and those that did not for esophageal, lung, and rectal resections (Figure 1B-D). For example, the mortality rate after esophageal resection in 2016 was 5.2% in hospitals not meeting the volume standard compared with 4.6% in hospitals meeting the volume standard (P for difference = .23). Similarly, the mortality rate after rectal resection in 2016 was 2.9% in hospitals not meeting the volume standard compared with 2.6% in hospitals meeting the volume standard (P for difference = .17). The mortality rate after lung resection in 2016 was 2.7% in hospitals not meeting the volume standard compared with 2.7% in hospitals meeting the volume standard (P for difference = .44). For pancreatic resection, however, differences between these groups of hospitals were sustained throughout the entire study period. For example, in 2016, the risk-adjusted mortality rate for hospitals meeting the Leapfrog volume standard was 3.8% (95% CI, 3.3%-4.3%) compared with 5.7% (95% CI, 5.1%-6.5%) for hospitals that did not meet the volume standard (Figure 1A). Analyses focusing on the incidence of complications also showed continued trends toward better outcomes but no differences between hospitals meeting and not meeting the Leapfrog volume standard for any procedure (eFigure 2A-D in the Supplement). Complications after pancreatic resection decreased from 32.1% in 2005 to 29.5% in 2016 (P for trend < .001). Complications after esophageal resection decreased from in 38.3% 2005 to 37.1% in 2016 (P for trend = .03). Complications after rectal resection decreased from 21.7% in 2005 to 19.6% in 2016 (P for trend < .001). Complications after lung resection decreased from 18.5% in 2005 to 16.0% in 2016 (P for trend = .002). The complication rate after esophageal resection in 2016 was 38.4% in hospitals not meeting the volume standard compared with 35.0% in hospitals meeting the volume standard (P for difference = .07). Similarly, the complication rate after rectal resection in 2016 was 20.2% in hospitals not meeting the volume standard compared with 19.2% in hospitals meeting the volume standard (P for difference = .31). The complication rate after lung resection in 2016 was 16.5% in hospitals not meeting the volume standard compared with 15.6% in hospitals meeting the volume standard (P for difference = .11).
Postoperative outcomes for hospitals in the highest and lowest volume quintiles are displayed in Table 3. The highest quintile of hospitals had significantly lower postoperative mortality for all procedures. For example, 30-day mortality after pancreatic resection was 3.5% in the highest volume quintile compared with 5.8% in the lowest volume quintile (odds ratio [OR], 0.56; 95% CI, 0.39-0.82). Results were similar for overall rates of postoperative complications (eTable in the Supplement). For example, complication rates after pancreatic resection were 29.1% in the highest volume quintile compared with 35.9% in the lowest volume quintile (OR, 0.69; 95% CI, 0.55-0.85). Similarly, complication rates after rectal resection were 19.8% in the highest volume quintile compared with 22.6% in the lowest volume quintile (OR, 0.81; 95% CI, 0.71-0.92).
The proportion of patients undergoing surgery in a hospital meeting the Leapfrog volume standard increased over time for pancreatic, lung, and rectal resections (Figure 2A). For example, in 2005, 46.7% of patients (2473 of 5296) who underwent a pancreatic resection had the procedure in a hospital meeting the volume standard. That proportion increased to 69.1% (7062 of 10 220) by 2016. The same trend was not observed for esophageal resection, where 31.0% (1617 of 5219) of patients had surgery in a hospital meeting the volume standard in 2005 compared with 31.1% (1723 of 5541) in 2016.
With the exception of esophageal resection, the proportion of hospitals meeting the Leapfrog volume standards increased over time. The proportion of hospitals meeting the volume standard for lung resection increased from 14.8% in 2005 to 22.9% in 2016 (Figure 2B). The regional availability of any hospital meeting the Leapfrog volume standard for each high-risk cancer operation varied widely (eFigure 3 in the Supplement). In the most recent year of the study period, 2016, 13.3% of hospital referral regions had at least 1 hospital that met the standard for esophageal resection compared with 59.8% of referral regions that offered at least 1 hospital meeting the volume standard for rectal resection.
In this longitudinal population-based study, 30-day mortality for 4 high-risk cancer operations improved over time. With the exception of pancreatic resection, the Leapfrog Group’s volume standards did not differentiate hospitals based on the incidence of postoperative mortality or complications. When stratifying hospitals based on greater differences in volume than those stratified by the Leapfrog standards, however, higher-volume hospitals continued to have lower mortality for all procedures. An increasing majority of patients underwent surgery in hospitals meeting the Leapfrog volume standard over time, suggesting trends toward greater centralization of care. That said, the overall proportion of hospitals meeting minimum volume standards remained low. These findings highlight specific tradeoffs related to setting volume standards high enough to effectively differentiate outcomes but low enough to be met by a reasonable number of hospitals that perform high-risk cancer surgery.
The motivations behind establishing volume criteria for certain high-risk operations are clear. Numerous studies have previously demonstrated greater surgical mortality in hospitals with less experience performing certain operations.1,2,11 Earlier reports on the association of volume to outcome in cancer surgery, using data from the 1990s, showed significantly higher mortality rates than more recent work, including our study.12-14 Although more current analyses have demonstrated that volume is still an important determinant of outcomes, the differences between hospitals deemed high-volume or low-volume have diminished over time.8,15,16 Recognition of the association of volume with outcomes has contributed to greater interest in centralizing complex surgical care in the United States and around the world.3,6,7,16 For example, earlier work showed that ad hoc centralization of pancreatic resection in the United States (not mandated by government policy, as is common in Europe) contributed to 49% of the decline in mortality between 1999 and 2008.8 The same study also suggested that centralization did not explain improvements in postoperative mortality rates after lung resection for cancer. Our study expands on previous work by demonstrating how hospitals have responded to well-publicized volume standards and the broader discussion on centralization to shape modern-day practice patterns for high-risk cancer surgery.
As complex surgical care becomes gradually safer overall, mortality continues to decrease for all major surgical procedures, thereby narrowing disparities between hospitals.15 Our comparison of the Leapfrog standard suggests that some lower-volume hospitals may be able to capitalize on their cumulative experience and provide safer care despite low volumes in a given year. This argument suggests that the influence of volume is diminishing over time. But it would be imprecise to state that volume is no longer an important determinant of outcomes. Higher volume thresholds will differentiate high-volume from high-quality (low mortality) hospitals. The inherent tradeoff is that with higher volume thresholds, far fewer hospitals will be able meet the minimum standard, and fewer patients will have access to the most experienced centers. For organizations such as the Leapfrog Group, this tradeoff would not align with their mission to broadly advocate for patient safety. In other words, it would be impractical for clinical societies and patient advocacy groups to promote sweeping safety standards attainable by only a small proportion of hospitals.
Our study should be interpreted within the context of several important limitations. The use of administrative claims data may have limited our ability to adequately adjust for risk. That said, outcome rates from this study are consistent with those previously published and inadequate risk adjustment would generally bias results toward worse outcomes at higher-volume referral centers that care for the patients with the most complex needs. That said, it is also possible that higher-risk patients have shifted to certain centers over time in ways that cannot be fully captured with claims data. Some clinicians and quality and safety experts may also be concerned that adherence to Leapfrog standards is not mandatory and hospitals are not required to demonstrate specific experience levels to perform high-risk cancer operations. Although this is accurate, and any debate over the validity of specific volume thresholds is fair, our analysis was not designed to make a specific critique on the Leapfrog group or its standards. Finally, it is possible that our volume estimates are inaccurate. However, we attempted to estimate procedural volumes in the most robust way possible—using multiple databases and 100% claims files—and therefore captured cases across all practice settings in the United States. It is also possible that payer mix differs by hospital volume, but our assessment of this potential issue did not show significant differences based on hospital volume. Some clinicians and quality and safety experts may be particularly concerned with our findings that esophageal resection did not exhibit a pattern of centralization from 2005 to 2016. However, research on the centralization of this procedure in the decade leading up to our study showed similar results.8
Patient advocacy groups, clinical societies, and various quality improvement organizations all have a powerful platform to signal priorities in quality to health care consumers and policy makers. Our data suggest that these efforts have been successful. Not only has patient safety continued to improve over time for these high-risk operations, the health care community is increasingly cognizant of the importance of hospitals’ volume and experience for delivering quality care. Depending on the outcomes in question, however, a more nuanced approach may now be necessary. Avoiding very low-volume surgery (ie, 1-2 complex cases per year) in hospitals should be a national priority. For some procedures, though, meaningful improvements in postoperative mortality may no longer stand as the most relevant quality and safety imperative. In certain health care markets, it may be less helpful to continue promoting volume standards that are either unattainable or unable to differentiate hospitals based on short-term outcomes. In these circumstances, it may be a higher-yield approach to shift our focus and quality reporting efforts toward guideline-compliant care (eg, appropriate neoadjuvant chemotherapy) or long-term survival.17,18 For example, a recent analysis identified 22-fold variation in adjuvant therapy rates among patients with locally advanced rectal cancer despite a demonstrable survival benefit.19
The following efforts would better align with how many clinical specialties are approaching modern-day quality improvement: focusing on a comprehensive set of structural, process, and outcome metrics that reflect the interdisciplinary aspects of trauma or oncologic care, for example.20-23 It would also promote great discussion around other important aspects of oncologic care, such as access to clinical trials and the appropriateness of care. These changes can and should reflect patients’ preferences with respect to cancer care and surgery in particular. For example, a recent national survey suggests that some, but not all, patients would be willing to travel greater distances for safer surgery.24 Other patients would opt for a more familiar hospital even if it meant a higher risk of complications. Within this context, physicians can work more closely with payers who seek to create optimal provider networks around complex care. Through these efforts we acknowledge that although volume continues to be an important determinant of outcomes for certain high-risk procedures, drawing attention to the systems of care that are longitudinally associated with outcomes in patients with cancer may be the new most important patient safety issue.
Results of the study demonstrated that the Leapfrog Group’s minimum volume standards did not differentiate hospitals based on outcomes for 3 of the 4 high-risk cancer operations. Nonetheless, few hospitals were able to meet these relatively modest volume standards. These findings highlight important tradeoffs between establishing effective volume thresholds and practical expectations for hospital adherence and, potentially, patient access to centers that meet those standards. In addition to structural metrics such as volume standards, clinical societies and patient safety organizations may consider prioritizing different process and outcome measures that not only reduce harm but also improve the longitudinal care of patients with complex clinical problems such as cancer.
Accepted for Publication: May 27, 2019.
Corresponding Author: Kyle H. Sheetz, MD, MSc, Center for Healthcare Outcomes and Policy, Institute for Healthcare Policy and Innovation, University of Michigan, 2800 Plymouth Rd, NCRC B016, Room 100N-11, Ann Arbor, MI 48109 (email@example.com).
Published Online: August 14, 2019. doi:10.1001/jamasurg.2019.3017
Author Contributions: Drs Sheetz and Nathan had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Sheetz, Smith, Nathan.
Drafting of the manuscript: Sheetz, Chhabra, Dimick.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Sheetz.
Administrative, technical, or material support: Dimick, Nathan.
Supervision: Dimick, Nathan.
Conflict of Interest Disclosures: Dr Dimick reported receiving personal fees from ArborMetrix outside the submitted work. No other disclosures were reported.
Funding/Support: This study was supported by the Association for Academic Surgery Resident Research Fellowship Agency for Healthcare Research and Quality (AHRQ) grant 2T32HS000053-27 (Dr Sheetz); Institute for Healthcare Policy and Innovation Clinician Scholars Program grant AHRQ T32HS000053 and the National Institutes of Health (NIH) Division of Loan Repayment (Dr Chhabra); National Institutes of Health/National Institute on Aging (NIH/NIA) grant R01AG039434, AHRQ grant R01HS023597 (Dr Dimick); and AHRQ grant K08HS024763-01, NIH/NIA R01AG039434 (Dr Nathan).
Role of the Funder/Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: Dr Dimick is the Surgical Innovation Editor for JAMA Surgery but was not involved in any of the decisions regarding review of the manuscript or its acceptance.
Create a personal account or sign in to: