The numbers at the bottom represent the number of surgical procedures among patients 65 years and older identified with at least 1% inpatient mortality in the Healthcare Cost and Utilization Project Nationwide Inpatient Sample (HCUP NIS) in eTables 1, 2, 3, and 4 in the Supplement. CCS indicates Clinical Classifications Software; PHC4, Pennsylvania Health Care Cost Containment Council.
Shown is the distribution of mortality estimates for the identified procedures across subgroups varying by age and admission acuity. Each box corresponds to the interquartile range of the data, the 25th or 75th percentiles, and the black line in the middle is the median. The line above the box extends to the largest data point less than or equal to the 75th percentile plus 1.5 times the interquartile range. The lower line is formed analogously. Observations beyond this are individually plotted.
eTable 1. Collection of 227 Operations Identified in the NIS as High Risk for Patients Age 65 and Older
eTable 2. Collection of 224 Operations Identified in the NIS as High Risk for Patients Age 65 and Older for Emergent or Urgent Admissions
eTable 3. Collection of 163 Operations Identified in the NIS as High Risk for Patients Age 65 and Older for Elective Admissions
eTable 4. Collection of 154 Operations Identified in the NIS as High Risk for Patients Age Less Than 65
Customize your JAMA Network experience by selecting one or more topics from the list below.
Schwarze ML, Barnato AE, Rathouz PJ, et al. Development of a List of High-Risk Operations for Patients 65 Years and Older. JAMA Surg. 2015;150(4):325–331. doi:10.1001/jamasurg.2014.1819
No consensus exists regarding the definition of high-risk surgery in older adults. An inclusive and precise definition of high-risk surgery may be useful for surgeons, patients, researchers, and hospitals.
To develop a list of high-risk operations.
Design, Setting, and Participants
Retrospective cohort study and modified Delphi procedure. The setting included all Pennsylvania acute care hospitals (Pennsylvania Health Care Cost Containment Council [PHC4] April 1, 2001, to December 31, 2007) and a nationally representative sample of US acute care hospitals (Nationwide Inpatient Sample [NIS], Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality January 1, 2001, to December 31, 2006). Patients included were those 65 years and older admitted to PHC4 hospitals and those 18 years and older admitted to NIS hospitals. We identified International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) procedure codes associated with at least 1% inpatient mortality in the PHC4. We used a modified Delphi procedure with 5 board-certified surgeons to further refine this list by excluding nonoperative procedures and operations that were unlikely to be the proximate cause of mortality and were instead a marker of critical illness (eg, tracheostomy). We then cross-validated this list of ICD-9-CM codes in the NIS.
Main Outcomes and Measures
Modified Delphi procedure consensus of at least 4 of 5 panelists and proportion agreement in the NIS.
Among 4 739 522 admissions of patients 65 years and older in the PHC4, a total of 2 569 589 involved a procedure, encompassing 2853 unique procedures. Of 1130 procedures associated with a crude inpatient mortality of at least 1%, 264 achieved consensus as high-risk operations by the modified Delphi procedure. The observed inpatient mortality in the NIS was at least 1% for 227 of 264 procedures (86%) in patients 65 years and older. The pooled inpatient mortality for these identified high-risk procedures performed on patients 65 years and older was double the pooled inpatient mortality for correspondingly identified high-risk operations for patients younger than 65 years (6% vs 3%).
Conclusions and Relevance
We developed a list of procedure codes to identify high-risk surgical procedures in claims data. This list of high-risk operations can be used to standardize the definition of high-risk surgery in quality and outcomes–based studies and to design targeted clinical interventions.
High-risk surgery is not well defined, but surgeons know it when they see it. Surgery can be high risk owing to patient-specific factors or operation-specific factors.1,2 However, teasing out these commingled contributors can be challenging. There is little debate that open repair of an abdominal aortic aneurysm is high-risk surgery. However, this operation is almost exclusively performed on older patients, most of whom have preexisting cardiovascular disease or risk factors for vascular disease. As such, the operation is high risk partly owing to the characteristics of the patients on whom it is routinely performed. Nonetheless, the operation itself has inherent risks given the need for laparotomy and the cardiac stress engendered by aortic cross-clamping.
Some investigators3-5 have characterized high-risk surgery by identifying operations that are associated with statistically significant inpatient mortality. Although these lists identify operations that a surgeon might characterize as high risk, the collection of procedures is contaminated by surgery associated with caring for patients with critical illness such as tracheostomy, ventriculostomy, and wound debridement. Other investigators2,6 have focused more on patient factors, attempting to identify a high-risk group of patients who have any surgical procedure. Using another approach, Birkmeyer and colleagues7,8 have examined surgical quality and safety for more than 15 years using a specific group of major cardiovascular and cancer operations with high operative morbidity or mortality (abdominal aortic aneurysm repair, carotid endarterectomy, coronary artery bypass grafting, aortic valve repair, pancreatectomy, esophagectomy, gastrectomy, and lung resection). This strategy more precisely identifies high-risk surgery and covers 54 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes and 344 766 operations performed annually.9,10 However, the list is limited, excluding many operations that are typically considered high risk such as thoracic aneurysm repair, organ transplantation, and neurosurgical procedures. Furthermore, the list contains procedures that are primarily performed electively. No general consensus exists about a broader definition of high-risk surgery.
Quiz Ref IDAn inclusive and precise definition of high-risk surgery may be useful for multiple purposes. First, surgeons can use this information to characterize the nature of a proposed operation with patients and their families. Second, researchers can use this information to evaluate trends, outliers, and successful treatment of postoperative complications (rescue). Third, hospitals can use this information to estimate procedure-specific mortality based on patient age and admission acuity. We sought to expand the list of high-risk operations beyond the historically used list of elective major cardiovascular and oncological operations for patients 65 years and older. We then evaluated this more inclusive list using a second data set to determine the boundaries of our confidence in this list and used patient age and admission acuity to examine how this list might change with these variables.
Authors from the University of Pittsburgh conducted Pennsylvania Health Care Cost Containment Council (PHC4) analysis under a data use agreement with the PHC4, with approval of the University of Pittsburgh Institutional Review Board. Authors from the University of Wisconsin conducted Nationwide Inpatient Sample (NIS) analysis under a data use agreement with the Agency for Healthcare Research and Quality, with approval of the University of Wisconsin Institutional Review Board. Informed consent was not needed to use these deidentified data sets.
Quiz Ref IDTo develop a list of high-risk procedures, we first identified operations associated with at least 1% mortality in a locally available database. We then sorted these procedures into high-risk operations and non–high-risk operations using a modified Delphi procedure to identify procedures that were the likely proximate cause of mortality and not simply associated with high mortality. We then determined the level of confidence that the procedures in this group had a mortality of at least 1% with a second data set (Figure 1).
We chose a threshold of 1% mortality to categorize operations as high risk because operative risk has historically been equated to operative death11,12 and because in the contemporary era at least 1% mortality signifies substantial operative risk.10 We used the PHC4 data file, which records all non–Veterans Affairs hospital admissions in Pennsylvania. We sampled all admissions among patients 65 years and older between April 1, 2001, and December 31, 2007, and identified procedures by ICD-9-CM codes associated with a mean crude in-hospital mortality of at least 1%. The PHC4 data comprise a convenience sample similar in size to a 5% sample of Medicare beneficiaries.
To facilitate review of the procedures, we then applied a modification of the Healthcare Cost and Utilization Project (HCUP) Clinical Classifications Software (CCS) to organize these ICD-9-CM surgical codes into cohesive groups. The HCUP CCS incorporates 3900 ICD-9-CM codes from January 1980 to September 2011 and 9000 Current Procedural Terminology procedure codes from January 1992 to January 2011 into 244 clinically meaningful categories.13-15 However, these categorizations were not designed to account for surgical complexity or risk in the groupings and needed to be adapted for our purposes. The CCS categories composed of exclusively nonoperative procedures (eg, home health services, chemotherapy, laboratory, etc) were excluded from review because they were not surgically relevant. Three surgical reviewers (Y.Y.H., C.M.D., and A.C.K.) independently examined the CCS categorization of the remaining 2071 ICD-9-CM procedure codes. We eliminated the CCS-defined categories that were clinically ambiguous, and their surgical ICD-9-CM codes were assigned to existing or newly created categories. For example, category 99 (other operating room gastrointestinal therapeutic procedures) was largely heterogeneous and was eliminated. The ICD-9-CM codes were incorporated into existing categories such as exploratory laparotomy (code 89), and new categories were created for biliary minor (code 99.1), biliary major (code 99.2), pancreatic (code 99.3), liver minor (code 99.4), and liver major (code 99.5). Of 2071 ICD-9-CM procedure codes reviewed, 300 (15%) required adjudication by a board-certified surgeon (C.C.G.) because at least one of the primary surgical reviewers disagreed with the new classification. Our revised classification included 219 original CCS categories that did not contain surgical procedures or that captured surgical complexity and risk well and an additional 50 new categories as described above.
Quiz Ref IDTo eliminate procedures that were markers of the severity of illness (eg, tracheostomy) rather than the proximate cause of mortality, we used a modified Delphi procedure. The first author presented the procedures identified in the PHC4 organized within the new CCS groups to 5 other members of the team, all of whom are fellowship-trained, academic, board-certified surgeons (M.L.S., H.B.N., E.R.W., G.D.K., and C.C.G.) who have been practicing 5 to 10 years. The procedures were then iteratively sorted to separate the procedures that should be included on a list of high-risk operations from those that should not be included on such a list. The specific goal was to include procedures that had both an operative mortality of at least 1% and were likely the proximate cause of the patient’s death for all patients 65 years and older. With each round, brief statements were provided of the reasoning for excluding any procedure from the high-risk surgery list or for including a procedure on the high-risk surgery list that had been excluded in prior iterations. After each modified Delphi procedure round, we collected and collated responses. For each subsequent round, a list of all the procedures was presented to allow the panel to consider their responses in light of the anonymous annotations that described respondents’ reasoning from the previous round. This strategy was used for each subsequent round for a total of 4 rounds, at which time there was minimal change in consensus ratings.
We then calculated the crude inpatient mortality of the procedures meeting consensus in the NIS from the HCUP of the Agency for Healthcare Research and Quality from January 1, 2001, to December 31, 2006. Each year, the NIS captures all admissions for patients 18 years and older in a stratified nationally representative 20% sample of US acute care hospitals.16 Specifically, we identified all admissions with one of the procedures identified in the modified Delphi procedure listed in the principal procedure field and then estimated the survey-weighted population total number of admissions and the crude inpatient mortality proportion (95% CI) for each procedure over the 2001 to 2006 time frame.16 We then transformed estimated proportions to the logit scale and performed 1-sided hypothesis tests for whether inpatient mortality for a given procedure was less than 1% vs at least 1%. We report 1-sided P values from these tests, with the type I error rate set at .05. For our primary analysis, we limited our cohort to patients 65 years and older and then stratified our analysis by admission acuity (emergent or urgent vs elective). Finally, to determine whether this list could be extrapolated to patients younger than 65 years, we repeated the analysis in the cohort of patients younger than 65 years. For each cohort, we estimated the survey-weighted total number of annual procedures performed nationally and the survey-weighted mortality of this pooled group of procedures.
Among 4 739 522 admissions of patients 65 years and older in the PHC4 between April 1, 2001, and December 31, 2007, a total of 2 569 589 involved a procedure, encompassing 2853 unique (surgical and nonsurgical) procedures. There were 1130 distinct ICD-9-CM procedures associated with a mean crude inpatient mortality of at least 1%. These procedures accounted for 40% of the procedures identified during the study period.
The response rate was 100% to all rounds of the modified Delphi procedure. After 4 iterations, there was complete consensus (5 of 5 panelists) for 219 procedures and near-complete consensus (4 of 5 panelists) for 45 additional operations that were believed to be surgical operations (performed in an operating room by a surgeon) and the likely proximate cause of the associated mortality. These 264 operations were used for subsequent analysis. This broad collection of operations includes cardiac and thoracic surgery, as well as neurosurgical, vascular, gastrointestinal, and urological operations.
Among patients 65 years and older in the HCUP NIS between January 1, 2001, and December 31, 2006, there were 832 452 sample admissions for patients 65 years and older with 1 of 264 ICD-9-CM procedure codes identified by our modified Delphi procedure listed as the principal procedure. The crude inpatient mortality in the NIS was statistically at least 1% for 227 of 264 operations (86%) identified by the modified Delphi procedure from the PHC4 sample (eTable 1 in the Supplement). The weighted mortality estimates ranged from 1% (95% CI, 1%-2%; P = .004) for creation of esophagogastric sphincter (ICD-9-CM code 44.66) to 68% (95% CI, 57%-79%; P < .001) for implantation of an external ventricular assist device (ICD-9-CM code 37.65). Of 37 procedures for which mortality was not statistically at least 1% in the NIS sample, 13 procedures did not occur in the NIS data set during the study period, suggesting that these procedures are rarely performed or that the specific procedure code is rarely used in patients 65 years and older. The remaining 24 procedure codes were identified in the NIS, but the 1-sided P value was greater than .05 (ie, the mortality was not reliably ≥1%). We confirmed that 37 of 44 procedures where there was a 4 of 5 consensus with the modified Delphi procedure (as opposed to 5 of 5 consensus) had a mortality of at least 1% in the NIS.
Using the weighted national estimate, this final group of 227 ICD-9-CM operative procedure codes accounts for 4 019 773 operations performed on patients 65 years and older between 2001 and 2006, or 669 962 primary operations per year. For the 6-year study period, the estimated numbers of operations identified for each major category were 165 148 urological, 60 439 hepatobiliary, 1 341 757 nonhepatobiliary gastrointestinal (including esophageal), 736 042 noncardiac vascular, 1 332 626 cardiac, 163 841 thoracic, and 192 750 neurosurgical.
To determine the effect of admission acuity on the high-risk procedure list, we examined 264 procedures identified by the modified Delphi procedure according to admission acuity. The observed inpatient mortality was significantly greater than or equal to 1% for patients 65 years and older during emergent or urgent admissions for 224 of 264 procedures identified by the modified Delphi procedure in the PHC4 (eTable 2 in the Supplement). Two hundred fourteen of these procedures were included in the collection of 227 procedures identified as statistically significantly high risk for all types of admissions in patients 65 years and older. Ten ICD-9-CM procedures were uniquely high risk for patients 65 years and older during emergent or urgent admissions only, including unilateral adrenalectomy (code 7.22), endarterectomy (code 38.10), arm vessel replacement (code 38.43), abdominal vein resection or replacement (code 38.47), occlusion of leg vein (code 38.89), high gastric bypass (code 44.31), proctopexy (code 48.76), bile duct excision (code 51.69), transureteroureterostomy (code 56.75), and repair of ureter (code 56.89). Using weighted national estimates, there were 1 646 535 high-risk operations performed on patients 65 years and older during emergent or urgent admissions, accounting for 45% of these procedures performed across all admission types.
The observed inpatient mortality was statistically at least 1% for patients 65 years and older during elective admission for 163 of 264 procedures identified in the PHC4 (eTable 3 in the Supplement). Using the weighted national estimate, these procedures accounted for 1 812 512 operations in patients 65 years and older over the 6-year study period. Quiz Ref IDThe pooled in-hospital mortality across this group of procedures performed during elective admissions was 3% compared with the pooled inpatient mortality of 10% for 224 procedures in the emergent or urgent setting.Figure 2 shows the distribution of mortality estimates across all identified procedures and across the subgroups of procedures restricted by admission acuity.
In patients younger than 65 years, we identified 934 673 operations in the 20% sample of the NIS associated with the final 264 procedure codes determined by our modified Delphi procedure. The observed inpatient mortality for patients younger than 65 years in the NIS for operations was statistically at least 1% for 154 of 264 procedures (58%). Quiz Ref IDThe pooled in-patient mortality for these procedures in younger patients was 3%, statistically lower than the 6% pooled inpatient mortality for high-risk operations for patients 65 years and older across all admissions. Using the weighted national estimates, approximately 2 709 026 operations were performed for these 154 procedure codes in patients younger than 65 years during the 2001 to 2006 study period. This translates to 451 504 high-risk operations annually on patients younger than 65 years. When examining the NIS population as a whole, younger patients account for only about half (49%) of the procedures performed for these selected 154 high-risk operations (eTable 4 in the Supplement).
Using 2 population-based cohorts and a modified Delphi procedure, we identified a list of 227 high-risk operations in patients 65 years and older. We found that older patients are recipients of more than 650 000 of these procedures annually. The pooled inpatient mortality for high-risk procedures performed on patients 65 years and older was double the pooled inpatient mortality for the procedures on this list with a mortality of at least 1% for patients younger than 65 years (6% vs 3%).
Our results have importance for surgeons, researchers, policy makers, and patients. For surgeons, conceptualizing an operation as high risk, particularly for an older patient, can be used to assist the decision-making calculus. By expanding this characterization beyond 8 to 14 primary procedures and excluding operations that are simply markers of critical illness and unlikely to be the proximate cause of mortality, our inclusive list defines a large cohort of operations for which the risks and benefits of surgery should be carefully evaluated to assure that trade-offs between surgical treatment and nonsurgical options have been fully considered. For some operations on this list (eg, lung transplantation and aortic valve replacement), such considerations are readily apparent. However, for other procedures (eg, open right hemicolectomy, which carries a 2% inpatient mortality for patients ≥65 years during an elective admission), recognition and classification of this operation as high risk may prompt more careful deliberation and enhance consideration of nonsurgical options. Although this list does not precisely link specific risk factors with a defined subset of isolated surgical complications, it characterizes high-risk surgery and yields a group of procedures that should trigger more consideration in terms of the risks and benefits, particularly for patients 65 years and older.
For researchers and policy makers, expansion of the list of high-risk operations adds to the methodological capacity to study surgical safety and quality and judge the effects of interventional strategies for a large number of operations that carry considerable surgical mortality. This robust collection of operations with statistically significant risk can be used to define a larger cohort of at-risk patients to assess the effects of quality and safety improvement efforts targeted at surgical outcomes in general (for example prehabilitation or readmission reduction programs).
For patients 65 years and older, the designation of an operation as high risk may present an opportunity for the patient to pause and consider the value of surgery or alternative treatments or prepare for the real potential of an unwanted outcome. Although mortality statistics should be accurately and specifically disclosed to patients before surgery, interpretation of numerical information is often difficult for patients to assimilate,17,18 and patients may struggle to associate the quantitative description of a 1% or 2% operative mortality with an understanding that the operation is high risk. Notably, during all types of admissions, the number of operations considered high risk for patients 65 years and older is considerably larger than the number of high-risk operations for younger patients (227 vs 154), and the pooled mortality is twice as high for older patients. Although these estimates of mortality are not stratified by comorbid conditions or by the less easily ascertained but likely more relevant designation of frailty,19,20 the widespread use of surgery in older Americans demonstrated in this cohort, as well as the substantially higher mortality associated with these procedures, suggests the statistically significant relevance of age as a contributor to surgical outcomes.
Our findings have important strengths and limitations. By using 2 comprehensive inpatient discharge data sources, we were able to assess the results of our modified Delphi procedure and increase the precision of the mortality estimates. However, administrative data sets are vulnerable to coding errors,21 and both data sets capture in-hospital mortality only, which does not capture all mortality in the early postoperative period. Although we used current data available at the time of our study, these data will likely change over time with improvements in clinical care and will need to be reconsidered in light of the International Statistical Classification of Diseases, Tenth Revision. Furthermore, our analysis was restricted to patient age and admission acuity to describe factors associated with operative mortality. We did not specifically risk stratify patients by known single-organ system risk predictors22-24 or physician-determined risk scores (eg, the American Society of Anesthesiologists classification25,26). Also, measures of clinical frailty, a potentially more accurate risk predictor,19,27 are not readily accessible among the discharge data used in this study. As such, our list provides a general notion of operative intrinsic risk, and additional risk calculators (eg, the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator)28 designed for precise risk prediction are necessary to tailor individual risk discussions. Our results suggest that caution should be used in applying this list to patients younger than 65 years because the list is more limited (only 154 procedures) but indicate that extrapolation may be useful in certain circumstances. Furthermore, our strategy was to develop the list for patients 65 years and older and then examine this list for patients younger than 65 years. There may be some procedures for patients younger than 65 years that were not picked up by this strategy such that the list for younger patients is incomplete. Although we separated operations performed during elective admissions from those performed during emergent or urgent admissions, this designation does not allow us to account for operations that were performed emergently during elective admissions. As such, the elective surgery list most likely includes some operations that were performed emergently. Finally, although our modified Delphi procedure was anonymous, respondents may have perceived pressure to conform to the group process,29,30 which could potentially overstate the degree of consensus that we achieved.
We developed a list of 227 operations that carry at least 1% operative mortality for patients 65 years and older. These high-risk operations are performed on more than half a million patients 65 years and older annually. Our results provide a standard to define high-risk surgery that can be used in clinical and research settings.
Accepted for Publication: July 3, 2014.
Corresponding Author: Margaret L. Schwarze, MD, Department of Surgery, University of Wisconsin Hospital and Clinics, 600 Highland Ave, Room G5/315, Clinical Science Center, Madison, WI 53792 (email@example.com).
Published Online: February 18, 2015. doi:10.1001/jamasurg.2014.1819.
Author Contributions: Dr Schwarze had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Schwarze, Barnato, Dodgion, Kwok, Greenberg.
Acquisition, analysis, or interpretation of data: Barnato, Rathouz, Zhao, Neuman, Winslow, Kennedy, Hu, Greenberg.
Drafting of the manuscript: Schwarze, Rathouz.
Critical revision of the manuscript for important intellectual content: Barnato, Rathouz, Zhao, Neuman, Winslow, Kennedy, Hu, Dodgion, Kwok, Greenberg.
Statistical analysis: Schwarze, Rathouz, Zhao, Hu, Greenberg.
Administrative, technical, or material support: Schwarze, Barnato, Dodgion, Greenberg.
Study supervision: Barnato, Greenberg.
Conflict of Interest Disclosures: None reported.
Funding/Support: The project described was supported by the Clinical and Translational Science Awards program, previously through grant 1UL1RR025011 from the National Center for Research Resources and now by grant 9U54TR0000427 from the National Center for Advancing Translational Sciences. Dr Schwarze receives funding from the Greenwall Faculty Scholars Program. Dr Barnato receives funding by grant R01AG035112 from the National Institutes of Health.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The authors conducted the analysis under a data use agreement with the Pennsylvania Health Care Cost Containment Council (PHC4). The following statement is provided and required by the PHC4: PHC4 has provided this data in an effort to further PHC4’s mission of educating the public and containing health care costs in Pennsylvania. PHC4, its agents and staff, have made no representation, guarantee, or warranty, expressed or implied, that the data—financial, patient, payor and physician specific information—are error-free, or that the use of the data will avoid differences of opinion or interpretation, or disputes with those who use published reports or purchased data. PHC4, its agents and staff, will bear no responsibility or liability for the results of the analysis, or consequences of its use.
Additional Contributions: The Healthcare Cost and Utilization Project Data Partners contributed to the Nationwide Inpatient Sample and the state organizations involved (http://www.hcup-us.ahrq.gov/db/hcupdatapartners.jsp).