Figure 1. Comparison of risk-adjusted mortality after trauma and emergency and elective general surgery procedures among hospitals participating within the National Trauma Data Bank and the American College of Surgeons National Surgical Quality Improvement Program. To facilitate the comparison of individual hospital performance across trauma and emergency and elective general surgery procedures, hospitals have been ordered according to the observed to expected ratio (O/E) for trauma surgery in the emergency and elective general surgery figures. The high and low outlier legend in this figure refers to the color rendition. Outlier status, however, is partially redundant with location; low outliers tend toward the left portion of the plots and high outliers tend toward the right.
Figure 2. Comparison of risk-adjusted serious morbidity after trauma and emergency and elective general surgery procedures among hospitals participating within the National Trauma Data Bank and the American College of Surgeons National Surgical Quality Improvement Program. To facilitate the comparison of individual hospital performance across trauma and emergency and elective general surgery procedures, hospitals have been ordered according to the observed to expected ratio (O/E) for trauma surgery in the emergency and elective general surgery figures. The high and low outlier legend in this figure refers to the color rendition. Outlier status, however, is partially redundant with location; low outliers tend toward the left portion of the plots and high outliers tend toward the right.
Ingraham AM, Haas B, Cohen ME, Ko CY, Nathens AB. Comparison of hospital performance in trauma vs emergency and elective general surgery: implications for acute care surgery quality improvement. Arch Surg. Published online March 19, 2012. doi:10.1001/archsurg.2012.71.
eTable 1. Predictors of Mortality among Trauma Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality Improvement Program
eTable 2. Predictors of Mortality among Emergency General Surgery Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality Improvement Program
eTable 3. Predictors of Mortality among Elective General Surgery Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality Improvement Program
eTable 4. Predictors of Serious Morbidity among Trauma Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality Improvement Program
eTable 5. Predictors of Serious Morbidity among Emergency General Surgery Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality
eTable 6. Predictors of Serious Morbidity among Elective General Surgery Patients Treated at Hospitals Contributing to the American College of Surgeons National Trauma Data Bank and National Surgical Quality Improvement Program
This supplementary material has been provided by the authors to give readers additional information about their work.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Ingraham AM, Haas B, Cohen ME, Ko CY, Nathens AB. Comparison of Hospital Performance in Trauma vs Emergency and Elective General Surgery: Implications for Acute Care Surgery Quality Improvement. Arch Surg. 2012;147(7):591–598. doi:10.1001/archsurg.2012.71
Author Affiliations: Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois (Drs Ingraham, Cohen, and Ko); Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, Ohio (Dr Ingraham); Division of Surgery and Trauma, Keenan Research Centre in the Li Ka Shing Knowledge, Institute of St. Michael's Hospital, Toronto, Ontario, Canada (Drs Haas and Nathens); and Department of Surgery, University of California at Los Angeles and VA Greater Los Angeles Healthcare System, Los Angeles, California (Dr Ko).
Hypotheses As emergency general surgery (EMGS) and trauma care are increasingly being provided by the same personnel with overlapping resources, we postulated that the quality of care provided to EMGS and trauma patients would be similar. We also evaluated the relationship between trauma and elective general surgery (ELGS) care, believing that performance would be similar across these services as it reflects institutional culture.
Design Retrospective cohort study comparing hospital performance in trauma and EMGS care and in trauma and ELGS care. Regression models for mortality and serious morbidity were constructed for trauma, EMGS, and ELGS hospitals contributing to both the National Trauma Data Bank (2007) and American College of Surgeons National Surgical Quality Improvement Program (2005-2008).
Setting Forty-six hospitals.
Main Outcome Measures Correlations of observed to expected ratios were examined. Outlier status (hospitals with CIs of observed to expected ratios excluding 1.0) was compared using weighted κ.
Results There was no significant relationship between trauma and EMGS mortality (r = −0.01, P = .94; κ = −0.10, P = .61) or between trauma and ELGS mortality (r = 0.23, P = .12; κ = 0.07, P = .62). There was no significant relationship between trauma and EMGS morbidity (r = 0.21, P = .17; κ = 0.04, P = .63) or between trauma and ELGS morbidity (r = 0.16, P = .30; κ = 0.11, P = .37). No hospitals were consistently low or high outliers across all 3 groups.
Conclusions Trauma performance improvement programs are well established compared with those for EMGS. Although EMGS patients use similar structures and processes as trauma patients, there is a lack of correlation between the quality of care provided to trauma and EMGS patients; EMGS should be incorporated into trauma performance improvement programs.
Substantial resources are being devoted toward developing and maintaining programs to measure and improve the quality of care provided to general surgery and trauma patients. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) provides validated, risk-adjusted 30-day morbidity and mortality outcomes after general, vascular, and some subspecialty surgical procedures. This program affords private sector hospitals the ability to conduct detailed, blinded quality comparisons with the other participating hospitals.1 Participation in the ACS NSQIP has been associated with improvement of hospital performance over time.2 Of 118 hospitals that participated in 2006 and 2007, 82% improved in risk-adjusted morbidity and 66% improved in risk-adjusted mortality; performance improvement (PI) was seen at high-performing and poorly performing hospitals.
Additional programs supported by the ACS focus specifically on PI in trauma care. The National Trauma Data Bank (NTDB) is the largest collection of trauma registry data available, containing records from more than 2 million patients in the United States, Canada, and Puerto Rico. Beginning in 2008, the ACS established the Trauma Quality Improvement Program (TQIP) for benchmarking trauma centers.3 The TQIP uses the existing data collection mechanism afforded by the NTDB and ensures data quality through a training program for data abstractors, the National Trauma Data Standard, and data quality assessments.
Although these quality improvement programs are relatively well established, the relationship between hospital-level performance after traumatic injury and general surgery procedures is unknown. Although these areas of surgery are currently served by 2 independent PI programs, the relationship between the hospital-level outcomes across these surgical fields is of particular interest due to evolution of the acute care surgery paradigm. The acute care surgeon is a specialist who is broadly trained in elective and emergency general surgery, trauma surgery, and surgical critical care.4 Since 2003, several leading surgical organizations have collaborated to develop and expand the acute care surgery training paradigm and subspecialty.5 With the development of acute care surgery and the trend toward surgical subspecialization, trauma surgeons are increasingly functioning as acute care surgeons and caring for trauma and emergency general surgery patients. Given the significant resources devoted to quality improvement and given that emergency general surgery and trauma care is increasingly being provided by the same personnel with overlapping resources, a better understanding of the relationship between outcomes after traumatic injury and emergency and elective general surgery procedures would be useful in identifying processes for surgical quality improvement.
The objective of this study was to compare the outcomes of trauma patients and patients undergoing emergency or elective general surgery procedures at the same institution. We postulated that the quality of care provided to trauma and emergency and elective general surgery patients would be similar, believing that hospital performance reflected institutional culture and, thus, would be similar across all services.
We conducted a retrospective cohort study of hospital outcomes for trauma and general surgery patients being treated at hospitals contributing to both the NTDB (version 8.1) and the ACS NSQIP (2005-2008). Appropriate hospital identifiers were used to identify common hospitals.
All ACS or state-designated level I and II trauma centers contributing at least 100 patients to the NTDB were included. Centers were limited to those contributing Abbreviated Injury Scale scores and comorbidities to allow for adequate risk adjustment. Owing to known underreporting of complications in the NTDB,6,7 only centers that were known to report complications were included (as evidenced by the presence of ≥1 urinary tract infection or pneumonia). Complications were identified by means of either the NTDB complication field or International Classification of Diseases, Ninth Revision, codes. A similar strategy has been used in a previously published work8 to address the potential for differential reporting of complications across centers. In the NTDB data, missing values for the presence of shock and the Glasgow Coma Scale motor score (approximately 3% and 11%, respectively) were estimated using single imputation.9
The ACS NSQIP samples the first 40 cases performed during consecutive 8-day cycles. The sampling procedure is affected by hospital surgical volume and is subject to several “oversampling” restrictions, including the inclusion of no more than 3 laparoscopic cholecystectomies in any 8-day cycle. Hospitals were required to submit a minimum of 20 emergency general surgery cases during the study period to be included in this study.
Using data derived from the NTDB, we applied patient inclusion and exclusion criteria from the TQIP to identify a cohort of injured patients at each study center.10 We identified all patients 16 years and older with an Injury Severity Score of at least 9 and at least 1 severe (Abbreviated Injury Scale score ≥3) injury (for blunt injuries: in the head, face, neck, thorax, abdomen, spine, and upper or lower extremity region; for penetrating injuries: in the neck, thorax, or abdomen region). Patients injured as a result of poisoning, suffocation, drowning, overexertion, environmental causes, or burns were excluded, as were isolated hip fractures in the elderly (age ≥65 years). Patients with gunshot wounds to the brain (E codes of E922.0-E922.9, E955.0-E955.4, E965.0-E965.4, E979.4, E985.0-E985.4, and E970 and ≥1 International Classification of Diseases, Ninth Revision, code in the range of 800-801.99 and 850-854.1) were identified and excluded. Patients with preexisting do-not-resuscitate directives and patients who were dead on arrival were not included. Finally, patients with an unknown emergency department or hospital discharge disposition were also excluded.
Using data from the ACS NSQIP, a cohort of elective general surgery patients and a cohort of emergency general surgery patients were identified. Patients were required to be 16 years or older and to have undergone a procedure between January 1, 2005, and December 31, 2008. Patients were designated as having a general surgery procedure based on primary Current Procedural Terminology codes. Patients were categorized as having an emergency or elective operation based on the status as determined by the surgeon or anesthesiologist.11 Surgical Clinical Reviewers are instructed to review the anesthesia record followed by the operative note for documentation that the operation is an emergency. Because Surgical Clinical Reviewers are instructed to code operations that are documented as urgent as elective, our emergency cohort is uncontaminated by these borderline emergency cases.
The outcomes of interest were mortality and serious morbidity. Note that the NTDB abstracts in-hospital mortalities and complications, whereas Surgical Clinical Reviewers for the ACS NSQIP abstract adverse events occurring within 30 days of the index operation. In the ACS NSQIP, Surgical Clinical Reviewers determine mortality through examination of medical records, attempts to contact patients a minimum of 3 times via telephone or mail, and queries of the Social Security Death Index and the National Obituary Archives. Serious morbidity in trauma patients was defined as systemic sepsis, pulmonary embolism, acute respiratory distress syndrome, acute renal failure, stroke/cerebrovascular accident, cardiac arrest, myocardial infarction, and pneumonia. Serious morbidity in the ACS NSQIP cohorts included organ/space surgical site infection, wound dehiscence, sepsis/septic shock, pulmonary embolism, renal failure (acute or chronic), neurologic event (cerebrovascular accident or coma lasting >24 hours), cardiac arrest, myocardial infarction, bleeding, and pneumonia. For both data sets, patients could experience more than 1 of the listed complications.
We evaluated mortality and serious morbidity as a measure of performance. Forward stepwise logistic regression models were developed using the covariates detailed in Table 1. Models were constructed for mortality and serious morbidity for trauma and emergency and elective general surgery patients; thus, 6 logistic regression models were constructed. Model performance was assessed using c-statistics for discrimination. (Reference can be made to eTables 1-6 for details of the models constructed.) In the ACS NSQIP data, a preliminary logistic regression model yielded the linear risk (logit scale) associated with each of 137 Current Procedural Terminology procedure categories. This linear risk variable was then used in subsequent models.
Probabilities of the outcome of interest obtained from the logistic regression model were used to calculate observed to expected (O/E) ratios. Probabilities were summed for each hospital to estimate the hospitals' risk-adjusted morbidity or mortality. For each hospital, the observed number of events (O) was divided by the risk-adjusted expected number of events (E) to produce an O/E ratio. An O/E ratio of 1.0 indicated that the number of observed events equaled the number of expected events. An O/E ratio less than 1.0 indicated better-than-expected outcomes; a ratio greater than 1.0 indicated worse-than-expected outcomes. If the 90% CI for mortality or the 95% CI for serious morbidity did not include 1.0, then the risk-adjusted outcome was deemed statistically significant, and the hospital was designated as an “outlier.”12 Hospitals were divided into 4 approximately equal groups based on O/E ratios. To compare hospital performance, correlation of O/E ratios and outlier status were compared, and the association of hospital outcomes based on quartile designation was compared using weighted κ. All data manipulation and analyses were performed using a commercially available software program (SAS, version 9.2; SAS Institute, Inc).
During the study period, 46 hospitals contributed to both the NTDBand the ACS NSQIP. These hospitals treated 32 557 trauma patients and 134 495 general surgery patients. There were 120 256 (89.4%) elective general surgery and 14 239 (10.6%) emergency general surgery patients identified. Trauma patients tended to be young and male and to sustain blunt injuries, with nearly half being assigned an Injury Severity Score greater than 15 (Table 2). Emergency general surgery patients tended to be younger and were more likely to be assigned an American Society of Anesthesiologists class of 4 or 5 than were elective general surgery patients. Emergency general surgery patients also tended to have a lower baseline functional status and a higher comorbidity burden than did elective general surgery patients.
Crude mortality was 7.5% (n = 2455) for trauma patients, 6.6% (n = 944) for emergency general surgery patients, and 1.4% (n = 1631) for elective general surgery patients. Crude serious morbidity was 10.7% (n = 3471) for trauma patients, 16.2% (n = 2306) for emergency general surgery patients, and 6.1% (n = 7356) for elective general surgery patients.
The O/E ratios and number of outliers for mortality and serious morbidity are given in Table 3 and Figure 1 and Figure 2. For mortality, there was no significant relationship comparing O/E ratios between trauma and emergency general surgery care (r = −0.01, P = .94) or between trauma and elective general surgery care (r = 0.23, P = .12). Similarly, for serious morbidity, there was no significant relationship between trauma and emergency general surgery care (r = 0.21, P = .17) or between trauma and elective general surgery care (r = 0.16, P = .30). For both mortality and serious morbidity, no hospitals were consistently low or high outliers across all 3 groups.
The associations of hospital outcomes based on quartile rank demonstrated no significant agreement.13 For mortality, the weighted κ between quartile ranks for trauma and emergency general surgery patients was −0.10 (P = .61); comparing outcomes after trauma and elective general surgery, the weighted κ was 0.07 (P = .62). For serious morbidity, the weighted κ between quartile ranks for trauma and emergency general surgery patients was 0.04 (P = .63); comparing outcomes after trauma and elective general surgery, the weighted κ was 0.11 (P = .37).
In this retrospective cohort study, we sought to determine whether a relationship exists between hospital-level outcomes for trauma and general surgery care using data from the NTDB and ACS NSQIP. We demonstrated a high risk of mortality and serious morbidity associated with traumatic injuries and emergency general surgery procedures. We also found no significant association between hospital performance after traumatic injuries and emergency and elective general surgery procedures.
We initially postulated that the quality of care provided to trauma and emergency and elective general surgery patients would be similar. With patients, government, and payers demanding high-quality medical care, many health care institutions have devoted significant resources to developing and maintaining a “culture of quality.”14 One component of this commitment is often adherence to process measures. Perhaps more important, however, is the finding that increased hospital-level compliance with process measures that directly improve care is associated with improvement in unmeasured performance metrics and, subsequently, patient outcomes.15 This supports the concept of a hospital-level culture of quality, which affects all patients at a single institution. Although we had anticipated that the hospital-level quality of care for trauma and emergency and elective general surgery patients would be similar, these results suggest that this culture of quality may not affect all patient cohorts equally.
There might be several explanations as to why there was no evidence of any relationship between trauma and general surgical outcomes. It is plausible that hospital-level quality improvement initiatives might be procedure specific and, therefore, might not equally benefit all patient populations. For example, adherence to preoperative antibiotic processes of care may have a greater effect on specific outcomes, such as surgical site infection, in general surgery than in trauma care.16 Thus, hospitals with high compliance for processes of care that are specific to one population of surgical patients may have disparate outcomes when comparing performance between 2 surgical populations. In addition, current quality improvement initiatives may have relatively small associations with risk-adjusted outcomes. A cross-sectional study of hospitals participating in the ACS NSQIP and the Surgical Care Improvement Project evaluated correlations between compliance with Surgical Care Improvement Project process measures and ACS NSQIP risk-adjusted outcomes. Of the 16 correlations performed, only 1 demonstrated a significant association with risk-adjusted outcomes (appropriate antibiotic prophylaxis and surgical site infection [P = .004]). Furthermore, inclusion of compliance with antibiotic administration within 1 hour prior to incision and appropriate antibiotic prophylaxis caused only a slight improvement in model quality.17
The absence of an association between outcomes after trauma and emergency general surgery procedures, specifically, is also unexpected given the similarities between the patients, processes, and providers (ie, often hemodynamically unstable patients requiring critical care and emergency surgical and ancillary services) that are common between these 2 patient populations. Furthermore, trauma centers have long had PI programs as a requirement for verification. Given that these patient cohorts (emergency general surgery and trauma) are most frequently under the purview of the same providers, one would believe that outcomes would be similar. However, acute care surgery services have only recently been formally developed and implemented. Indeed, owing to limitations of the NTDB and the ACS NSQIP, characterization of the providers (ie, fellowship training, incorporation into an acute care surgery service, and the presence of service-level algorithms for specific diagnoses) caring for patients in this study cannot be provided. Given the novelty of acute care surgery, it is plausible that this paradigm of care has not matured to the extent that acute care surgery patients derive benefit from the existing trauma PI programs. The lack of an association between trauma and emergency general surgery outcomes has significant implications for quality improvement in these areas and may lead to the development of quality improvement programs that follow the model of trauma.
The ACS and the trauma community as a whole have a well-established commitment to trauma PI. The ACS Committee on Trauma has delineated guidelines for trauma care in Resources for Optimal Care of the Injured Patient, which was first published in 197618; this document details specific structures and processes, including multidisciplinary PI programs, that an acute care hospital should have in place to provide optimal care to injured patients. With this document, the PI processes for trauma are well described and standardized. With the advent of the TQIP, the trauma community can also externally benchmark trauma center performance.
In comparison, PI for emergency general surgery procedures is less well established. Although the external benchmarking of hospital outcomes for general surgery procedures has been in place since 1991 with the advent of the NSQIP in the Veterans Affairs Health System, unlike that of trauma care, PI for general surgery has traditionally relied on morbidity and mortality conferences and only recently has been supported on a national level by guidelines on processes of care (ie, the Surgical Care Improvement Project). However, many of these PI initiatives are not specific to emergency general surgery procedures. In addition, unlike other surgical subspecialties, such as pediatric (ACS NSQIP–Pediatric), thoracic (Society of Thoracic Surgeons), or bariatric (Bariatric Surgery Center Network) surgery, among others, a specific quality improvement program, accreditation program, or registry for emergency general surgery has not been developed.
Specific efforts should be directed at ensuring the quality of emergency general surgery care. However, this mandate is complicated by the variety of settings in which emergency general surgery care is provided. With the expansion of the acute care surgery paradigm, emergency general surgery patients at primarily academically affiliated, high-volume hospitals will increasingly use many of the same structures and processes as trauma patients. At these institutions, given the commitment and resources currently devoted to trauma PI, the quality of emergency general surgery care may naturally fall under the purview of trauma/acute care surgeons. However, as acute care surgery services are not feasible in every hospital setting, methods to ensure the quality of emergency general surgery care at hospitals that do not incorporate this paradigm should be implemented. In either setting, strategies (ie, collection of variables specific to emergency general surgery in current systems, detailed reviews of cases with morbidity or mortality events, and processes for identifying unexpected events) to ensure the quality of emergency general surgery care need to be developed. Successful quality improvement in emergency general surgery will need the support and guidance of national surgical organizations for development and support.
The present study must be interpreted in light of several limitations. First, this study is based on a nonrandom sample of hospitals. Thus, the results may be specific to hospitals contributing to both the NTDB and ACS NSQIP or to the specific distribution of hospitals included in this study. Evaluating outcomes from a larger number of hospitals or hospitals that have not committed significant resources to quality improvement through participation in the ACS NSQIP may yield different results. Second, the risk-adjustment may not be sufficient, because there may be unmeasured characteristics of patients or disease processes that serve as potential confounders. As one example, neither the NTDB nor the ACS NSQIP captures data regarding withdrawal of support, which plays an important role in mortality statistics for the elderly and severely injured. Note, however, that the need for withdrawal of support might reflect previous adverse events related to poor-quality care (eg, failure to prevent a venous thromboembolism that causes a pulmonary embolism with cardiac arrest and subsequent anoxic brain injury). Third, as mentioned previously, underreporting of complications is common in the NTDB.6,7 Although this may potentially affect the morbidity but not the mortality measure, we mitigated the potential impact of this limitation by including only hospitals that submitted at least 1 pneumonia or urinary tract infection deriving morbidity from the complication field and International Classification of Diseases, Ninth Revision, codes. Fourth, it would be beneficial to repeat this study after the TQIP database has matured as the data integrity of the TQIP mirrors that of the ACS NSQIP. Despite these limitations, this study addresses an important, timely question as hospitals, providers, and the surgical leadership strive to improve the delivery and quality of trauma and emergency surgery care.
In conclusion, using clinical data from the ACS, we evaluated the relationship between outcomes after traumatic injury and emergency and elective general surgery procedures at hospitals contributing to both the NTDB and ACS NSQIP. We demonstrated no significant association between hospital performance after traumatic injuries and emergency and elective general surgery procedures. Given the significant resources devoted to surgical quality improvement and the potential for trauma and emergency general surgery patients to rely on common resources and health care professionals, combining resources for emergency general surgery quality improvement with those for trauma PI should be explored.
Correspondence: Angela M. Ingraham, MD, MS, Department of Surgery, Mail Location 0558, University of Cincinnati, 231 Albert Sabin Way, Room 1586, Cincinnati, OH 45267-0558 (email@example.com).
Accepted for Publication: December 19, 2011.
Published Online: March 19, 2012. doi:10.1001/archsurg.2012.71
Author Contributions:Study concept and design: Ingraham, Haas, and Ko. Acquisition of data: Ingraham. Analysis and interpretation of data: Ingraham, Haas, Cohen, and Nathens. Drafting of the manuscript: Ingraham, Haas, and Nathens. Critical revision of the manuscript for important intellectual content: Ingraham, Haas, Cohen, and Ko. Statistical analysis: Ingraham, Haas, Cohen, and Nathens. Obtained funding: Ko. Administrative, technical, and material support: Ko. Study supervision: Ko.
Financial Disclosure: None reported.
Funding/Support: Dr Ingraham is supported by the Clinical Scholars in Residence Program at the ACS. Dr Haas is supported by a Canadian Institutes of Health Research fellowship. Dr Nathens is supported by a Canada Research Chair in Systems of Trauma Care.
Previous Presentation: This study was presented as a scientific poster at the 69th Annual Meeting of the American Association for the Surgery of Trauma; September 22, 2010; Boston, Massachusetts.