ACS TQIP indicates American College of Surgeons Trauma Quality Improvement Program; MTQIP, Michigan Trauma Quality Improvement Program; NTDB, National Trauma Data Bank.
The dotted line indicates the transition from the preintervention period to the postintervention period, and the error bars indicate 95% CIs. ACS TQIP indicates American College of Surgeons Trauma Quality Improvement Program; MTQIP, Michigan Trauma Quality Improvement Program; VTE, venous thromboembolism.
eFigure. MTQIP venous thromboembolism process measure compliance by year.
eTable. NTDB, ACS TQIP, and MTQIP data elements for outcomes and processes.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Hemmila MR, Cain-Nielsen AH, Jakubus JL, Mikhail JN, Dimick JB. Association of Hospital Participation in a Regional Trauma Quality Improvement Collaborative With Patient Outcomes. JAMA Surg. 2018;153(8):747–756. doi:10.1001/jamasurg.2018.0985
How does hospital participation in a regional quality collaborative for trauma surgery affect patient outcomes over time?
In this cohort study using National Trauma Data Bank data, hospital participation in a regional collaborative quality initiative, the Michigan Trauma Quality Improvement Program, significantly improved risk-adjusted results over time. Michigan Trauma Quality Improvement Program participation was independently associated with improved outcomes for major complications and venous thromboembolism compared with nonparticipation or American College of Surgeons Trauma Quality Improvement Program participation.
The regional collaborative quality improvement model provides the optimal framework to deliver improved patient outcomes while promoting compliance with processes of care for trauma patients.
The American College of Surgeons Trauma Quality Improvement Program (ACS TQIP) provides feedback to hospitals on risk-adjusted outcomes. The Michigan Trauma Quality Improvement Program (MTQIP) goes beyond the provision of feedback alone, focusing on collaborative quality improvement. It is unknown whether the addition of a collaborative approach to benchmark reporting improves outcomes.
To evaluate the association of hospital participation in the ACS TQIP (benchmark reporting) or the MTQIP (benchmark reporting and collaborative quality improvement) with outcomes compared with control hospitals that did not participate in either program.
Design, Setting, and Participants
In this cohort study, data from the National Trauma Data Bank from 2009 to 2015 were used. A total of 2 373 130 trauma patients 16 years or older with an Injury Severity Score of 5 or more were identified from 98 ACS TQIP hospitals, 23 MTQIP hospitals, and 429 nonparticipating hospitals, based on program participation status in 2011. A difference-in-differences analytic approach was used to evaluate whether hospital participation in the ACS TQIP or the MTQIP was associated with improved outcomes compared with nonparticipation in a quality improvement program.
Hospital participation in MTQIP, a quality improvement collaborative, compared with ACS TQIP participation and nonparticipating hospitals.
Main Outcomes and Measures
In-hospital mortality, mortality or hospice, major complications, and venous thromboembolism events were assessed.
Of the 2 373 130 included trauma patients, 64.2% were men and 73.0% were white, and the mean (SD) age was 50.7 (21.9) years. After accounting for patient factors and preexisting time trends toward improved outcomes, there was a statistically significant improvement in major complications after (vs before) hospital enrollment in the MTQIP collaborative compared with nonparticipating hospitals (odds ratio [OR], 0.89; 95% CI, 0.83-0.95) or ACS TQIP hospitals (OR, 0.88; 95% CI, 0.82-0.94). A similar result was observed for venous thromboembolism (MTQIP vs nonparticipating: OR, 0.78; 95% CI, 0.69-0.88; MTQIP vs ACS TQIP: OR, 0.84; 95% CI, 0.74-0.95), for which MTQIP targeted specific performance improvement efforts. Hospital participation in both ACS TQIP and MTQIP was associated with improvement in mortality or hospice (ACS TQIP vs nonparticipating: OR, 0.90; 95% CI, 0.87-0.93; MTQIP vs nonparticipating: OR, 0.88; 95% CI, 0.81-0.96). Hospitals participating in MTQIP achieved the lowest overall risk-adjusted mortality in the postenrollment period (4.2%; 95% CI, 4.1-4.3).
Conclusions and Relevance
This study demonstrates that hospital participation in a regional collaborative quality improvement program is associated with improved patient outcomes beyond benchmark reporting alone while promoting compliance with processes of care.
Over the past decade, there has been substantial investment in creating large clinical registries aimed at providing outcomes benchmarking for hospital quality improvement. A well-known national program is the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP).1-3 A counterpart program exists for trauma care, the American College of Surgeons Trauma Quality Improvement Program (ACS TQIP).4-6 The ACS TQIP began as a pilot program in 2008 and opened for formal enrollment in 2010. The program provides participating hospitals with detailed reports describing their risk-adjusted outcomes based on data extracted from the medical record and entered into a large clinical data repository. For trauma, the provision of risk-adjusted mortality as a nonpublic report card in an earlier study was found to not be associated with achievement of improved outcomes postimplementation.7
To our knowledge, whether the addition of a collaborative quality initiative (CQI) effort to benchmark reporting alone improves outcomes is unknown. Collaborative quality initiative programs are founded on bringing together participants to learn from their collective data, develop best practices, perform quality improvement by disseminating learned information longitudinally, and use social capital to influence change.8-10 In Michigan, we implemented a CQI for trauma, the Michigan Trauma Quality Improvement Program (MTQIP).11 This collaborative has focused on specific areas of performance improvement in the care of trauma patients (eg, venous thromboembolism [VTE] prophylaxis). Previously, to our knowledge, there has been no direct head-to-head comparison of these 2 possible approaches to quality improvement in relation to a control group of nonparticipating hospitals for trauma.
In this study, we use the National Trauma Data Bank (NTDB) to investigate whether trauma center participation in MTQIP, an approach that uses both benchmark reporting and collaborative quality improvement, improves outcomes beyond hospitals participating in national benchmarking (ACS TQIP) or control hospitals that do not participate in either quality program. To control for background trend changes in outcomes that occur with time, a difference-in-differences analytic approach was used.12
Critical differences exist between regional CQI programs and national benchmarking. In addition to the provision of feedback reports and hosting meetings, MTQIP measures each trauma center on their participation and quality improvement results annually.11 The MTQIP hospital performance index is developed by the coordinating center with guidance from an advisory committee and discussed with participating hospital surgeon champions before being finalized. The current MTQIP hospital performance index has 70% of its points allocated to performance-based measures.13 Examples of outcomes and process measures scored on the performance index include data accuracy, overall trauma center quality improvement progress (eg, mortality and major complications rates), blood to plasma transfusion ratio compliance, timely VTE prophylaxis administration, type of VTE prophylaxis administration, and prophylactic inferior vena cava filter use. At the end of each calendar year, points are allocated within participation and performance categories on the performance index, and total points achieved is calculated for each participating hospital. Only the final total aggregate score from 0 to 100 is reported to the sponsor, Blue Cross Blue Shield of Michigan, for use in their Value Partnerships Program. Implementation of a CQI program can be challenging, and information is available on potential barriers to collaborative creation and how to overcome them.11,13,14 Approval for this study was obtained from the Michigan Medicine institutional review board, and informed consent was waived because deidentified data were used.
Data obtained from the NTDB for admission years 2009 to 2015 were used to create the main analysis data set (Figure 1). These data contain voluntary submissions of trauma registry data from trauma centers located primarily in the United States and Canada. A separate participant user file of data with information from hospitals participating in ACS TQIP during the 2010 admission year was obtained. An additional data set was extracted from the MTQIP master data file for patient admission years 2009 to 2015. Patients were excluded if they were younger than 16 years, had an Injury Severity Score less than 5, were transferred out of the existing hospital emergency department (ED) to another hospital for definitive care, exhibited no signs of life in the ED (ie, ED systolic blood pressure of 0, ED heart rate of 0, and ED Glasgow Coma Scale [GCS] score of 3),15 or had a mechanism of injury not consistent with a blunt or penetrating trauma event. Patients who had an Injury Severity Score of 5 or greater but did not have anatomic body region Abbreviated Injury Scale information were also excluded. Race/ethnicity data were extracted from the hospital medical record system and entered into the trauma registry for each patient by the trauma registrars. Information on race/ethnicity was assessed for use in risk-adjustment models.
Hospitals that identified themselves as being an ACS Committee on Trauma–verified or a state-designated level III or IV trauma center were excluded. To minimize incomplete data, we excluded hospital data for a given year if the patient volume was less than 200 cases for that year. We also excluded calendar year data for hospitals if they reported less than a 0.5% rate of major complications or mortality in a given year.16 Major complications were defined as the occurrence of a decubitus ulcer, deep surgical site infection, myocardial infarction, organ/space surgical site infection, pneumonia, pulmonary embolus, stroke/cerebral vascular accident, or systemic sepsis.17 The source of all data definitions for data elements in the NTDB is the National Trauma Data Standard.18
Within NDTB and ACS TQIP data, hospitals were identified by a common facility code. We matched patients from the NTDB on continuous covariates (ie, year of injury, age, ED systolic blood pressure, ED heart rate, ED GCS motor score, and ED temperature) with 2010 ACS TQIP data and MTQIP data. We matched hospitals with the most hits (ie, number of cases matched on all variables), then performed sanity checks for the Michigan hospitals based on characteristics available in the facility data of the NTDB and MTQIP. No incongruities were found using this method. Using the matched hospital facility codes, we were able to create 3 groups of patients within the NTDB data: nonparticipating, ACS TQIP, and MTQIP. The control group (nonparticipants) consisted of patients treated in hospitals that were not participating in either ACS TQIP or MTQIP at the beginning of 2011. The ACS TQIP group contained patients treated in hospitals identified as having been enrolled in ACS TQIP prior to 2011. The MTQIP group included patients who were cared for at hospitals that started participating in MTQIP on January 1, 2011. All hospitals participating in MTQIP were also enrolled in ACS TQIP from 2011 onward. Crossover, or the addition of hospitals to groups based on later changes in enrollment status for ACS TQIP or MTQIP, was not allowed.
Additional process measure data for VTE prophylaxis were available in the MTQIP data. Data elements included the drug type of first VTE prophylaxis and the date and time of the administration of the first dose of VTE prophylaxis. Data collection for these process measures began in January 2012.11,19
Mortality, major complications, and VTE events were assessed to determine if enrollment in ACS TQIP or MTQIP was associated with improved outcomes compared with the nonparticipation cohort. Mortality was defined as an ED or hospital disposition of death. The mortality or hospice outcome is a composite measure signifying an in-hospital death or hospital disposition of discharge to hospice.20 A VTE event was defined as the occurrence of a deep venous thrombosis or pulmonary embolism. All outcomes were measured up to the time of discharge from the hospital.
To assess differences in hospital and patient characteristics across the cohorts, we used χ2 tests for categorical variables and analysis of variance F tests for continuous variables. A difference-in-differences approach was used to assess changes in mortality, major complications, and VTE rates preintervention and postintervention.12,21-23 The preintervention period was defined as 2009 to 2010 and the postintervention period as 2011 to 2015.
We used mixed-effects logistic regression models to estimate the difference-in-differences effect and adhered to proposed minimum best practices for risk-adjustment when using NTDB data.24 To adjust for case mix, we created patient risk scores and modeled the outcome of interest in a logistic regression using relevant covariates (ie, sex, age category, race/ethnicity category, heart rate category, systolic blood pressure category, GCS motor category, mechanism of injury, Injury Severity Score category, Abbreviated Injury Scale score in anatomic regions, transfer in, and comorbidities present in 0.5% or more of the population) and then used the linear predictions from these models as a measure of patient risk. We included these risk scores in the mixed-effects logistic regression models as well as a linear time trend (year of admission) to control for secular trends. To test the difference-in-differences effect, we included a categorical variable for cohort (nonparticipating/ACS TQIP/MTQIP), an indicator for the period (preintervention/postintervention), and an interaction term between cohort and time in the model. The interaction of cohort × time is the term of interest in the model and can be interpreted as whether the expected change in outcome, preintervention vs postintervention, differed by cohort. Alternatively, the difference-in-differences estimator can be interpreted as whether the cohort effect differed across time (or whether the time effect was different across cohorts).
We included a hospital-specific random effect in our model to account for heterogeneity across hospitals. We did not include specific hospital attributes, such as volume and teaching status, that could contribute to such heterogeneity, as explicitly modeling these components was not of interest in our study. To assess whether our data had parallel trends in outcomes preintervention, we tested the significance of an interaction term (year × cohort) in the preintervention period using generalized linear models.25 In some instances, specific incidents had missing values for potentially important covariates (eg, ED systolic blood pressure, ED heart rate, and ED GCS motor score). To minimize bias, these values were categorized and accounted for using indicator variables.
Results are expressed as risk-adjusted outcomes or, in the case of difference-in-differences analyses, as odds ratios. 95% confidence intervals were calculated using variance estimates that account for patient clustering within hospitals. A P value less than .05 was used as the threshold for statistical significance, and all reported P values were 2-sided. All statistical analyses were performed in SAS version 9.4 (SAS Institute).
It is acknowledged that the NTDB remains the full and exclusive copyrighted property of the ACS. The American College of Surgeons is not responsible for any claims arising from works based on the original data, text, tables, or figures.
Data from 5 630 305 patients from 1044 hospitals present in the NTDB from 2009 to 2015 were used. A total of 2 373 130 patients from 550 hospitals were selected for study using the inclusion and exclusion criteria (Figure 1). Based on enrollment data for 2011, patients were placed into 1 of 3 cohorts: nonparticipating, ACS TQIP (hospital participating in benchmark reporting feedback), or MTQIP (hospital participating in benchmark reporting and collaborative quality improvement).
Hospital and patient characteristics for the trauma centers represented in each enrollment cohort are detailed in Table 1. There were many differences in these attributes on univariate analysis, and these parameters were accounted for in the risk-adjustment models for the outcomes investigated. Risk-adjusted outcomes before and after collaborative enrollment are shown in Table 2. The relative percentage change in these outcomes is also listed.
Baseline differences were identified in the preintervention period for in-hospital mortality, mortality or hospice, major complications, and VTE between the 3 cohorts (Table 1). The MTQIP group had the lowest baseline mortality rate, and nonparticipating hospitals had the lowest baseline major complication and VTE rate (Figure 2). While the levels of baseline outcomes differed by cohort, there was no evidence of nonparallel trends in the baseline outcomes from 2009 to 2010 for mortality, mortality or hospice, major complications, and VTE.
Using a difference-in-differences approach, we found that ACS TQIP hospitals had significant improvements in the outcomes of in-hospital mortality, mortality or hospice, and VTE over time compared with nonparticipating hospitals (Table 3). Similarly, MTQIP hospitals had a greater decrease in the outcomes of mortality or hospice, major complications, and VTE over time compared with nonparticipating hospitals. In a direct comparison, MTQIP hospitals produced superior improvement in the risk-adjusted outcomes of major complication (24.1% vs 11.0% relative reduction) and VTE (28.1% vs 10.0% relative reduction) compared with ACS TQIP trauma centers. Of the 3 cohorts, the CQI approach used by MTQIP produced the lowest risk-adjusted outcome results over time and at the conclusion of the study for in-hospital mortality, mortality or hospice, major complications, and VTE.
In this study, there was a time trend toward improved outcomes in major complications and VTE for all 3 groups examined. However, hospitals that participated in the MTQIP CQI showed the greatest percentage reduction in major complications, and the difference-in-differences analysis identified that hospital participation in MTQIP was independently associated with improved outcomes for major complications and VTE compared with hospitals that participated in ACS TQIP (benchmark reporting alone) or nonparticipating hospitals. Hospital participation in MTQIP, a quality program that emphasizes benchmark reporting and collaborative quality improvement, produced the best risk-adjusted results overall at the conclusion of the study for all of the outcomes examined.
The Michigan Trauma Quality Improvement Program has specifically targeted VTE prophylaxis as a focus of quality improvement. Changes in rates of compliance with VTE prophylaxis for the entire MTQIP collaborative have been described previously.11,19 The VTE prophylaxis compliance data for the MTQIP cohort in this study illustrates increased use of low-molecular-weight heparin as the drug type used from 2012 to 2015 (36.6% vs 47.2%) (eFigure in the Supplement). Better compliance with the timing of administration for the first dose of pharmacologic VTE prophylaxis was also observed (42.9% vs 58.3% administered first dose within 48 hours after admission). A summary of data elements in the NTDB, ACS TQIP, and MTQIP illustrates the evolution over time of data collection for examining processes of care (eTable in the Supplement).
The primary strength of this study and its findings is that it included 3 successive levels of quality improvement program participation. The nonparticipating group of hospitals consisted of trauma centers that did not participate in a quality program. A second cohort allowed for the independent assessment of the effect of national benchmark reporting through hospital participation in ACS TQIP on risk-adjusted outcomes. The MTQIP cohort enabled an evaluation of how the addition of a regional collaborative quality improvement effort to benchmark reporting could influence patient outcomes. Prior studies evaluating quality improvement research have been hindered by the lack of a control group.26 Thus, it has been difficult to isolate true quality improvement from a regression to the mean effect. Cohen et al27 found that most hospitals participating in ACS NSQIP improved their results for morbidity and mortality after enrolling in the program from 2006 to 2013 and that the magnitude of quality improvement increased with time in the program. However, their study did not include a control group to establish whether the participating hospitals improved to a greater degree than other nonparticipant hospitals.28 Subsequent studies that did use a hospital control group of nonparticipants in ACS NSQIP found no improvement beyond secular trends for ACS NSQIP participants.26,29 To overcome these problems, our study used a control group and 2 progressive levels of intervention, which allowed us to isolate the effects of benchmark reporting alone and benchmark reporting in combination with a collaborative quality improvement program.
Our findings suggest that a regional collaborative quality improvement program when coupled with outcomes benchmarking is highly effective in optimizing clinical outcomes for a population of trauma patients. The regional collaborative approach used by CQI programs in Michigan is established on simple but key principles. Hospitals share common problems. Performance improvement is local, with unique solutions tailored to the environment of each hospital. Regional CQIs are a form of efficient information exchange. Collaboratives can provide access to data and flexible incorporation of data elements focused on problems of interest to the participants. Collaboration allows for a diversity of ideas and rapid dissemination of new information and findings.8,10,11,30,31 The mechanism by which MTQIP focuses on selection and measurement of compliance with multiple process measures reflects the culmination of collaboration based on the aforementioned principles. The MTQIP collaborative has succeeded in significantly reducing serious complications, decreasing resource use, and improving process measure compliance in trauma patients.11,19,32 The cost of complications is high and is typically borne by third-party payers.33,34 Hence, support of a regional CQI for trauma care may represent an effective investment to achieve optimal health care value.
The Zero Preventable Deaths Campaign for trauma care identified the pragmatic, rapid model for learning known as focused empiricism, developed and used by the Joint Trauma System as evidence of a learning health system that improves patient outcomes and reduces preventable mortality.35,36 A learning health care system, as defined by the Institute of Medicine, is characterized by a consistent emphasis on a collaborative approach that shares data and insights across boundaries to drive better, more efficient medical practice and patient care.37 Both MTQIP and the Joint Trauma System are, at their core, educational vehicles that serve as examples of a learning health system.
This study has certain limitations. The NTDB is a clinical database and contains data that may be unverified or missing. We attempted to limit this effect by excluding data from trauma centers and yearly submissions that appeared to be incomplete. Within the NTDB, there is minimal information recorded on processes of care, and hence, we were not able to examine the effect of process of care information on outcomes for the nonparticipating and ACS TQIP groups. It is acknowledged that approximately 170 trauma centers likely crossed over from the nonparticipating cohort to the ACS TQIP cohort during the later years of the study in accordance with increased enrollment in the national program. We elected to use an intent-to-treat stratification in the analyses and made comparisons with both the nonparticipating and ACS TQIP groups to mitigate this effect. The following complication data elements were either added during the study or underwent significant data definition changes and were not included in the major complications grouping: acute kidney injury, acute respiratory distress syndrome, cardiac arrest with resuscitative efforts by health care worker, catheter-related bloodstream infection, unplanned return to the operating room, and unplanned admission to the intensive care unit.
In summary, our study found that rates of inpatient mortality and VTE improved differently over time based on hospital participation in ACS TQIP vs nonparticipation in benchmark reporting. We also found that hospital participation in MTQIP, a regional CQI, produced the best risk-adjusted results and was independently associated with improved outcomes for major complications and VTE compared with nonparticipation or ACS TQIP participation. This study demonstrates that hospital participation in a regional CQI was associated with improved patient outcomes while promoting compliance with processes of care.
Accepted for Publication: February 10, 2018.
Corresponding Author: Mark R. Hemmila, MD, Department of Surgery, University of Michigan, North Campus Research Complex, Bldg 16, Room 139E, 2800 Plymouth Rd, Ann Arbor, MI 48109-2800 (email@example.com).
Published Online: May 2, 2018. doi:10.1001/jamasurg.2018.0985
Author Contribution: Dr Hemmila and Ms Cain-Nielsen had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: Hemmila, Cain-Nielsen, Jakubus, Dimick.
Drafting of the manuscript: Hemmila, Cain-Nielsen, Dimick.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Hemmila, Cain-Nielsen, Dimick.
Obtained funding: Hemmila, Dimick.
Administrative, technical, or material support: Hemmila, Jakubus, Mikhail, Dimick.
Study supervision: Hemmila, Dimick.
Conflict of Interest Disclosures: Drs Hemmila and Mikhail and Mss Cain-Nielsen and Jakubus receive salary support from Blue Cross Blue Shield of Michigan and the Blue Care Network (a nonprofit mutual company) through their support of the Michigan Trauma Quality Improvement Program. Dr Dimick is a founder of and has an equity interest in ArborMetrix Inc, which provides software and analytics for measuring hospital quality and efficiency. The company had no role in the study. No other disclosures were reported.
Funding/Support: This study was supported by a Blue Cross Blue Shield of Michigan and Blue Care Network Collaborative Quality Initiatives grant for administration of the Michigan Trauma Quality Improvement Program (Dr Hemmila). This study was also supported by grant R01AG039434 from the National Institute on Aging (Dr Dimick).
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: The authors acknowledge the contribution of the participating trauma medical directors, trauma program managers, Michigan Trauma Quality Improvement Program clinical reviewers, and trauma registrars to the collaborative effort of the Michigan Trauma Quality Improvement Program.
Create a personal account or sign in to: