Mild injuries (horizontal line indicates mean group survival, 0.99; 95% confidence interval,
0.99-0.99). Error bars indicate 95% confidence intervals.
Moderate injuries (horizontal line indicates mean group survival, 0.75; 95% confidence interval,
0.75-0.76). Error bars indicate 95% confidence intervals.
Severe injuries (horizontal line indicates mean group survival, 0.35; 95% confidence interval,
0.34-0.37). Error bars indicate 95% confidence intervals.
Shafi S, Friese R, Gentilello LM. Moving Beyond Personnel and ProcessA Case for Incorporating Outcome Measures in the Trauma Center Designation Process. Arch Surg. 2008;143(2):115-119. doi:10.1001/archsurg.2007.29
Similarly designated trauma centers do not achieve similar outcomes.
Forty-seven American College of Surgeons–verified level I trauma centers that contributed more than 1000 patients to the National Trauma Data Bank (from January 1999 to December 2003) were identified.
Main Outcome Measures
Patients were classified into the following 3 injury severity groups using a combination of anatomical and physiological measures:
mild (Injury Severity Score [ISS] of <25 with systolic blood pressure [SBP] of ≥90 mm Hg [n = 184 650]), moderate (ISS of ≥25 with SBP of ≥90 mm Hg or ISS of <25 with SBP of <90
mm Hg [n = 22 586]), and severe (ISS of ≥25 with SBP of <90 mm Hg [n = 4243]). The mean survival for each group was calculated. Individual centers were considered outliers if their patient survival was statistically significantly different from the mean survival for each severity group.
The mean survival of patients with mild, moderate, and severe injuries was 99%, 75%, and 35%, respectively. For mild injuries, survival at 5 centers (11%) was significantly worse than that at their counterpart centers. With increasing injury severity, the percentages of outcome disparities increased (15% of centers for moderate injuries and 21%
of centers for severe injuries) and persisted in subgroups of patients with head injuries, patients sustaining penetrating injuries, and older (>55 years) individuals.
When treating patients with similar injury severity, similarly designated level I trauma centers may not achieve similar outcomes,
suggesting the existence of a quality chasm in trauma care. Trauma center verification may require the use of outcome measures when determining trauma center status.
Decades of concerted efforts by trauma professionals and patient advocacy groups have led to the development of trauma centers in most states.1 There are data to suggest that the development of these centers has improved the care of injured patients across the country.2,3 A critical component of these systems is the use of explicit criteria for the availability of personnel,
equipment, and services through the process of trauma center verification by the American College of Surgeons (ACS).4 Trauma center verification criteria in their current format are based primarily on structures and processes that are deemed essential for the provision of optimal care. Hence, when injured patients are brought to a verified trauma center, it is assumed that optimal resources are available to appropriately care for their injuries. The verification process seeks to ensure that similarly designated trauma centers around the country have similarly optimized patient outcomes.
Recent studies by MacKenzie et al3 and Demetriades et al5 demonstrated that survival rates of trauma patients are better at designated trauma centers compared with those at nondesignated hospitals, as well as at level I trauma centers compared with those at nondesignated hospitals and level II trauma centers. However, it remains unknown if similarly designated trauma centers achieve similar outcomes. Institutional variations in treatment outcomes of other diseases (eg, in intensive care units and after cardiac surgery, esophagectomy, and abdominal aortic aneurysm repair) are well documented.6- 8 These variations are often attributed to differences in resources available at different institutions.9 This logic has sometimes led to the consolidation of specialized services at fewer institutions and has been incorporated in the initiative for improving quality of care by the Leapfrog Group.10
This argument may not be valid for designated trauma centers because the designation and verification process ensures that similar resources are available at different institutions with similar designations.
In other words, one would expect similar outcomes at similarly designated trauma centers, adjusting for differences in injury severity and patient characteristics. We hypothesized that similarly designated trauma centers do not achieve similar clinical outcomes in terms of survival to discharge from initial hospitalization after trauma.
The National Trauma Data Bank of the ACS was analyzed. The National Trauma Data Bank, which collects data from participating trauma centers in a standardized format, is the largest database of inpatient trauma patients in the United States (N = 1 130 581).
Patients who were treated at ACS-verified level I trauma centers during a 5-year period (January 1999 to December 2003) with known survival status, final Injury Severity Score (ISS), and recorded systolic blood pressure at presentation to the emergency department were included in the study. To minimize variations that may be associated with a low volume of patients, only centers that contributed more than 1000
patients to the National Trauma Data Bank during the 5-year study period were included. The final study population consisted of 211 479
patients from 47 ACS-verified level I trauma centers.
For risk adjustment, the study population was classified into groups of injury severity based on a combination of anatomical and physiological measures. The ISS was used to measure anatomical injury severity and shock at presentation, defined as a systolic blood pressure of less than 90 mm Hg on arrival in the emergency department, and was used as a physiological measure of injury severity. Patients were first classified into groups based on anatomical severity of injuries (ISS of 1-8, 9-15, 16-24, 25-34, or 35-75); the mean survival for these groups was 0.99, 0.98, 0.93, 0.71, and 0.54, respectively. Based on the sharp decline in survival associated with an ISS of at least 25, this value was chosen to define severe anatomical injury.
Patients were then classified into 4 groups using a 2 ×
2 table based on the 2 ISS groups (ISS of <25 vs ≥25) and on the presence of shock at presentation (Table 1). Because survival in patients with an ISS of less than 25 with shock and those with an ISS of at least 25 without shock was similar, these patients were collapsed into 1 group. Therefore,
the study population was finally classified into the following 3 injury severity groups: mild injury (ISS of <25 without shock [n = 184 650];
mean survival, 0.99 [95% confidence interval (CI), 0.99-0.99]), moderate injury (ISS of <25 with shock or ISS of ≥25 without shock [n = 22 586];
mean survival, 0.75 [95% CI, 0.75-0.76]), and severe injury (ISS of ≥25 with shock [n = 4243]; mean survival, 0.35 [95% CI,
0.34-0.37]). These survival rates were used to define the normative survival rates for patients within each injury severity subgroup.
The survival rate of patients at each trauma center was compared with the normative survival for each severity group using 95% CIs.
Trauma centers were then classified as performing statistically significantly better or worse than their counterpart centers if their patient survival rates were different from the normative survival rate for each severity subgroup. All data analyses were performed using commercially available software (SPSS for Windows, version 13.0; SPSS Inc, Chicago, Illinois). P < .05 was considered statistically significant for all statistical analyses.
The 47 trauma centers included in the study were more likely to be publicly funded (n = 28), academic (n = 34)
institutions, with a median of 444 hospital beds and 18 trauma intensive care unit beds. The mean (SD) age of the study patients was 36 (21)
years, 69% were male, and 83% sustained blunt injuries. The mean (SD)
ISS was 10 (10), the mean (SD) trauma-related ISS (TRISS) probability of survival was 0.94 (0.17), and the actual survival was 95%. The distribution of injuries was the head in 22%, face in 18%, neck in 1.5%, chest in 15%, abdomen in 10%, spine in 9%, spinal cord in 1%,
upper extremity in 21%, and lower extremity in 26%.
Overall, survival at 28 of 47 trauma centers differed from that of their counterpart centers in at least 1 injury severity group.
Of these, 12 performed better than their counterpart centers, 13 performed worse than their counterpart centers, and 3 achieved mixed results (ie, better than their counterpart centers in one injury severity group and worse in others). Two centers differed from their counterpart centers in all 3 injury severity groups; one center performed significantly worse in all 3 injury severity groups, whereas the other center performed worse in the mild and moderate injury severity groups but better in the severe injury group.
Twelve centers significantly differed from their counterpart centers in 2 of 3 injury severity groups. Of these, 4 centers performed worse than the group means in both severity groups, 6 centers performed better than the group means, and 2 centers had mixed performances (ie, performed better than the group mean in one injury severity group and worse than the group mean in the other injury severity group).
Fifteen centers significantly differed from their counterpart centers in only 1 injury severity group. Of these, 6 centers performed better than the mean, and 9 centers performed worse than the mean.
In the mild injury group, the normative survival was 0.99 (95%
CI, 0.99-0.99). The mean survival at individual trauma centers ranged from 0.88 to 1.00 (Table 2).
There were 3 trauma centers with mean survival that was significantly higher than the group mean, and there were 5 centers with mean survival that was significantly lower than the group mean (Figure 1). In the moderate injury group,
the normative survival was 0.75 (95% CI, 0.75-0.76). The mean survival at individual trauma centers ranged from 0.56 to 0.96 (Table 2). Nine centers reported a mean survival that was significantly higher than the group mean, and 7
centers reported a survival rate that was significantly lower than the group mean (Figure 2). In the severe injury group, the normative survival was 0.35 (95% CI,
0.34-0.37). The mean survival at individual trauma centers ranged from 0.13 to 1.00 (Table 2).
Nine centers reported a survival rate that was significantly higher than the group mean, and 10 centers reported a survival rate that was significantly lower than the group mean (Figure 3).
Although there was a correlation between increasing mortality and the 3 injury severity groups used for this analysis, severity adjustment alone may not account for the differences in patient characteristics seen at each center. To further assess whether other risk factors accounted for the outcome differences observed, we examined subgroups of patients with head injuries, patients sustaining penetrating injuries,
and older (>55 years) individuals. Similar variations in severity-adjusted survival between centers persisted in all 3 of these subgroups (Tables 3, 4, and 5).
Despite similar resources, processes, and personnel availability,
this study shows that all designated level I trauma centers do not achieve similar survival when treating patients with similar injury severity. These variations in outcomes may represent a substantial quality chasm in the delivery of trauma care. Recognition of the potential existence of a quality chasm would be an essential first step toward closing such a gap.
The trauma center verification process has been the cornerstone of trauma system development in this country for more than 3 decades.
Led by the ACS, this effort has been one of the most remarkable success stories in the field of trauma care. The first set of trauma center verification criteria was published in 1976 as an “optimal resource document,” and the verification process was started in 1987.4 Based on current ACS verification criteria,
several states designate hospitals as level I, II, III, or IV centers.
Some states have developed individual designation processes that are largely based on ACS criteria. In states with no legislative mandate for trauma systems, individual centers may request and receive verification directly from the ACS.
The verification process includes detailed evaluation and documentation of personnel and resources available to care for patients with a wide spectrum of injuries, as well as the existence of processes and protocols to use those personnel and resources appropriately. All verified level I trauma centers are presumed to have the capability of providing optimal resources for the provision of care for every aspect of injury,
from prevention through rehabilitation. In its current form, this procedure has successfully served its purpose and has led to the development of a physical infrastructure to care for injured patients in most regions of the country. At last count, there were 190 level I trauma centers, 263 level II trauma centers, and scores of level III and level IV trauma centers nationwide.1
If confirmed, our preliminary data suggest that the logical next step for the trauma community is to move beyond focusing on personnel and processes and to start focusing on the outcomes achieved by the use of those resources. Trauma centers are well suited for this next step because they have traditionally provided national leadership in the development of performance improvement and quality assurance methods and have required registry data to measure processes and outcomes.
We believe that trauma centers should continue to lead the way in quality improvement by developing methods that include outcomes measures as part of their verification process.
Variations in outcomes are well recognized in health care. The ACS National Surgical Quality Improvement Program is a risk-adjusted program to improve the quality of care by moving the surgical community toward accountability by conducting hospital-based comparisons of patient outcomes. After implementing this outcomes-based program in 128 hospitals, the Veterans Health Administration reported a 27% decrease in surgical mortality and a 45% decrease in surgical morbidity.11
We performed severity adjustment by creating separate groups of injury severity based on anatomical and physiological factors.
The narrow CIs around the mean survival rates seem to confirm that patients within each of the 3 injury severity groups had similar injury severity. The persistence of wide variations in outcomes at different trauma centers, despite ACS verification of adequate resources, suggests that differences in outcomes cannot be attributed to differences in institutional resources or injury severity. Hence, there is a need to search for other factors that might explain these differences.
It is possible that the designation process does not measure all of the resources that are needed to optimize outcome. For example,
in a large multicenter study12 of academic level I trauma centers, the presence of an in-house attending trauma surgeon, a resource required by the ACS, did not improve outcomes,
while the presence of a trauma and surgical critical care fellowship program, which is not required for verification, was associated with reduced mortality. Another possibility is that our results reflect inadequate use or deployment of resources required for verification of level I trauma center status. We believe that this is the most plausible explanation and underscores the need for incorporating patient outcomes into the verification process.
The study has certain limitations, the most important of which is the validity of our risk-adjustment method. The injury severity groups defined in the study do not consider all of the differences that may exist between patient populations such as age distribution,
specific types of injuries, mechanism of injury, preexisting conditions,
socioeconomic status, prehospital care, transport time, urban vs rural location, and other factors.13- 15 We addressed some of the issues by looking at subgroups of patients with head injuries,
patients sustaining penetrating injuries, and older individuals, and the disparities persisted.
Despite this limitation, findings from this study suggest that institutional variations in outcomes of trauma patients exist. A logical next step in this type of analysis is the development of better indexes of risk stratification using robust statistical models similar to the National Surgical Quality Improvement Program system.16 However, that process may be slow in evolution because most centers still use TRISS methods for risk stratification,
which was developed almost 3 decades ago and primarily involved patients treated at community hospitals.17
In conclusion, the results of this study suggest the presence of substantial variation in patient survival among similarly designated level I trauma centers treating patients with similar injury severity.
To confirm the existence of this quality chasm in trauma care, there is a clear need to develop and validate risk-adjustment models that can be used to compare performances across trauma centers. The current process of trauma center verification may need to move beyond personnel and process measures to incorporate risk-adjusted outcome measures when determining trauma center status.
Correspondence: Shahid Shafi, MD,
MPH, Division of Burn, Trauma, and Surgical Critical Care, Department of Surgery, University of Texas Southwestern Medical Center, 5323
Harry Hines Blvd, Mail Code 9158, Dallas, TX 75390-9158 (email@example.com).
Accepted for Publication: September 4, 2006.
Author Contributions:Study concept and design: Shafi, Friese, and Gentilello. Acquisition of data: Shafi. Analysis and interpretation of data: Shafi and Gentilello. Drafting of the manuscript: Shafi. Critical revision of the manuscript for important intellectual content: Shafi, Friese, and Gentilello. Statistical analysis: Shafi. Administrative, technical, and material support: Shafi and Gentilello. Study supervision: Gentilello.
Financial Disclosure: None reported.