[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.205.176.107. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure.
Risk-Adjusted Mortality Rates by Hospital Performance on Quality Metrics
Risk-Adjusted Mortality Rates by Hospital Performance on Quality Metrics

A, Hospitals according to their rates of intracranial pressure (ICP) monitor placement and risk-adjusted mortality. B, Hospitals according to their rates of craniotomy and risk-adjusted mortality.

Table 1.  
Characteristics of Patients With Severe TBI Who Were Admitted to LA County Trauma Centers, 2009-2010
Characteristics of Patients With Severe TBI Who Were Admitted to LA County Trauma Centers, 2009-2010
Table 2.  
Characteristics of Hospitals in the LA County Trauma System, 2009-2010
Characteristics of Hospitals in the LA County Trauma System, 2009-2010
Table 3.  
Association Between Hospital-Level Compliance With Quality Metrics and Inpatient Mortality
Association Between Hospital-Level Compliance With Quality Metrics and Inpatient Mortality
1.
Coronado  VG, Xu  L, Basavaraju  SV,  et al; Centers for Disease Control and Prevention (CDC).  Surveillance for traumatic brain injury-related deaths—United States, 1997-2007. MMWR Surveill Summ. 2011;60(5):1-32.
PubMed
2.
Selassie  AW, Zaloshnja  E, Langlois  JA, Miller  T, Jones  P, Steiner  C.  Incidence of long-term disability following traumatic brain injury hospitalization, United States, 2003. J Head Trauma Rehabil. 2008;23(2):123-131.
PubMedArticle
3.
Farhad  K, Khan  HMR, Ji  AB, Yacoub  HA, Qureshi  AI, Souayah  N.  Trends in outcomes and hospitalization costs for traumatic brain injury in adult patients in the United States. J Neurotrauma. 2013;30(2):84-90.
PubMedArticle
4.
Ghajar  J, Hariri  RJ, Narayan  RK, Iacono  LA, Firlik  K, Patterson  RH.  Survey of critical care management of comatose, head-injured patients in the United States. Crit Care Med. 1995;23(3):560-567.
PubMedArticle
5.
Brain Trauma Foundation, American Association of Neurological Surgeons, Congress of Neurological Surgeons.  Guidelines for the management of severe traumatic brain injury. J Neurotrauma. 2007;24(suppl):S1-S106.
PubMedArticle
6.
Bratton  SL, Chestnut  RM, Ghajar  J,  et al; Brain Trauma Foundation; American Association of Neurological Surgeons; Congress of Neurological Surgeons; Joint Section on Neurotrauma and Critical Care, AANS/CNS.  Guidelines for the management of severe traumatic brain injury. [published correction appears in J Neurotrauma. 2008;25(3):276-278]. J Neurotrauma. 2007;24(suppl 1):S37-S44.
PubMed
7.
Bullock  MR, Chesnut  R, Ghajar  J,  et al.  Guidelines for the surgical management of traumatic brain injury. Neurosurgery. 2006;58(3):S2-1–S2-3. doi:10.1227/01.NEU.0000210361.83548.D0.
8.
Fakhry  SM, Trask  AL, Waller  MA, Watts  DD, IRTC Neurotrauma Task Force.  Management of brain-injured patients by an evidence-based medicine protocol improves outcomes and decreases hospital charges. J Trauma. 2004;56(3):492-500.
PubMedArticle
9.
Arabi  YM, Haddad  S, Tamim  HM,  et al.  Mortality reduction after implementing a clinical practice guidelines-based management protocol for severe traumatic brain injury. J Crit Care. 2010;25(2):190-195.
PubMedArticle
10.
Shafi  S, Barnes  SA, Rayan  N,  et al.  Compliance with recommended care at trauma centers: association with patient outcomes. J Am Coll Surg. 2014;219(2):189-198.
PubMedArticle
11.
Hesdorffer  DC, Ghajar  J.  Marked improvement in adherence to traumatic brain injury guidelines in United States trauma centers. J Trauma. 2007;63(4):841-848.
PubMedArticle
12.
Morse  RB, Hall  M, Fieldston  ES,  et al.  Hospital-level compliance with asthma care quality measures at children’s hospitals and subsequent asthma-related outcomes. JAMA. 2011;306(13):1454-1460.
PubMedArticle
13.
Howell  EA, Zeitlin  J, Hebert  PL, Balbierz  A, Egorova  N.  Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity. JAMA. 2014;312(15):1531-1541.
PubMedArticle
14.
Werner  RM, Bradlow  ET.  Relationship between Medicare’s hospital compare performance measures and mortality rates. JAMA. 2006;296(22):2694-2702.
PubMedArticle
15.
Ryan  AM, Burgess  JF  Jr, Tompkins  CP, Wallack  SS.  The relationship between Medicare’s process of care quality measures and mortality. Inquiry. 2009;46(3):274-290.
PubMed
16.
Stulberg  JJ, Delaney  CP, Neuhauser  DV, Aron  DC, Fu  P, Koroukian  SM.  Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303(24):2479-2485.
PubMedArticle
17.
Bilimoria  KY, Chung  J, Ju  MH,  et al.  Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure. JAMA. 2013;310(14):1482-1489.
PubMedArticle
18.
Kernisan  LP, Lee  SJ, Boscardin  WJ, Landefeld  CS, Dudley  RA.  Association between hospital-reported Leapfrog Safe Practices Scores and inpatient mortality. JAMA. 2009;301(13):1341-1348.
PubMedArticle
19.
Qian  F, Lustik  SJ, Diachun  CA, Wissler  RN, Zollo  RA, Glance  LG.  Association between Leapfrog safe practices score and hospital mortality in major surgery. Med Care. 2011;49(12):1082-1088.
PubMed
20.
Glance  LG, Dick  AW, Osler  TM,  et al.  Relationship between Leapfrog Safe Practices Survey and outcomes in trauma. Arch Surg. 2011;146(10):1170-1177.
PubMedArticle
21.
Dawes  AJ, Sacks  GD, Cryer  HG,  et al; Los Angeles County Trauma Consortium.  Intracranial pressure monitoring and inpatient mortality in severe traumatic brain injury: a propensity score-matched analysis. J Trauma Acute Care Surg. 2015;78(3):492-501.
PubMedArticle
22.
Adams  JL. The Reliability of Provider Profiling: A Tutorial. Santa Monica, CA: RAND Corp; 2009.
23.
Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment. Health Serv Res. 2010;45(6, pt 1):1614-1629.
PubMedArticle
24.
Dimick  JB, Ghaferi  AA, Osborne  NH, Ko  CY, Hall  BL.  Reliability adjustment for reporting hospital outcomes with surgery. Ann Surg. 2012;255(4):703-707.
PubMedArticle
25.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-46.e1.
PubMedArticle
26.
Alali  AS, Fowler  RA, Mainprize  TG,  et al.  Intracranial pressure monitoring in severe traumatic brain injury: results from the American College of Surgeons Trauma Quality Improvement Program. J Neurotrauma. 2013;30(20):1737-1746.
PubMedArticle
27.
Biersteker  HAR, Andriessen  TMJC, Horn  J,  et al.  Factors influencing intracranial pressure monitoring guideline compliance and outcome after severe traumatic brain injury. Crit Care Med. 2012;40(6):1914-1922.
PubMedArticle
28.
Farahvar  A, Gerber  LM, Chiu  Y-L, Carney  N, Härtl  R, Ghajar  J.  Increased mortality in patients with severe traumatic brain injury treated without intracranial pressure monitoring. J Neurosurg. 2012;117(4):729-734.
PubMedArticle
29.
Talving  P, Karamanos  E, Teixeira  PG,  et al.  Intracranial pressure monitoring in severe head injury: compliance with Brain Trauma Foundation guidelines and effect on outcomes: a prospective study. J Neurosurg. 2013;119(5):1248-1254.
PubMedArticle
30.
Tang  A, Pandit  V, Fennell  V,  et al.  Intracranial pressure monitor in patients with traumatic brain injury. J Surg Res. 2015;194(2):565-570.
PubMedArticle
31.
Schreiber  MA, Aoki  N, Scott  BG, Beck  JR.  Determinants of mortality in patients with severe blunt head injury. Arch Surg. 2002;137(3):285-290.
PubMedArticle
32.
Farahvar  A, Gerber  LM, Chiu  Y-L,  et al.  Response to intracranial hypertension treatment as a predictor of death in patients with severe traumatic brain injury [published correction appears in J Neurosurg. 2011;115(1):191]. J Neurosurg. 2011;114(5):1471-1478.
PubMedArticle
33.
Karamanos  E, Teixeira  PG, Sivrikoz  E,  et al.  Intracranial pressure versus cerebral perfusion pressure as a marker of outcomes in severe head injury: a prospective evaluation. Am J Surg. 2014;208(3):363-371.
PubMedArticle
34.
Chesnut  RM, Temkin  N, Carney  N,  et al; Global Neurotrauma Research Group.  A trial of intracranial-pressure monitoring in traumatic brain injury. N Engl J Med. 2012;367(26):2471-2481.
PubMedArticle
35.
Khuri  SF, Najjar  SF, Daley  J,  et al.  Comparison of surgical outcomes between teaching and nonteaching hospitals in the Department of Veterans Affairs. Ann Surg. 2001;234(3):370-383.
PubMedArticle
36.
Ayanian  JZ, Weissman  JS.  Teaching hospitals and quality of care: a review of the literature. Milbank Q. 2002;80(3):569-593, v.
PubMedArticle
37.
Donabedian  A.  Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):166-206.
PubMedArticle
38.
Birkmeyer  JD, Dimick  JB, Birkmeyer  NJO.  Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg. 2004;198(4):626-632.
PubMedArticle
39.
Hawn  MT, Itani  KM, Gray  SH, Vick  CC, Henderson  W, Houston  TK.  Association of timely administration of prophylactic antibiotics for major surgical procedures and surgical site infection. J Am Coll Surg. 2008;206(5):814-819.
PubMedArticle
40.
Urbach  DR, Govindarajan  A, Saskin  R, Wilton  AS, Baxter  NN.  Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370(11):1029-1038.
PubMedArticle
41.
Berenholtz  SM, Pronovost  PJ, Lipsett  PA,  et al.  Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32(10):2014-2020.
PubMedArticle
42.
Pronovost  P, Needham  D, Berenholtz  S,  et al.  An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):2725-2732.
PubMedArticle
43.
Berwick  DM.  Measuring surgical outcomes for improvement: was Codman wrong? JAMA. 2015;313(5):469-470.
PubMedArticle
Original Investigation
Pacific Coast Surgical Association
October 2015

Compliance With Evidence-Based Guidelines and Interhospital Variation in Mortality for Patients With Severe Traumatic Brain Injury

Author Affiliations
  • 1Department of Surgery, David Geffen School of Medicine, UCLA (University of California, Los Angeles)
  • 2Robert Wood Johnson Clinical Scholars Program, UCLA
  • 3Department of Surgery, Veterans Affairs Greater Los Angeles Healthcare System, Los Angeles, California
  • 4Department of Neurosurgery, University of Southern California, Los Angeles
  • 5Emergency Medical Services Agency, County of Los Angeles Department of Health Services, Los Angeles, California
  • 6Department of Neurosurgery, David Geffen School of Medicine, UCLA
JAMA Surg. 2015;150(10):965-972. doi:10.1001/jamasurg.2015.1678
Abstract

Importance  Compliance with evidence-based guidelines in traumatic brain injury (TBI) has been proposed as a marker of hospital quality. However, the association between hospital-level compliance rates and risk-adjusted clinical outcomes for patients with TBI remains poorly understood.

Objective  To examine whether hospital-level compliance with the Brain Trauma Foundation guidelines for intracranial pressure monitoring and craniotomy is associated with risk-adjusted mortality rates for patients with severe TBI.

Design, Setting, and Participants  All adult patients (N = 734) who presented to a regional consortium of 14 hospitals from January 1, 2009, through December 31, 2010, with severe TBI (ie, blunt head trauma, Glasgow Coma Scale score of <9, and abnormal intracranial findings from computed tomography of the head). Data analysis took place from December 2013 through January 2015. We used hierarchical mixed-effects models to assess the association between hospital-level compliance with Brain Trauma Foundation guidelines and mortality rates after adjusting for patient-level demographics, severity of trauma (eg, mechanism of injury and Injury Severity Score), and TBI-specific variables (eg, cranial nerve reflexes and findings from computed tomography of the head).

Main Outcomes and Measures  Hospital-level risk-adjusted inpatient mortality rate and hospital-level compliance with Brain Trauma Foundation guidelines for intracranial pressure monitoring and craniotomy.

Results  Unadjusted mortality rates varied by site from 20.0% to 50.0% (median, 42.6; interquartile range, 35.5-46.2); risk-adjusted rates varied from 24.3% to 56.7% (median, 41.1; interquartile range, 36.4-47.8). Overall, only 338 of 734 patients (46.1%) with an appropriate indication underwent placement of an intracranial pressure monitor and only 134 of 335 (45.6%) underwent craniotomy. Hospital-level compliance ranged from 9.6% to 65.2% for intracranial pressure monitoring and 6.7% to 76.2% for craniotomy. Despite widespread variation in compliance across hospitals, we found no association between hospital-level compliance rates and risk-adjusted patient outcomes (Spearman ρ = 0.030 [P = .92] for ICP monitoring and Spearman ρ = −0.066 [P = .83] for craniotomy).

Conclusions and Relevance  Hospital-level compliance with evidence-based guidelines has minimal association with risk-adjusted outcomes in patients with severe TBI. Our results suggest that caution should be taken before using compliance with these measures as independent quality metrics. Given the complexity of TBI care, outcomes-based metrics, including functional recovery, may be more accurate than current process measures at determining hospital quality.

Introduction

Traumatic brain injury (TBI) remains a substantial source of morbidity and mortality in the United States, accounting for nearly one-third of all injury-related deaths.13 Following reports of widespread variation in care,4 the Brain Trauma Foundation (BTF) published the first set of clinical guidelines for the treatment of TBI in 1995. In their most recent revision, the BTF guidelines include management strategies, treatment thresholds, and indications for the use of invasive procedures, namely, intracranial pressure (ICP) monitoring and craniotomy.57 In addition to receiving more streamlined clinical care, individuals who are treated based on BTF guidelines appear to have better clinical outcomes, including lower risk-adjusted mortality rates and higher rates of functional recovery.810 As a result, compliance with these guidelines has been proposed as a marker of hospital quality, and hospitals throughout the country are attempting to improve their TBI outcomes by increasing their levels of guideline-compliant care.11

However, despite substantial face validity, there is growing evidence that guideline compliance alone is an inaccurate and inadequate measure of hospital quality.12,13 Multiple studies have demonstrated a loose association between compliance with Medicare’s Hospital Compare measures (eg, giving aspirin within 24 hours of an acute myocardial infarction) and inpatient mortality rates.14,15 More specific to surgery, hospitals that score well on perioperative safety measures do not appear to have lower rates of surgical site infection,16 venous thromboembolism,17 or mortality.1820 To our knowledge, there remains no definitive evaluation of the connection between guideline compliance and hospital quality in severe TBI.

Our study had the following 2 aims: (1) to document levels of compliance with BTF guidelines for ICP monitoring and craniotomy in a large regional trauma system and (2) to examine the association between hospital-level compliance with these guidelines and risk-adjusted mortality rates. In so doing, we sought to determine whether compliance with BTF guidelines represents a viable marker of hospital quality in the treatment of patients with severe TBI.

Methods
Study Design and Data Sources

The Los Angeles (LA) County Trauma Consortium was formed in 2013 as a collaboration between LA County’s 14 trauma centers, the county’s Emergency Medical Services Agency, and health services researchers from 2 local universities. A description of the consortium and its research objectives has been published previously.21Quiz Ref ID Briefly, we developed a prospective registry of all patients with severe TBI in LA County during a 2-year period (January 1, 2009, through December 31, 2010). Inclusion was based on 3 criteria: (1) blunt head trauma, (2) a Glasgow Coma Scale (GCS) score of less than 9 on arrival, and (3) abnormal intracranial findings from the initial computed tomographic scan of the head. These criteria were chosen based on the definition of severe TBI in the BTF guidelines. We excluded patients who died on arrival or were younger than 18 years at the time of injury. Of the 14 hospitals in the consortium, 1 is a designated children’s hospital and was effectively eliminated from our analysis based on the age restriction.

Trained trauma-program managers at each center prospectively identified patients who met the inclusion criteria and abstracted relevant patient-level data into our protected electronic registry. Hospital characteristics were obtained from 2 sources. Basic structural characteristics were taken from the American Hospital Association website (http://www.ahadataviewer.com). Trauma-program managers were also surveyed regarding care practices at their institution during the study period, specifically whether there was a designated neuro–critical care unit, a general or neurosurgery residency program, or a clinical protocol for the treatment of severe TBI. This study was approved by the Office of Human Research Protection Program at UCLA.

Patient-Level Variables

Our primary outcome at the patient level was death due to any cause during the initial hospitalization for TBI (inpatient mortality). Placement of an ICP monitor in the first 72 hours after arrival was recorded as an independent field in our clinical registry. Craniotomy during the first 72 hours was determined based on International Classification of Diseases, Ninth Edition, procedure codes (eTable 1 in the Supplement).

Covariates were selected to parallel the risk-adjustment techniques used by the Trauma Quality Improvement Program (TQIP), the largest clinical registry of traumatically injured patients in the United States, for patients with TBI. The TQIP includes the following 23 variables in its regression models: demographics, vital signs, mechanism of injury, Injury Severity Score, GCS score, and 14 medical comorbidities. Three variables (dialysis, concurrent corticosteroid use, and cardiac arrest) were recorded but not used in the statistical analyses because of their low prevalence and high collinearity.

In addition to the 20 TQIP variables, we included the following 11 TBI-specific variables captured by our registry: pupil reactivity, international normalized ratio, and 9 separate intracranial findings on the initial computed tomographic scan of the head (Table 1). Pupil reactivity was defined as present or absent based on the initial neurologic or trauma surgery consultation. International normalized ratio values were divided into normal (≤1.4) and elevated (>1.4) based on the initial measurements in the emergency department. All 9 findings from computed tomography of the head were coded independently as present or absent based on the official radiology report.

Hospital-Level Quality Metrics

We determined compliance with guidelines for each hospital by dividing the number of patients who received a particular therapy (ie, ICP monitoring or craniotomy) by the number of patients with an indication for that therapy based on BTF guidelines. The BTF guidelines currently recommend ICP monitoring for all patients with TBI who have a GCS score of less than 95; because this group included all patients in our sample, we divided the number of patients who underwent ICP monitoring at a given hospital by that hospital’s sample size. The BTF guidelines also recommend craniotomy for patients with a GCS score of less than 9 in the setting of epidural hematoma, subdural hematoma with signs of midline shift or mass effect, or intraparenchymal contusion with signs of mass effect.7 We identified patients at each hospital who met these criteria and then calculated the rate of patients who underwent craniotomy in this group.

Risk-Adjusted Mortality Rate

We developed a hierarchical mixed-effects logistic regression model to predict inpatient mortality after controlling for all 31 patient-level covariates (20 TQIP + 11 TBI-specific variables) and hospital-specific random effects. We predicted the risk-adjusted mortality rates for each hospital after controlling for patient mix and then used empirical Bayesian techniques to adjust each estimate for its reliability.2224 This approach, also referred to as shrinkage adjustment, filters out statistical noise due to the small sample size of certain clusters and reweights values toward the overall sample mean based on each estimate’s reliability.25

Statistical Analysis

Bivariate tables were generated to compare patients who died during their hospitalization with those who survived until discharge. For unadjusted data, we used χ2 tests of independence for categorical variables and either 2-sample t tests or Mann-Whitney tests for continuous variables, depending on the distribution. We also ranked hospitals by their compliance with BTF guidelines and then divided hospitals into terciles based on these rankings. This process was conducted separately for ICP monitoring and craniotomy such that the distribution of hospitals within terciles could differ for each quality metric.

We evaluated the association between hospital-level compliance rates and risk-adjusted mortality rates in 2 manners. First, we plotted hospitals’ compliance rates against their risk-adjusted mortality rates and calculated Spearman rank correlations. This analysis was conducted separately for each quality metric. Second, we added hospitals’ guideline compliance in terciles to our hierarchical regression model and predicted the risk-adjusted mortality rate for each tercile. This analysis was also conducted separately for each quality metric using only patients who were eligible for that measure. We then compared unadjusted mortality rates across terciles using the Kruskal-Wallis test and risk-adjusted mortality rates using Wald tests with standard errors generated via the delta method. All P values were 2-sided and P ≤ .05 was considered significant.

We conducted several sensitivity analyses to test the robustness of our estimates. First, we calculated the reliability of our registry data by comparing our independent data field for ICP monitoring with the International Classification of Diseases, Ninth Edition, procedure codes (eTable 2 in the Supplement). Because this method demonstrated a high level of consistency (κ = 0.852), we used our independent data field. Second, 43 patients (5.7%) had missing data in the following 2 variables: international normalized ratio (36 patients) and heart rate (7 patients). Multiple analyses demonstrated no informative pattern to the missing data. To use all the available data, we imputed missing values via a multivariate normal regression technique after controlling for the full set of covariates, including placement of an ICP monitor, craniotomy, and mortality. As an additional test of our findings, we conducted our analyses using only complete cases. This analysis did not affect our results, and we reported regression results for the full sample with imputed values for patients with missing data. Third, we restricted our analyses to patients with an isolated head injury (Abbreviated Injury Score <2 in all body regions except the head and neck); this restriction did not affect our findings and we report the data for the entire sample. Finally, we conducted our regression analyses using hospital compliance as a continuous variable. Because this method also did not affect our results, we reported terciles for interpretability. Statistical analyses were conducted from December 2013 through January 2015 using STATA/IC, version 13.0 (StataCorp LP).

Results

During the study period, 753 adult patients sustained severe TBI; 19 patients (2.5%) died on arrival and were excluded from further analysis. Of the 734 patients who survived to admission, the mean age was 46.3 years, 177 patients (24.1%) were female, and 296 (40.3%) were Hispanic (Table 1). The median Injury Severity Score was 26 (interquartile range [IQR], 21-35) and the median GCS score was 3 (IQR, 3-6). Falls were the most common mechanism of injury (235 [32.0%]) followed by auto vs pedestrian (211 [28.8%]). Nearly two-thirds of patients had at least 1 reactive pupil on arrival (482 [65.7%]). Overall, 296 patients (40.3%) died during their inpatient hospitalization.

Most trauma centers were not for profit (9 [69.2%]) and maintained an academic affiliation (5 [38.5%] major and 4 [30.8%] minor) (Table 2). Less than half had a neurocritical care unit (5 [38.5%]), a general or neurosurgery resident program (6 [46.2%]), or a protocol for treating severe TBI during the study period (6 [46.2%]). Unadjusted mortality rates ranged from 20.0% to 50.0% by hospital (median, 42.6%; IQR, 35.5%-46.2%); risk-adjusted mortality rates ranged from 24.3% to 56.7% (median, 41.1%; IQR, 36.4%-47.8%).

Only 338 patients (46.1%) underwent placement for an ICP monitor despite all patients meeting BTF criteria. Quiz Ref IDRates of compliance with ICP monitoring ranged by center from 9.6% to 65.2% (median, 47.4; IQR, 40.0-51.7); the lowest tercile of hospitals had a median compliance rate of 38.7% compared with 54.8% in the highest tercile (Table 2). Quiz Ref IDIn our sample, 335 patients (45.6%) had an indication for craniotomy and 134 of these patients (40.0%) underwent the procedure. Rates of compliance with craniotomy also varied by center from 6.7% to 76.2% (median, 41.7%; IQR, 33.3%-50.0%); the lowest tercile of hospitals had a median compliance rate of only 9.5% compared with 61.7% in the highest tercile.

The Figure presents each hospital according to its rates of guideline compliance and risk-adjusted mortality. Quiz Ref IDThere was no correlation between compliance rates and mortality for either ICP monitoring (Spearman ρ = 0.030; P = .92) (Figure, A) or craniotomy (Spearman ρ = −0.066; P = .83) (Figure, B). Similarly, there was no association between compliance with BTF guidelines and either unadjusted or risk-adjusted mortality based on our hierarchical regression model (Table 3). For both quality metrics, there appeared to be a trend toward lower mortality rates in hospitals with higher compliance based on unadjusted data; however, this trend disappeared after risk adjustment. The middle tercile on compliance for both ICP monitoring and craniotomy had the lowest risk-adjusted mortality rates (33.8% and 47.1%, respectively).

Discussion

In response to widespread variation in care, the BTF established clinical practice guidelines for TBI in the hope of promoting high-quality health care. Despite their efforts, we found no relationship between hospitals’ compliance with 2 BTF guidelines and risk-adjusted mortality in a large regional trauma system. Quiz Ref IDOur results suggest that caution should be taken before using compliance with these measures as independent quality metrics.

While ours is the first study, to our knowledge, to explore differences in hospitals’ use of craniotomy, other studies have investigated the association between hospital compliance with ICP monitoring and risk-adjusted mortality in patients with severe TBI.26,27 Using data from TQIP, Alali and colleagues26 reported improved survival rates for patients who were treated at hospitals in the highest vs the lowest quartile of ICP monitor use after controlling for patient- and hospital-level covariates. There are several possible explanations for our differing results. First, Alali and colleagues26 excluded patients with either “nonsurvivable”26(p1738) head injuries or significant nonhead injuries. Because we included all patients with severe TBI, our patient sample was more severely injured by comparison, as is evident by the difference in unadjusted mortality rates (35.5% in TQIP vs 40.3% in our sample). As a result, certain invasive treatments may have been less effective in our population, making interhospital differences in their use a less significant predictor of patient outcomes. Second, TQIP includes mostly large academic hospitals, which tend to be more similar to one another than to other hospitals in a given geographic region. Of our 13 adult trauma centers, only 3 currently participate in TQIP (all level I academic centers) and fewer than half have access to many of the hallmarks of academic medical centers, including specialized intensive care units or residency programs. Given the variety of hospitals involved, differences in compliance with BTF guidelines may simply play a smaller role in determining hospital quality for regional trauma systems, especially when one considers the fact that differences in ICP monitoring explained less than 10% of the interhospital variation in mortality rates in TQIP.26

Our results also appear to conflict with multiple patient-level analyses, including one from our consortium, that have consistently demonstrated a survival benefit for ICP monitoring in patients with severe TBI.21,28,29 Again, there are at least 2 possible explanations for this discrepancy: (1) differences in who receives ICP monitoring and (2) how effectively information from invasive monitors is incorporated into clinical management. First, even among the subset of patients with severe TBI, there is considerable heterogeneity such that not all patients benefit equally from invasive monitoring.30 Therefore, 2 hospitals with similar rates of ICP monitoring might effectively be providing different levels of care—and achieving different patient outcomes—if the subset of patients being monitored differs significantly between the 2 institutions.

Second, by recording only whether an ICP monitor was placed, guidelines may insufficiently capture critical aspects in the management of severe TBI, including how health care professionals react to changes in ICP or use related medical therapies (eg, hypertonic saline solution) to treat intracranial hypertension. Many publications regarding ICP monitoring suggest that clinical outcomes depend more on the timely management of intracranial hypertension than the use of particular medications or procedures.3133 This idea is typified by the only randomized controlled trial34 of ICP monitoring, which showed no difference in mortality between monitored and nonmonitored patients but took advantage of trained neurologists and neurosurgeons at the bedside acting as “functional” ICP monitors. Therefore, if hospitals that place more ICP monitors do not also use the additional information to adapt their treatment of intracranial hypertension, then the rates of placement may be uncorrelated (or even inversely correlated) with risk-adjusted mortality rates.

Our study has several limitations. First, our sample included the most severely injured patients in LA County and, as such, our results may not generalize to other regions or to less severely injured patients. Second, because of the delay between data collection and analysis, it is possible that health care practices have changed. However, as neither the guidelines nor the members of the trauma system changed during this period, we believe our results remain applicable to the current state of trauma care in LA County. Third, our sample size was relatively small at the hospital level, which limited our ability to compare hospitals directly and to control for hospital-level structural variables, such as teaching status or patient volume. Because other studies have suggested these factors may affect quality,35,36 our results may be confounded if differences in structural characteristics drive variation in both compliance and mortality rates. However, because the BTF guidelines apply equally to patients regardless of their treating hospital, we believe that controlling for certain hospital characteristics is not only irrelevant but may bias our results by explaining away meaningful differences in patient outcomes. Additional studies using a larger hospital-level sample are needed to better understand the associations between structural variables, inpatient mortality, and guideline compliance. Fourth, we had no information on the rates of withdrawal of care and therefore could not control for the ways in which different withdrawal practices between centers might have affected center-specific mortality rates. Finally, we were unable to characterize patients’ functional status at the time of discharge, and it is unclear whether differences in the use ICP monitoring and craniotomy are associated with differences in functional recovery.

These limitations notwithstanding, we believe our results have important implications for quality assessment in severe TBI. The debate between process and outcome has continued since the earliest publications on quality measurement.37 Process measures, including guideline compliance, offer the ease of measurement, sensitivity to change, and real-time feedback demanded by many proponents of quality improvement. Their downside, however, rests in the difficulty of determining which processes actually lead to better care.38 Our results suggest that current BTF guidelines fail to improve hospital quality both by measuring the wrong processes (ie, if a monitor is placed rather than how a monitor is used in management) and by doing so in the wrong group of patients (all patients with severe TBI rather than only those who would benefit from invasive monitoring).

However, recent experiences from several large multi-institutional quality-improvement programs suggest that even tracking the “perfect” process measure in the most appropriate subset of patients may be insufficient to actually improve patient outcomes.16,39,40 These findings have 3 implications for quality measurement, especially for conditions as complex as TBI. First, multiple related process measures may be needed for each clinical outcome (eg, placement of an ICP monitor plus thresholds for treatment plus goals for maintaining cerebral perfusion pressure).41,42 Second, process measurement must be combined with clinically important outcome metrics, such as mortality and functional recovery, to prevent hospitals from improving performance on one metric at the expense of others. Finally, it is not enough to simply measure performance and track progress over time.43 Instead, hospitals must build systems that use these data to influence physician behavior and improve patient care.

Conclusions

Despite improvements in care, mortality due to TBI remains both common and variable from hospital to hospital. Our results demonstrate no association between hospitals’ compliance with 2 BTF guidelines and risk-adjusted mortality, suggesting that neither measure should be used as an independent marker of hospital quality.

Back to top
Article Information

Accepted for Publication: April 3, 2015.

Corresponding Author: Aaron J. Dawes, MD, Ronald Reagan UCLA Medical Center, Department of Surgery, David Geffen School of Medicine, UCLA, 757 Westwood Plaza, Room B7-11, Los Angeles, CA 90095 (adawes@mednet.ucla.edu).

Published Online: July 22, 2015. doi:10.1001/jamasurg.2015.1678.

Author Contributions: Drs Dawes and Ko had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Dawes, Sacks, Gruen, Preston, Gorospe, Ko.

Acquisition, analysis, or interpretation of data: Dawes, Sacks, Cryer, Gruen, Preston, Gorospe, Cohen, McArthur, Russell, Maggard-Gibbons.

Drafting of the manuscript: Dawes, Sacks.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Dawes, Sacks, Cryer, Preston, Gorospe, McArthur, Russell.

Administrative, technical, or material support: Cryer, Preston, Gorospe, Cohen.

Study supervision: Cryer, Gruen, Russell, Maggard-Gibbons, Ko.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by the Veterans Affairs Robert Wood Johnson Clinical Scholars Program (Drs Dawes and Sacks) and the Veterans Affairs Office of Academic Affiliations (Dr Dawes).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: The Los Angeles County Trauma Consortium members were Cynthia Marin, RN, MSN, and Pavel Petrik, MD, Antelope Valley Hospital, Lancaster, California; Laura Schneider, RN, and Gudata Hinika, MD, California Hospital Medical Center, Los Angeles; Elizabeth Cleek, RN, and Jeffrey Upperman, MD, Children’s Hospital Los Angeles, Los Angeles, California; Heidi Hotz, RN, and Daniel Margulies, MD, Cedars-Sinai Medical Center, Los Angeles, California; Kimberly Murphy, RN, MSN, and David Hanpeter, MD, Providence Holy Cross Medical Center, Mission Hills, California; Robin Tyler, RN, and Brant Putnam, MD, Harbor-UCLA Medical Center, Torrance, California; Susan Thompson, RN, MSN, and Amal Obaid, MD, Huntington Memorial Hospital, Pasadena, California; Gilda Cruz-Manglapus, RN, and Ranbir Singh, MD, Henry Mayo Newhall Memorial Hospital, Valencia, California; Desiree Thomas, RN, and Brian Acker, MD, Long Beach Memorial Medical Center, Long Beach, California; Melanie Crowley, RN, MSN, and Shawki Saad, MD, Northridge Hospital Medical Center, Northridge, California; Renee Smith, RN, MSN, and Tchaka Shepherd, MD, St Francis Medical Center, Lynwood, California; Patricia Meier, RN, and James Murray, MD, St Mary Medical Center, Long Beach, California; Marilyn Cohen, RN, and H. Gill Cryer, MD, PhD, Ronald Regan UCLA Medical Center, Los Angeles, California; and Sixta Navarrete, RN, and Demetrios Demetriades, MD, LAC + USC (Los Angeles County and University of Southern California) Medical Center, Los Angeles.

Previous Presentation: This study was presented in part at the 86th Annual Meeting of the Pacific Coast Surgical Association; February 20, 2015; Monterey, California.

Additional Contributions: Nader Pouratian, MD, and Matt Garrett, MD, Department of Neurosurgery, UCLA, and Paul Vespa, MD, Departments of Neurosurgery and Neurology, UCLA, contributed to the project and Susan Paddock, PhD, RAND Pardee Graduate School, and John Adams, PhD, MS, Kaiser Permanente Department of Research and Evaluation, provided statistical review. None received financial compensation.

References
1.
Coronado  VG, Xu  L, Basavaraju  SV,  et al; Centers for Disease Control and Prevention (CDC).  Surveillance for traumatic brain injury-related deaths—United States, 1997-2007. MMWR Surveill Summ. 2011;60(5):1-32.
PubMed
2.
Selassie  AW, Zaloshnja  E, Langlois  JA, Miller  T, Jones  P, Steiner  C.  Incidence of long-term disability following traumatic brain injury hospitalization, United States, 2003. J Head Trauma Rehabil. 2008;23(2):123-131.
PubMedArticle
3.
Farhad  K, Khan  HMR, Ji  AB, Yacoub  HA, Qureshi  AI, Souayah  N.  Trends in outcomes and hospitalization costs for traumatic brain injury in adult patients in the United States. J Neurotrauma. 2013;30(2):84-90.
PubMedArticle
4.
Ghajar  J, Hariri  RJ, Narayan  RK, Iacono  LA, Firlik  K, Patterson  RH.  Survey of critical care management of comatose, head-injured patients in the United States. Crit Care Med. 1995;23(3):560-567.
PubMedArticle
5.
Brain Trauma Foundation, American Association of Neurological Surgeons, Congress of Neurological Surgeons.  Guidelines for the management of severe traumatic brain injury. J Neurotrauma. 2007;24(suppl):S1-S106.
PubMedArticle
6.
Bratton  SL, Chestnut  RM, Ghajar  J,  et al; Brain Trauma Foundation; American Association of Neurological Surgeons; Congress of Neurological Surgeons; Joint Section on Neurotrauma and Critical Care, AANS/CNS.  Guidelines for the management of severe traumatic brain injury. [published correction appears in J Neurotrauma. 2008;25(3):276-278]. J Neurotrauma. 2007;24(suppl 1):S37-S44.
PubMed
7.
Bullock  MR, Chesnut  R, Ghajar  J,  et al.  Guidelines for the surgical management of traumatic brain injury. Neurosurgery. 2006;58(3):S2-1–S2-3. doi:10.1227/01.NEU.0000210361.83548.D0.
8.
Fakhry  SM, Trask  AL, Waller  MA, Watts  DD, IRTC Neurotrauma Task Force.  Management of brain-injured patients by an evidence-based medicine protocol improves outcomes and decreases hospital charges. J Trauma. 2004;56(3):492-500.
PubMedArticle
9.
Arabi  YM, Haddad  S, Tamim  HM,  et al.  Mortality reduction after implementing a clinical practice guidelines-based management protocol for severe traumatic brain injury. J Crit Care. 2010;25(2):190-195.
PubMedArticle
10.
Shafi  S, Barnes  SA, Rayan  N,  et al.  Compliance with recommended care at trauma centers: association with patient outcomes. J Am Coll Surg. 2014;219(2):189-198.
PubMedArticle
11.
Hesdorffer  DC, Ghajar  J.  Marked improvement in adherence to traumatic brain injury guidelines in United States trauma centers. J Trauma. 2007;63(4):841-848.
PubMedArticle
12.
Morse  RB, Hall  M, Fieldston  ES,  et al.  Hospital-level compliance with asthma care quality measures at children’s hospitals and subsequent asthma-related outcomes. JAMA. 2011;306(13):1454-1460.
PubMedArticle
13.
Howell  EA, Zeitlin  J, Hebert  PL, Balbierz  A, Egorova  N.  Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity. JAMA. 2014;312(15):1531-1541.
PubMedArticle
14.
Werner  RM, Bradlow  ET.  Relationship between Medicare’s hospital compare performance measures and mortality rates. JAMA. 2006;296(22):2694-2702.
PubMedArticle
15.
Ryan  AM, Burgess  JF  Jr, Tompkins  CP, Wallack  SS.  The relationship between Medicare’s process of care quality measures and mortality. Inquiry. 2009;46(3):274-290.
PubMed
16.
Stulberg  JJ, Delaney  CP, Neuhauser  DV, Aron  DC, Fu  P, Koroukian  SM.  Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303(24):2479-2485.
PubMedArticle
17.
Bilimoria  KY, Chung  J, Ju  MH,  et al.  Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure. JAMA. 2013;310(14):1482-1489.
PubMedArticle
18.
Kernisan  LP, Lee  SJ, Boscardin  WJ, Landefeld  CS, Dudley  RA.  Association between hospital-reported Leapfrog Safe Practices Scores and inpatient mortality. JAMA. 2009;301(13):1341-1348.
PubMedArticle
19.
Qian  F, Lustik  SJ, Diachun  CA, Wissler  RN, Zollo  RA, Glance  LG.  Association between Leapfrog safe practices score and hospital mortality in major surgery. Med Care. 2011;49(12):1082-1088.
PubMed
20.
Glance  LG, Dick  AW, Osler  TM,  et al.  Relationship between Leapfrog Safe Practices Survey and outcomes in trauma. Arch Surg. 2011;146(10):1170-1177.
PubMedArticle
21.
Dawes  AJ, Sacks  GD, Cryer  HG,  et al; Los Angeles County Trauma Consortium.  Intracranial pressure monitoring and inpatient mortality in severe traumatic brain injury: a propensity score-matched analysis. J Trauma Acute Care Surg. 2015;78(3):492-501.
PubMedArticle
22.
Adams  JL. The Reliability of Provider Profiling: A Tutorial. Santa Monica, CA: RAND Corp; 2009.
23.
Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment. Health Serv Res. 2010;45(6, pt 1):1614-1629.
PubMedArticle
24.
Dimick  JB, Ghaferi  AA, Osborne  NH, Ko  CY, Hall  BL.  Reliability adjustment for reporting hospital outcomes with surgery. Ann Surg. 2012;255(4):703-707.
PubMedArticle
25.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-46.e1.
PubMedArticle
26.
Alali  AS, Fowler  RA, Mainprize  TG,  et al.  Intracranial pressure monitoring in severe traumatic brain injury: results from the American College of Surgeons Trauma Quality Improvement Program. J Neurotrauma. 2013;30(20):1737-1746.
PubMedArticle
27.
Biersteker  HAR, Andriessen  TMJC, Horn  J,  et al.  Factors influencing intracranial pressure monitoring guideline compliance and outcome after severe traumatic brain injury. Crit Care Med. 2012;40(6):1914-1922.
PubMedArticle
28.
Farahvar  A, Gerber  LM, Chiu  Y-L, Carney  N, Härtl  R, Ghajar  J.  Increased mortality in patients with severe traumatic brain injury treated without intracranial pressure monitoring. J Neurosurg. 2012;117(4):729-734.
PubMedArticle
29.
Talving  P, Karamanos  E, Teixeira  PG,  et al.  Intracranial pressure monitoring in severe head injury: compliance with Brain Trauma Foundation guidelines and effect on outcomes: a prospective study. J Neurosurg. 2013;119(5):1248-1254.
PubMedArticle
30.
Tang  A, Pandit  V, Fennell  V,  et al.  Intracranial pressure monitor in patients with traumatic brain injury. J Surg Res. 2015;194(2):565-570.
PubMedArticle
31.
Schreiber  MA, Aoki  N, Scott  BG, Beck  JR.  Determinants of mortality in patients with severe blunt head injury. Arch Surg. 2002;137(3):285-290.
PubMedArticle
32.
Farahvar  A, Gerber  LM, Chiu  Y-L,  et al.  Response to intracranial hypertension treatment as a predictor of death in patients with severe traumatic brain injury [published correction appears in J Neurosurg. 2011;115(1):191]. J Neurosurg. 2011;114(5):1471-1478.
PubMedArticle
33.
Karamanos  E, Teixeira  PG, Sivrikoz  E,  et al.  Intracranial pressure versus cerebral perfusion pressure as a marker of outcomes in severe head injury: a prospective evaluation. Am J Surg. 2014;208(3):363-371.
PubMedArticle
34.
Chesnut  RM, Temkin  N, Carney  N,  et al; Global Neurotrauma Research Group.  A trial of intracranial-pressure monitoring in traumatic brain injury. N Engl J Med. 2012;367(26):2471-2481.
PubMedArticle
35.
Khuri  SF, Najjar  SF, Daley  J,  et al.  Comparison of surgical outcomes between teaching and nonteaching hospitals in the Department of Veterans Affairs. Ann Surg. 2001;234(3):370-383.
PubMedArticle
36.
Ayanian  JZ, Weissman  JS.  Teaching hospitals and quality of care: a review of the literature. Milbank Q. 2002;80(3):569-593, v.
PubMedArticle
37.
Donabedian  A.  Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):166-206.
PubMedArticle
38.
Birkmeyer  JD, Dimick  JB, Birkmeyer  NJO.  Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg. 2004;198(4):626-632.
PubMedArticle
39.
Hawn  MT, Itani  KM, Gray  SH, Vick  CC, Henderson  W, Houston  TK.  Association of timely administration of prophylactic antibiotics for major surgical procedures and surgical site infection. J Am Coll Surg. 2008;206(5):814-819.
PubMedArticle
40.
Urbach  DR, Govindarajan  A, Saskin  R, Wilton  AS, Baxter  NN.  Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370(11):1029-1038.
PubMedArticle
41.
Berenholtz  SM, Pronovost  PJ, Lipsett  PA,  et al.  Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32(10):2014-2020.
PubMedArticle
42.
Pronovost  P, Needham  D, Berenholtz  S,  et al.  An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):2725-2732.
PubMedArticle
43.
Berwick  DM.  Measuring surgical outcomes for improvement: was Codman wrong? JAMA. 2015;313(5):469-470.
PubMedArticle
×