Effect of Published Scientific Evidence on Glycemic Control in Adult Intensive Care Units

Importance  Little is known about the deadoption of ineffective or harmful clinical practices. A large clinical trial (the Normoglycemia in Intensive Care Evaluation and Survival Using Glucose Algorithm Regulation [NICE-SUGAR] trial) demonstrated that strict blood glucose control (tight glycemic control) in patients admitted to adult intensive care units (ICUs) should be deadopted; however, it is unknown whether deadoption occurred and how it compared with the initial adoption.

Objective  To evaluate glycemic control in critically ill patients before and after the publication of clinical trials that initially suggested that tight glycemic control reduced mortality (Leuven I) but subsequently demonstrated that it increased mortality (NICE-SUGAR).

Design, Setting, and Participants  Interrupted time-series analysis of 353 464 patients admitted to 113 adult ICUs from January 1, 2001, through December 31, 2012, in the United States using data from the Acute Physiology and Chronic Health Evaluation database.

Main Outcomes and Measures  The physiologically most extreme blood glucose level on day 1 of ICU admission defined glycemic control as tight control (glucose level, 80-110 mg/dL; to convert to millimoles per liter, multiply by 0.0555), hypoglycemia (glucose level, <70 mg/dL), and hyperglycemia (glucose level, ≥180 mg/dL). Temporal changes in each marker were examined using mixed-effects segmented linear regression.

Results  Before the publication of Leuven I, 17.2% (95% CI, 16.2%-18.2%) of ICU admissions had tight glycemic control, 3.0% (95% CI, 2.6%-3.5%) had hypoglycemia, and 40.2% (95% CI, 38.8%-41.5%) had hyperglycemia. After the publication of Leuven I, there were significant increases in the relative proportion of admissions with tight glycemic control (1.7% per quarter; 95% CI, 1.2%-2.3%; P < .001) and hypoglycemia (2.5% per quarter; 95% CI, 1.9%-3.2%; P < .001) and decreases in those with hyperglycemia (0.6% per quarter; 95% CI, 0.4%-0.9%; P < .001). Following the publication of NICE-SUGAR, there was no change in the proportion of patients with tight glycemic control or hyperglycemia. There was an immediate decrease in the relative proportion of patients with hypoglycemia (22.4%; 95% CI, 13.2%-30.1%; P < .001) but no subsequent change over time.

Conclusions and Relevance  Among patients admitted to adult ICUs in the United States, there was a slow steady adoption of tight glycemic control following publication of a clinical trial that suggested benefit, with little to no deadoption following a subsequent trial that demonstrated harm. There is an urgent need to understand and promote the deadoption of ineffective clinical practices.

Implementing best practices into bedside clinical care improves the quality and value of health care. This change includes adopting clinical practices with evidence of effectiveness and deadopting practices with evidence of ineffectiveness or harm. However, research shows that health care systems routinely fail to make effective use of evidence.¹ The Institute of Medicine cites a period of approximately 17 years for scientific evidence from randomized clinical trials to be implemented into routine care.² This estimate refers to the time required to adopt a new intervention; it is unclear if a similar time is required to deadopt ineffective or harmful clinical practices.

Tight glycemic control, the practice of maintaining blood glucose levels between 80 and 110 mg/dL (to convert to millimoles per liter, multiply by 0.0555) using intensive intravenous insulin, provides an important example of a clinical practice (not a pharmaceutical product or technology) for which a sentinel publication suggested benefit (ie, supported adoption) and subsequent confirmatory research demonstrated harm (ie, supported deadoption).^3-5 The rationale for tight glycemic control is based on laboratory and observational data that demonstrated an association between hyperglycemia, increased risk of infection, and decreased survival.⁶ The first large randomized clinical trial (Leuven I)³ demonstrated that providing tight glycemic control to a cohort of predominantly surgical patients admitted to intensive care units (ICUs) compared with conventional glucose control (blood glucose level, <200 mg/dL) resulted in 1 life saved for every 29 patients treated. These data resulted in calls for the widespread adoption of tight glycemic control among critically ill surgical patients, although many institutions applied tight glycemic control to all critically ill patients.^7-9 Subsequently, the Normoglycemia in Intensive Care Evaluation and Survival Using Glucose Algorithm Regulation (NICE-SUGAR) study, the largest multinational randomized clinical trial to examine tight glycemic control in the broadest cohort of medical and surgical ICU patients, demonstrated that tight glycemic control increased the risk of developing severe hypoglycemia and 90-day mortality.⁴ These data resulted in major international guidelines modifying their recommendations for the management of blood glucose in critically ill patients.^10,11

In our study, we take advantage of this natural experiment of published scientific evidence by using a large multicenter database to compare the adoption and deadoption of best practices into clinical care by examining glucose control practices in adult ICUs in the United States following the publication of Leuven I and NICE-SUGAR trials.

We conducted an interrupted time-series analysis using data from the Acute Physiology and Chronic Health Evaluation (APACHE) database to determine whether Leuven I (supporting adoption) and NICE-SUGAR (supporting deadoption) influenced the practice of glycemic control in patients admitted to adult ICUs. The APACHE database (Cerner Corporation) contains prospectively collected clinical, physiologic, and outcome data on patients admitted to adult ICUs that use the APACHE clinical information system for benchmarking and quality improvement.^12-14 Patient data were anonymized before analysis. The Conjoint Health Research Ethics Board at the University of Calgary approved this study.

We included patients admitted to adult ICUs from January 1, 2001, through December 31, 2012. We excluded patients without primary outcome data (blood glucose measurement within the first 24 hours of ICU admission), or those admitted following solid organ transplantation (APACHE score not a valid predictor of outcomes in these patients). Intensive care units that contributed fewer than 1000 admissions during the study period (ICUs entered and exited the study cohort during the 12-year period) were excluded to ensure the validity and generalizability of the results.¹⁵

The main exposure variables were the date of publication of Leuven I (November 8, 2001) and NICE-SUGAR (March 24, 2009). These trials were selected because both were published in a general medical journal with a high impact factor that was likely to have a high readership among ICU health care professionals and they represented the first and largest randomized clinical trials to examine tight glycemic control in adult ICUs. As such, these 2 studies were most likely to change clinical practice.

Blood glucose data, as selected by the APACHE algorithm, on day 1 of ICU admission were used to define the measures of glycemic control. As shown in eTable 1 in the Supplement, APACHE preferentially documents the physiologically most extreme glucose level. We selected day 1 of ICU admission to define glycemic control because most of the institutions that collected physiologic data for APACHE did so only for the first 24 hours after ICU admission and previous research demonstrated that the mean blood glucose level within the first day of ICU admission is a reliable surrogate of overall glycemic control.¹⁶ In addition, any potential bias introduced by using day 1 glucose is likely to be nondifferential with respect to time, which was the major exposure in this study. Glycemic control was defined as (1) tight glycemic control⁵ (glucose level, 80-110 mg/dL), (2) hypoglycemia (glucose level, <70 mg/dL), and (3) hyperglycemia (glucose level, ≥180 mg/dL).

Data are reported as mean (SD), median (interquartile range), or proportion (95% CI). To test the hypothesis that influential articles exerted changes on glucose control, we used mixed-effects segmented linear regression. Data from individual ICU admissions were aggregated for 3-month intervals (ie, quarters) and analyzed at the level of the ICU.^17-19 We developed separate models for each marker of glucose control (tight glycemic control, hypoglycemia, and hyperglycemia). The distribution of each glycemic control outcome was examined using histograms. Skewed data were transformed using logarithms for analysis and reported as a relative change in the proportion of admissions with the given measure of glucose control. Initial models included a term that estimated changes in the baseline trend over time and terms that estimated changes in the level and trend of the measure of glycemic control after publication of Leuven I and NICE-SUGAR, respectively.¹⁹ All models accounted for inter-ICU variability in baseline glucose control and variability in the longitudinal trends in glucose control. Each model was fit using robust variance estimates and accounted for autocorrelation between adjacent time points. We prespecified that all models be adjusted for changes in the proportion of patients with diabetes mellitus. The effect of other potentially confounding variables was assessed through backward stepwise variable elimination (P > .05 for variable elimination).

To test the hypothesis that patient and ICU factors modified the effect that influential articles may have had on glucose control, we used mixed-effects multivariable logistic regression. The effect of each predictor on the quarterly odds of tight glycemic control was evaluated by examining the significance of a term that represented the interaction between the predictor and time. Results were adjusted for inter-ICU variability and quarterly differences in patient characteristics. The following prespecified predictors were evaluated: (1) history of diabetes; (2) surgical admission diagnosis (vs medical); (3) diagnosis of sepsis at admission (vs nonseptic); (4) APACHE III/IV score of 50 or greater (vs APACHE score of <50); (5) type of ICU (medical, surgical, mixed medical/surgical); (6) teaching status (teaching vs nonteaching institution); and (7) annual ICU patient volume (low, intermediate, or high).

Additional analyses were conducted to examine the validity of the study’s main assumptions. We used the Bland-Altman method to determine whether day 1 glucose level was a reliable surrogate for overall glycemic control in the subset of patients for which the average glucose level during the first week in the ICU could be calculated.¹⁶ We examined whether changes in glycemic control differed according to whether day 1 vs day 2 blood glucose values were used to define glycemic control. We also evaluated whether differential dropout or enrollment of ICUs included in the study population introduced bias by limiting the primary analysis to ICUs that contributed data throughout the study period (a subset of those included in the main study population). Finally, we evaluated whether changes in glycemic control were robust to different measures of time by repeating the primary analysis using monthly instead of quarterly time intervals. All analyses were conducted using Stata, version 13.1 (StataCorp LP) and statistical significance was set at P < .01.

From January 1, 2001, through December 31, 2012, the APACHE database captured data on 490 922 admissions to 195 ICUs in 80 hospitals. After excluding admissions without a measurement for day 1 glucose (n = 71 347), patients admitted following solid organ transplantation (n = 17 083), and patients admitted to ICUs that contributed fewer than 1000 admissions during the study period (n = 24 631), the final study population consisted of 377 861 admissions (353 464 patients) to 113 ICUs in 56 hospitals. Patients were predominantly male (56.0%), with a mean (SD) age of 61.8 (17.1) years and a mean (SD) APACHE III/IV score of 55.2 (27.8). Most had a medical admission diagnosis (66.6%) and were admitted for an active life-supporting treatment (65.9%).²⁰ The ICU types were a diverse mix and most were in hospitals with a teaching affiliation (75.2%).

A comparison of patient and ICU characteristics throughout the study period demonstrated temporal variability in the mean APACHE III/IV score, the proportion of patients who received an active life-supporting treatment on ICU admission, the proportion of patients with diabetes, and the proportion of medical patients (Table 1). The ICU-level characteristics did not change significantly during the study.

Before the publication of Leuven I, 17.2% (95% CI, 16.2%-18.2%) of ICU admissions had tight glycemic control, 3.0% (95% CI, 2.6%-3.5%) had hypoglycemia, and 40.2% (95% CI, 38.8%-41.5%) had hyperglycemia (Table 2). As shown in Figure 1, after the publication of Leuven I, there were significant increases over time in the relative proportion of patients with tight glycemic control (1.7% per quarter; 95% CI, 1.2%-2.3%; P < .001) and hypoglycemia (2.5% per quarter; 95% CI, 1.9%-3.2%; P < .001) and significant decreases in the proportion of patients with hyperglycemia (0.6% per quarter; 95% CI, 0.4%-0.9%; P < .001). Following the publication of NICE-SUGAR, there was no change in the proportion of patients with tight glycemic control or hyperglycemia. There was an immediate decrease in the relative proportion of patients with hypoglycemia (22.4%; 95% CI, 13.2%-30.1%; P < .001) but no subsequent change over time. At the end of the study, 27.5% (95% CI, 26.5%-28.5%) of admissions had tight glycemic control, 5.2% (95% CI, 4.6%-5.7%) had hypoglycemia, and 33.0% (95% CI, 31.9%-34.1%) had hyperglycemia.

Patient and ICU-level predictors of tight glycemic control are shown in Figure 2. Following the publication of Leuven I, patient factors associated with an increase in tight glycemic control over time included the patient’s history of diabetes, surgical admission diagnosis, nonseptic diagnosis, and active intervention on day 1 in the ICU. Institutional factors associated with an increase in tight glycemic control included being a surgical or mixed medical/surgical or nonteaching ICU and intermediate annualized ICU volume. Following the publication of NICE-SUGAR, the only patient and institutional factors associated with a reduction in tight glycemic control were a history of diabetes and admission to a teaching ICU.

Bland-Altman plots (eFigure 1 in the Supplement) demonstrated that agreement between the physiologically most-extreme day 1 glucose level and the mean glucose level during the first week in the ICU was best for aggregate ICU-level data and for individual patient data when the day 1 glucose level was between 40 mg/dL and 360 mg/dL. Note that 95.7% of the study population had a day 1 glucose level of 360 mg/dL or less. Changes in tight glycemic control were similar whether glycemic control was defined using day 1 or day 2 blood glucose levels (eTable 2 and eFigure 2 in the Supplement). In addition, changes in tight glycemic control and hypoglycemia were similar in the subgroup of 10 ICUs (46 718 admissions) that contributed data throughout the study (eFigure 3 in the Supplement). However, in this subgroup, the proportion of patients with hyperglycemia remained constant throughout the study. As shown in eFigure 4 in the Supplement, there was considerable heterogeneity between ICUs with respect to time-dependent changes in tight glycemic control. Changes in glycemic control were similar whether time was measured in months or quarters. Mortality rates in the ICU decreased from 9.8% (95% CI, 9.0%-10.5%) at the beginning of the study (first study quarter) to 7.6% (95% CI, 7.0%-8.2%) at the end (final study quarter), with no significant variation related to the publication of Leuven I or NICE-SUGAR.

In this study, we found notable changes in the practice of glycemic control following a clinical trial that demonstrated the benefits of tight glycemic control (Leuven I), with comparatively less change following a methodologically more rigorous trial that demonstrated its harmful effects (NICE-SUGAR). Among patients admitted to adult ICUs in the United States, the publication of Leuven I was associated with statistically significant increases in tight glycemic control and hypoglycemia and a concomitant decrease in the burden of hyperglycemia (benefit of tight control). The publication of NICE-SUGAR was associated with an immediate reduction in hypoglycemia but no change in tight glycemic control, hypoglycemia, or hyperglycemia over time, suggesting that any potential early deadoption signified by the early decrease in hypoglycemia was not sustained. To our knowledge, this is the first study to compare adoption of a clinical practice following publication of a clinical trial that suggested a benefit with its deadoption after subsequent publication of a confirmatory trial that demonstrated harm. As such, it provides useful insights into the way health systems practice and respond to evidence.

Most studies^22-26 that have examined the adoption of evidence-based interventions pertain to therapies for patients with cardiovascular disease (eg, congestive heart failure). These studies generally demonstrate early adoption following positive clinical trials that are published in general medical journals with a high impact factor.^22-26 In contrast, our study demonstrates that tight glycemic control was slowly adopted after publication of Leuven I (absolute increase of only 5% in approximately 7 years) (Table 2). However, this finding is consistent with previous critical care research demonstrating slow adoption of lung-protective ventilation for patients with acute respiratory distress syndrome after publication of a large clinical trial supporting its use.^27-30 Although there is considerable overlap in patient care practices applied in cardiology and critical care, these studies highlight important differences in their respective tendency to adopt new interventions, with critical care generally lagging behind cardiology. The reasons for these differences in adoption practices by specialty are unclear; however, possible explanations include differences in intervention complexity (eg, providing lung protective ventilation for patients with acute respiratory distress syndrome vs prescribing aspirin for patients with coronary artery disease), and/or whether the intervention is proprietary.^25,31

When compared with other studies, this study also underscores the variability in the deadoption of ineffective interventions. Nesiritide (an intravenous vasodilator) was rapidly deadopted for use in patients with congestive heart failure following the publication of 2 meta-analyses that demonstrated its harmful effects.³² In contrast, there was no change in the use of percutaneous coronary intervention after publication of the Occluded Artery Trial, a large, multicenter, randomized clinical trial that demonstrated no benefit associated with percutaneous coronary intervention in patients with persistent occlusion of the infarct-related artery after a myocardial infarction.³³ On the other hand, use of the pulmonary artery catheter declined considerably during the past decade after several randomized clinical trials and meta-analyses failed to demonstrate a benefit when compared with less-invasive monitoring techniques.^15,18,34-36 Finally, Kaukonen and colleagues³⁷ recently demonstrated little change in the practice of glycemic control after the NICE-SUGAR trial in patients who were admitted to adult ICUs in Australia and New Zealand. However, none of the ICUs included in the study used tight glycemic control during the baseline period and nearly 20% participated in the NICE-SUGAR trial. Therefore, the lack of deadoption following NICE-SUGAR may reflect similarities between the baseline approach to glycemic control and the control arm of NICE-SUGAR. In contrast, none of the ICUs in our study were involved with NICE-SUGAR.

The results of this study add to the broader discipline of implementation science. First, the example of tight glycemic control in critically ill patients highlights the importance of confirmatory research in the evolution of any clinical practice. Scientific evidence slowly and passively diffuses through the literature, with early adopters of an intervention adopting new clinical practices after publication of high-profile positive clinical trials.^31,38 The experience with tight glycemic control illustrates the importance of seeking confirmatory research before widespread promotion of that clinical practice through active strategies of implementation. Second, the increase in hypoglycemia that accompanied the adoption of tight glycemic control after the publication of Leuven I suggests that interventions that facilitate the adoption of a new clinical practice should include measures to monitor and mitigate the potential harms associated with that clinical practice (eg, mandating prescription of a source of glucose for patients who are treated with intensive intravenous insulin). Third, attenuation by NICE-SUGAR of the decrease in hyperglycemia that followed the publication of Leuven I with no change in the incidence of tight glycemic control demonstrates that strategies promoting the deadoption of ineffective clinical practice should guard against potential misinterpretation of reversal evidence (eg, greater tolerance for hyperglycemia after the publication of NICE-SUGAR).³⁹ Fourth, the subgroup and sensitivity analyses presented in Figure 2 and eFigure 4 in the Supplement, respectively, indicate that both patient and institutional factors may differentially affect adoption and deadoption and interventions aimed at changing clinical practice will need to be tailored to the underlying practice and the desired direction of practice change. Finally, our data suggest that adoption of a clinical practice may occur more rapidly than deadoption and successful deadoption will require an active change model (eg, knowledge-to-action framework⁴⁰) rather than the more traditional passive diffusion of evidence.

The main strength of this study pertains to the generalizability of the results and their low risk of bias. We analyzed prospectively collected data in more than 350 000 patients and more than 100 ICUs during a 12-year period. We excluded ICUs that did not contribute meaningfully to the overall study population by setting a minimum requirement of 1000 admissions during the study. Furthermore, none of the ICUs that contributed data to the APACHE database were participants in either Leuven I or NICE-SUGAR. In addition, we examined a nonproprietary practice for which its use was unlikely to be heavily influenced by industry or regulation. Regarding the analysis, we used the strong quasiexperimental technique of interrupted time series analysis to separately analyze the relationship between each clinical trial and time trends in glucose control.

The main weakness of this study is the lack of ICU-level data pertaining to the implementation or modification of protocols to control blood glucose values before and after publication of Leuven I and NICE-SUGAR. We had to infer a strategy of tight glycemic control from the absolute value of day 1 glucose level but were unable to link this value to the use of intravenous insulin protocols. While specific insulin doses provided in response to glucose levels over time would have given us a more sensitive measure of adoption and deadoption, it is unlikely that the proxy we used—day 1 glucose level—introduced bias. When combined with the results of previous research,¹⁶ the Bland-Altman and sensitivity analyses performed in this study suggest that a day 1 glucose level is a reasonable surrogate marker of overall glucose control for most patients in the APACHE database. While it is possible that analyses in other databases may not demonstrate a similar agreement, any decrease in the strength of the association between day 1 glucose and overall glucose control is likely to be nondifferential with respect to time, suggesting that our analyses, if anything, are conservative. Furthermore, the use of day 1 glucose levels may also partially reflect care that was provided to patients before ICU admission (ie, emergency department, hospital ward, or operating room), highlighting that they are likely conservative estimates and potentially reflect changes in the way health care systems practice and respond to evidence. Additional limitations included a lack of data on health care professionals that precluded analyses that examined the effect of health care professionals’ characteristics on glucose control practices, limited data describing the characteristics of the different ICUs, and limited baseline glucose control data before publication of Leuven I. Finally, it is possible that other activities may have occurred during the study that could have affected adoption and deadoption of tight glycemic control (eg, publication of the original Surviving Sepsis Campaign Guidelines⁴¹); however, the data do not demonstrate any abrupt changes in practices that may be potentially related to another factor.

Among patients admitted to adult ICUs in the United States, there was slow adoption of tight glycemic control following an initial clinical trial that suggested that it reduced mortality (Leuven I), with little to no deadoption following a confirmatory trial that demonstrated that it increased mortality (NICE-SUGAR). This finding suggests that tight glycemic control has not been deadopted and there is an urgent need to understand and promote the deadoption of ineffective clinical practices.

Accepted for Publication: December 21, 2014.

Corresponding Author: Daniel J. Niven, MD, MSc, Department of Critical Care Medicine, University of Calgary, Peter Lougheed Centre, 3500 26th Ave NE, Calgary, AB T1Y 6J4, Canada (daniel.niven@albertahealthservices.ca).

Published Online: March 16, 2015. doi:10.1001/jamainternmed.2015.0157.

Author Contributions: Dr Niven had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Niven, Kramer.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Niven, Rubenfeld, Kramer.

Obtained funding: Niven.

Administrative, technical, or material support: Rubenfeld.

Study supervision: Rubenfeld, Stelfox.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was funded by an operating grant from the Canadian Intensive Care Foundation; a Clinician Fellowship Award from Alberta Innovates: Health Solutions and a Knowledge Translation Canada Student Fellowship and Training Program grant (Dr Niven); and a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates: Health Solutions (Dr Stelfox).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: Brenda Hemmelgarn, MD, PhD, FRCPC, University of Calgary, and Sharon Straus, MD, MSc, FRCPC, University of Toronto, provided comments on an earlier version of the manuscript. Neither were financially compensated.