Helmick RA, Knofsky ML, Braxton CC, Subramanian A, Byers P, Lan CKW, Awad SS. Mandated Self-reporting of Ventilator-Associated Pneumonia Bundle and Catheter-Related Bloodstream Infection Bundle Compliance and Infection Rates. JAMA Surg. 2014;149(10):1003-1007. doi:10.1001/jamasurg.2014.1627
As quality measures increasingly become tied to payment, evaluating the most effective ways to provide high-quality care becomes more important.
To determine whether mandated reporting for ventilator and catheter bundle compliance is correlated with decreased infection rates, and to determine whether labor-intensive audits are correlated with compliance.
Design, Setting, and Participants
Multiyear retrospective review of aggregated data from all patients admitted to 15 intensive care units in a Veterans Affairs hospital setting (the Veterans Integrated Service Network 16) from 2009 to 2011.
Ventilator-associated pneumonia and catheter-related bloodstream infections.
Main Outcomes and Measures
Mean rates of ventilator-associated pneumonia and catheter-related bloodstream infection were analyzed by year. Relationships between infection rates, self-reported compliance, and audits were analyzed by Pearson correlation.
During the study period, ventilator-associated pneumonia decreased from 2.50 to 1.60 infections per 1000 ventilator days (P = .07). The rate of pneumonia was not correlated with self-reported compliance overall (R = 0.19) or by individual year (2009, R = 0.30; 2010, R = 0.24; 2011, R = 0.46); there was a correlation in cardiac intensive care units (R = −0.70) but not other types of intensive care units (mixed, R = −0.18; medical, R = 0.42; surgical, R = 0.34). Catheter-related bloodstream infections decreased from 2.38 to 0.73 infections per 1000 catheter days (P = .04). The rate of catheter infection was not correlated with self-reported compliance overall (R = −0.18), by individual year (2009, R = −0.39; 2010, R = −0.42; 2011, R = 0.37), or by intensive care unit type (mixed, R = −0.19; cardiac, R = 0.55; medical, R = 0.17; surgical, R = −0.44).
Conclusions and Relevance
Current mandated self-reported compliance and audit measures are poorly correlated with decreased ventilator-associated pneumonia or catheter-related bloodstream infection.
The management of critical care patients is often complicated by nosocomial infection, often in the illest patients who cannot afford the added complication of additional infection. Health care–associated infection (HCAI) is a significant problem in the United States and abroad,1 with catheter-related bloodstream infection (CRBSI) rates reported as high as 5.7 infections per 1000 catheter days and ventilator-associated pneumonia (VAP) rates of 15.2 infections per 1000 ventilator days.2,3 The overall burden of all HCAI is believed to account for 99 000 deaths and an additional cost of $28 to $33 billion annually.4 To address these complications, evidence-based best-practice bundles were developed by the Institute for Healthcare Improvement to minimize the occurrence of VAP and CRBSI (Box). Compliance audits have arisen as a quality check to ensure that bundles are being followed and to assess the postintervention infection response to bundle implementation.5 A significant amount of labor is required to perform these audits and check the self-reported compliance; however, debate exists as to whether these efforts realize actual reductions in VAP and CRBSI.6- 8
Daily sedation vacation
Peptic ulcer disease prophylaxis
Deep venous thrombosis prophylaxis
Oral care with chlorhexidine gluconate
Maximal barrier protections on insertion
Chlorhexidine gluconate skin antisepsis
Site selection with avoidance of femoral site
Daily review of catheter necessity
All elements are the original bundle elements instituted in 2005 except oral care with chlorhexidine gluconate in the ventilator bundle, which is an addition to the original bundle.
The VAP and CRBSI bundles were implemented in the Veterans Affairs health system in 2005 and 2006, respectively. Given the amount of time for these bundles to become established practice, we hypothesized that a high self-reported compliance would be associated with lower rates of VAP and CRBSI, while a high audit rate would identify bundle compliance failures.
Quarterly Inpatient Evaluation Center compliance reports are issued for each Veterans Integrated Service Network. We evaluated aggregated data from 15 intensive care units (ICUs) in the Veterans Integrated Service Network 16 between 2009 and 2011, separately analyzing VAP and CRBSI rates. From each report, we aggregated the year-end data for each ICU’s VAP, ventilator days, self-reported compliance, and audit rate and calculated the VAP rate per 1000 ventilator days. We also aggregated the number of CRBSIs, total catheter days, self-reported compliance, and audit rate and calculated the CRBSI rate per 1000 catheter days. Compliance was defined in the reports as the lowest bundle element compliance, and the audit rate was the number of audits per ventilator or catheter days.
Data were grouped by year and by type of ICU (medical, cardiac, surgical, and mixed). Mean values for VAP rate and CRBSI rate, self-reported compliance rate, and audit rate were calculated and t test was used to assess for statistical significance. P < .05 was considered statistically significant. Correlations between HCAI rate and compliance rate, HCAI rate and audit rate, and compliance rate and audit rate were calculated with Pearson correlation by year and by unit type for VAP and CRBSI.
This research protocol was reviewed by the Baylor College of Medicine Institutional Review Board and was deemed to not constitute human subjects research. As the information is deidentified, no authorization or waiver of authorization by patients for release of individually identifiable health protected information was required.
Of the 15 ICUs included in the study, there were 5 mixed ICUs, 2 cardiac ICUs, 4 medical ICUs, and 4 surgical ICUs. Across all units, total ventilator days numbered 12 804 in 2009, 11 930 in 2010, and 10 840 in 2011. The total number of VAP cases numbered 32 in 2009, 34 in 2010, and 20 in 2011. The total catheter days numbered 21 654 in 2009, 18 566 in 2010, and 17 411 in 2011, with the respective yearly total number of CRBSIs numbering 45, 38, and 16.
During the study period, the number of ventilator days and the VAP rate progressively declined (2.50 to 1.60 infections per 1000 ventilator days), although the difference between the start of the study and the conclusion did not reach statistical significance (VAP rate, P = .07). Similarly, the total use of central catheters decreased during the study period with a similar decrease in CRBSI rate (2.38 to 0.73 infections per 1000 catheter days); the yearly infection rate was statistically different between the start and end of the study (P = .04) (Table 1).
The self-reported VAP bundle and CRBSI bundle compliance did show statistically significant differences during the course of the study (P = .01 and P = .03, respectively), but not in a steadily improving fashion. Mean compliance for the VAP bundle decreased from 86.20% in 2009 to 81.10% in 2010, and the increase to 89.80% in 2011 accounted for the significant difference. Mean CRBSI bundle compliance significantly increased from 95.86% in 2009 to 98.31% in 2010 but was stable at 97.51% in 2011. The audit rates for the VAP bundles and CRBSI bundles were statistically equivalent during all study years (P = .76 and P = .38, respectively) (Table 1).
Overall, the VAP and CRBSI rates, self-reported compliance rates, and audit rates did not correlate with one another, as evidenced by R values less than 0.6. Individual correlation coefficients between VAP or CRBSI rate and compliance rate varied by year. Additionally, there was a negative relationship between CRBSI rate and bundle compliance (R = −0.18) (Table 2).
When correlations were analyzed based on the type of ICU, results were equally variable. Prevention of VAP in the cardiac ICU was the only scenario in which self-reported compliance was correlated with VAP rates, evidenced by an R value less than −0.60 (R = −0.70). No other type of ICU reached this level of significance for relationships between compliance, audits, and VAP or CRBSI rates. The Pearson correlation coefficient between CRBSI rates and compliance rates came close to a significant R value of 0.60 (R = 0.55), but it is in the positive direction (Table 3).
Health care–associated infections have been reduced in multiple hospitals after the introduction of both CRBSI bundles9,10 and VAP bundles.11,12 The benefits of decreased VAP and CRBSI are measurable with demonstrable cost savings.6,11 In this study, improvements in the VAP and CRBSI rates continued to be realized during the study period and were significantly lower than pre–bundle implementation VAP and CRBSI rates; however, there was neither an inverse correlation with self-reported compliance nor a strong correlation of either direction with auditing. If bundle compliance were strongly correlated with decreased incidence of HCAI, then higher compliance should realize lower VAP and CRBSI rates. A poor correlation between bundle compliance and HCAI contradicts previous studies that have demonstrated decreased infections with increased VAP and CRBSI bundle compliance.6,13
In an intervention/washout study, Cheema et al14 used VAP bundle checklists to assess for bundle compliance. During the intervention period, intensive efforts at bundle compliance were emphasized on a daily basis; during the washout period, the checklists were maintained without any emphasis or reminders. Using an intense bundle compliance checklist, the VAP rate decreased from 4.2 to 0.7 infections per 1000 ventilator days, but during a checklist washout period the infection rate increased to 4.8 infections per 1000 ventilator days. Similarly, Hawe et al15 studied a VAP bundle using a passive/active implementation regimen. They found that during the passive implementation period, compliance was poor and VAP rates remained high; with active implementation measures, though, compliance improved and the VAP rate decreased from 19.2 to 7.5 infections per 1000 ventilator days. The passive phase of this trial had 0% compliance for all bundle elements, which has been reported to carry an increased odds risk for VAP.16
Our study examines the VAP and CRBSI rates in a situation very similar to the washout period of the study by Cheema and colleagues. The VAP and CRBSI bundle practices have been in place at the Veterans Affairs hospitals since 2006 and are part of routine care with mandated self-reported compliance rates, audit rates, and self-reported infection rates. This difference raises the question of a Hawthorne effect being responsible for previously demonstrated correlations between bundle compliance and decreased infections during study periods. Two reviews by Lawrence and Fulbrook7 and Zilberberg et al8 raise similar concerns over the body of literature regarding VAP bundles specifically. Particular attention was paid to the issue of bias—both publication bias and the nonblinded nature of bundle studies.
The steady decrease in the total number of ventilator days and catheter days during the study period is notable. A similar phenomenon was seen in the interventional CRBSI bundle study by Kim et al17; while the bundle intervention period was 3 months longer than the baseline period, the total number of catheter days was 12 000 fewer than in the observation period. Decreased central catheter and ventilator use certainly has an effect on lower infection rates, but decreased device use has been previously reported to contribute only part of the decrease in HCAI.18 Nevertheless, fewer ventilator days and fewer catheter days permit fewer opportunities for infectious complications.
The variability of correlation between HCAI rates and bundle compliance or audits across different types of ICUs is intriguing. Across all relationships and units, only VAP was found to be correlated with bundle compliance in the cardiac ICU with an R value of −0.70 (Table 3). We hypothesize that this strong relationship may be seen in a cardiac ICU setting because of differences between ventilated patients in the cardiac ICU vs medical or surgical ICU; however, further research is needed to support this theory. Otherwise, no other relationships reached a level of significance evidenced by an R value of greater than or equal to 0.60 or less than or equal to −0.60. While a prior report identifies a difference between adult and pediatric ICU central catheter bundle compliance and infection rates,19 no published reports to our knowledge make comparisons between infection rates and compliance in different types of adult ICUs. Again, the differences seen between the cardiac, medical, and surgical ICUs warrant further study.
While a strong body of evidence exists supporting the effect of bundle implementation on decreased rates of VAP and CRBSI, comparative studies examining the best collection of bundle practices are lacking. The Institute for Healthcare Improvement VAP bundle consists of 5 elements including semirecumbent positioning, deep venous thrombosis prophylaxis, peptic ulcer prophylaxis, daily sedation vacations, and daily assessment of weaning readiness. Two of the 5 bundle elements do not specifically address ventilator issues but rather the complications that are often seen with long-term ventilation, while a previously published review found no strong evidence for head-of-bed elevation recommendations.20 Additional evidence has been published indicating a strong decrease in VAP rates when respiratory therapists perform aggressive oral care.21,22 Debate is present regarding the best CRBSI bundle elements, as multiple published trials each report variations in the bundle elements used,9,17,23 while the widely accepted practice of central catheter site maintenance and dressings was not found to be correlated with CRBSI rates.24 A study by Furuya et al25 demonstrated improvement when at least 1 bundle element was implemented but could not demonstrate added benefit by use of the entire bundle.
Our data further fail to demonstrate a correlation between auditing, a labor- and cost-intensive process, and improvements either in bundle compliance or in VAP or CRBSI rates. While the costs of treating cases of VAP or CRBSI are frequently described, very little is written about the costs associated with bundle implementation. An analysis by Halton et al26 examined the cost-effectiveness of CRBSI bundles, finding that cost-effectiveness breaking points depend on the use of standard vs antibiotic-impregnated catheters, but no similar analysis exists for the VAP bundles.4 Ultimately the goals of the Institute for Healthcare Improvement campaign and bundles are to reduce infection and save lives; however, the emphasis currently seems focused on compliance reporting and auditing to ensure compliance. There are reports of alternative techniques such as daily bundle rounds or checklists, nursing-driven morbidity and mortality type reviews, and electronic bundle dashboards that all increase bundle compliance and have been shown to decrease VAP rates.3,27- 29
The rates of VAP and CRBSI have improved within the Veteran Affairs ICUs studied, which is attributable to effects of some of the bundle elements. Individual bundle elements are shown to reduce infection rates, and while data to support the optimal package of bundle elements that will most significantly reduce infection rates are not available, we believe that the bundle elements as outlined do help to reduce infection. However, our data do not support a correlation between audits or self-reported compliance and reduced infection. Given the current emphasis on effective and high-value health care, further research efforts should be focused on the most effective combination of bundle elements and should use more efficient compliance metrics instead of labor-intensive processes.
Corresponding Author: Samir S. Awad, MD, Michael E. DeBakey Veterans Affairs Medical Center, 2002 Holcombe Blvd, OCL (112), Room 5A-350, Houston, TX 77030 (firstname.lastname@example.org).
Accepted for Publication: August 6, 2013.
Published Online: August 27, 2014. doi:10.1001/jamasurg.2014.1627.
Author Contributions: Dr Helmick had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Byers, Awad.
Acquisition, analysis, or interpretation of data: Helmick, Knofsky, Braxton, Subramanian, Lan, Awad.
Drafting of the manuscript: Helmick, Byers, Awad.
Critical revision of the manuscript for important intellectual content: Helmick, Knofsky, Braxton, Subramanian, Lan, Awad.
Statistical analysis: Helmick, Awad.
Obtained funding: Awad.
Administrative, technical, or material support: Knofsky, Subramanian, Lan, Awad.
Study supervision: Braxton, Awad.
Conflict of Interest Disclosures: None reported.
Previous Presentation: This work was presented at the 2013 Annual Meeting of the Association of VA Surgeons; April 21, 2013; Milwaukee, Wisconsin.