Point score and risk stratification for pneumonia due to a resistant pathogen. P < .001 for trend.
Shorr AF, Zilberberg MD, Micek ST, Kollef MH. Prediction of Infection Due to Antibiotic-Resistant Bacteria by Select Risk Factors for Health Care–Associated Pneumonia. Arch Intern Med. 2008;168(20):2205-2210. doi:10.1001/archinte.168.20.2205
Copyright 2008 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2008
Pathogens such as methicillin-resistant Staphylococcus aureus (MRSA) and Pseudomonas aeruginosa now cause pneumonia in patients presenting to the hospital. The concept of health care–associated pneumonia (HCAP) attempts to capture this, but its predictive value is unclear.
We examined patients admitted with pneumonia; infection with a resistant pathogen served as the study end point. Health care–associated pneumonia was present if a patient met one of the following criteria: recent hospitalization, nursing home residence, long-term hemodialysis, or immunosuppression. We compared rates of resistant infection among patients meeting any criteria for HCAP with those who did not have HCAP and explored the individual components of the definition.
Among the cohort (n = 639), resistant pathogens were recovered in 289 (45.2%). Although each component of HCAP occurred more frequently in persons with resistant infections, the broad definition had a specificity of only 48.6% and misclassified one-third of the subjects. Logistic regression showed 4 variables associated with resistant pneumonia: recent hospitalization, nursing home residence, hemodialysis, and intensive care unit admission. A scoring system assigning 4, 3, 2, and 1 points, respectively, for each variable had moderate predictive power for segregating those with and without resistant bacteria. Among patients with fewer than 3 points, the prevalence of resistant pathogens was less than 20% compared with 55% and more than 75% in persons with scores ranging from 3 to 5 and more than 5 points, respectively (P < .001).
Although resistance is common in HCAP, not all component criteria for HCAP convey similar risk. Simple scoring tools may facilitate more accurate identification of persons with pneumonia caused by resistant pathogens.
Infections caused by resistant pathogens generally have been confined to hospitals. Risk for infection with organisms such as methicillin-resistant Staphylococcus aureus (MRSA) and Pseudomonas aeruginosa was thought to be limited to persons who had been hospitalized for acute illness for several days. However, with the diffusion of health care delivery and technology beyond the hospital, these pathogens have spread past the confines of an inpatient setting. Multiple reports specifically describe the increasing prevalence of such resistant organisms in subjects presenting de novo to the hospital.1- 3
This observation regarding increasing rates of resistant infections in persons arriving in hospital emergency departments (EDs) led to the evolution of the concept of health care–associated infection (HAI). Health care–associated infections are thought to represent a middle ground between hospital-onset infections and pure community-onset processes.1- 3 In other words, the motivating force driving the creation of the HAI categorization is recognition that risk factors for infection with resistant pathogens reflect particular patient characteristics and not purely the patient's location at the time of infection diagnosis. Much work developing and supporting the HAI syndrome has addressed pneumonia because this is a leading cause of infectious morbidity and mortality.1,4 Several analyses clearly identify health care–associated pneumonia (HCAP) as a distinct entity with unique microbiologic features and discrete clinical outcomes that more closely resemble those of traditional nosocomial pneumonia than community-acquired pneumonia.1,4 To this end, recent formal guidelines from the American Thoracic Society and the Infectious Disease Society of America for nosocomial pneumonia and community-acquired pneumonia indicate that HCAP mimics nosocomial pneumonia more in its bacteriologic features and that patients with risk factors for HCAP require broader-spectrum initial antimicrobial therapy.5,6
Unfortunately, many of the criteria used to define HCAP are arbitrary and based mainly on expert opinion. Commonly used patient factors proposed to indicate that a pneumonia reflects HCAP rather than community-acquired pneumonia include residence in a nursing home, recent hospitalization, prior antimicrobial exposure, the need for long-term hemodialysis, and underlying immunosuppression.5,6 One concern regarding the HCAP concept as currently envisioned is that it may lead to overuse of broad-spectrum antimicrobials in persons who do not have underlying infections with resistant organisms. Although the criteria defining HCAP may suggest that an individual is at heightened risk for infection with MRSA or P aeruginosa, these criteria do not necessarily afford sufficient precision. In turn, although some patients with HCAP will be more likely to receive therapy with agents that are active in vitro against the infecting pathogen because their therapy was not limited to community-acquired pneumonia pathogens, others may needlessly be exposed to the risks and costs related to use of overly broad antimicrobials. Simply put, physicians require more effective risk stratification tools to allow them to balance these 2 competing pressures.
We hypothesized that each of the unique criteria used to diagnose HCAP carried a differential risk for actual infection with a resistant pathogen. We also theorized that HCAP may be too broad as currently conceptualized and that better risk-scoring tools could be developed to identify subjects presenting to the ED with pneumonia due to resistant organisms.
We conducted a retrospective review of all patients admitted with pneumonia to a large, urban, tertiary care teaching hospital. We focused on the 3-year period from January 1, 2003, to December 31, 2005. All subjects initially presented to the ED. The hospital's institutional review board approved this study. Initial data from a component of this analysis have been published earlier.7
To be included in the study population, patients had to present to the ED with a syndrome consistent with bacterial pneumonia. More specifically, our definition of pneumonia was based on discharge diagnosis coding from the International Classification of Diseases, Ninth Revision. Beyond the discharge diagnosis, we required subjects to have at least 2 signs and symptoms of pneumonia when presenting (eg, fever or hypothermia, sputum production, cough, and elevated or depressed leukocyte count). We also mandated the presence of a new infiltrate on radiographic imaging results. One investigator (M.H.K.) reviewed all of the cases and imaging studies to confirm that each patient met these inclusion criteria. In addition, we restricted the analysis to individuals with microbiologic evidence of infection. Hence, we required subjects to have sputum or lower airway cultures that were positive for a pathogen. For lower airway cultures, we used quantitative methods. The investigator reviewing the medical record also confirmed that the pathogens recovered from the sputum culture represented the likely culprit organism. Alternatively, persons with blood or pleural cultures yielding a pathogen or urinary antigen findings that were positive for Streptococcus pneumonia or Legionella species were included. We excluded subjects without any positive culture findings or similar evidence of infection, and we also excluded persons younger than 18 years.
The presence of an infection with a resistant pathogen served as our primary end point. We specifically examined pathogens traditionally categorized as nosocomial. The particular pathogens that constituted this cohort of resistant organisms included MRSA, P aeruginosa, extended-spectrum β-lactamase–producing Klebsiella species, and other nonfermenting gram-negative rods. We completed a subgroup analysis restricting our attention to infection with MRSA alone.
We defined HCAP as being present if a patient met at least 1 of the following 4 criteria: recent hospitalization in the past 90 days, residence in a nursing home or a long-term care facility, long-term treatment with hemodialysis, or underlying immunosuppression. Immunosuppression was identified by the presence of neutropenia, concurrent use of an oral corticosteroid or other immunosuppressive agent (eg, cyclosporine), active chemotherapy for malignancy, or infection with human immunodeficiency virus. We required at least 5 days of corticosteroid exposure to meet criteria for immunosuppression. In addition, we recorded information regarding patient demographics (ie, age, sex, and race) and severity of illness based on the need for admission to the intensive care unit (ICU) or mechanical ventilation.
We compared rates of resistant infection among patients meeting any criteria for HCAP with those not fulfilling our HCAP designation and then explored the component parts of the definition. To compare categorical variables, we used the χ2 test and the Fisher exact test. For continuous variables we used the t test or nonparametric tests as appropriate. All tests were 2-tailed, and we assumed statistical significance if P < .05.
To identify factors independently associated with the presence of resistant infection, we relied on logistic regression. Variables significant at an α level of .15 that were considered biologically relevant were entered into the regression model. We assessed variables for colinearity and explored goodness of fit based on the Hosmer-Lemeshow test. To detect the presence of potential interaction, we used the Breslow-Day method and tested for interactions between the terms within the logistic model we developed. To assess for overfitting, we performed cross-validation. We reran the logistic model on 90% of the sample sequentially, dropping 10% of the population in each run. We then compared the mean accuracy of these analyses with the overall accuracy of the model seen with the entire cohort.
From the logistic regression findings, we created a predictive scoring tool to identify persons with pneumonia due to resistant infection. We converted the β coefficients from the logistic regression into whole integers representing points. We then explored the predictive value of the point score for correctly indicating the presence of infection with a resistant pathogen via a receiver operating characteristic (ROC) curve. Various break points for the score that grouped patients into cohorts at high risk for resistant infection vs non–high-risk cohorts were compared by determining differing areas under the ROC curve.
The entire cohort included 639 subjects. Resistant pathogens were identified in 289 cases (45.2%). Some of the infections were polymicrobial. The most common resistant pathogen was MRSA (n = 157), followed by P aeruginosa (n = 120). The remaining 47 organisms were extended-spectrum β-lactamase Klebsiella species or nonfermenting gram-negative rods. The most frequent nonresistant pathogen was S pneumoniae (n = 130).
Table 1 shows the clinical characteristics of patients with and without resistant infections. We did not observe differences in the distribution of demographic features based on whether the culprit pathogen was or was not resistant. However, patients with resistant infection were more likely to meet at least 1 of the criteria for HCAP. For example, 86.9% of those with infection with a resistant organism met at least 1 HCAP criterion, whereas approximately half of those with infection related to a susceptible pathogen fulfilled the HCAP definition (P < .001). Therefore, the specificity of the broad HCAP definition for the presence of a resistant pathogen (48.6%) was limited, and the definition misclassified approximately one-third of the patients.
Although each individual item in the HCAP definition was more prevalent in patients with resistant infection, the frequency of these criteria across the cohort with resistant infection varied. For example, more than 80% of patients with resistant infection had been recently hospitalized or admitted from a nursing home. Conversely, only 10% of those with resistant infection needed long-term hemodialysis. We further noted that patients with resistant pneumonia were more likely to need care in the ICU or to undergo mechanical ventilation. More specifically, those with pneumonia due to a resistant pathogen were nearly twice as likely to require admission to the ICU.
Logistic regression (Table 2) identified 4 variables that were independently associated with the identification of a resistant pathogen. Patients recently hospitalized were more than 4 times as likely to have a resistant infection (adjusted odds ratio [OR], 4.21; 95% confidence interval [CI], 2.89-6.15; P < .001), whereas admission from a nursing home or the need for long-term hemodialysis approximately doubled the risk for recovery of a resistant pathogen. Need for ICU admission was the final factor related to resistant bacterial pneumonia (adjusted OR, 1.62; 95% CI, 1.14-2.28; P = .007). Neither the need for mechanical ventilation nor the presence of underlying immunosuppression was associated with identification of a resistant pathogen. In addition, no demographic factor was associated with identification of resistant bacteria. We noted no interactions within the logistic regression or with the Breslow-Day findings. Cross-validation did not indicate overfitting. The point estimates for the variables that remained significant predictors in the model in each of the sequential 10 analyses with 90% of the cohort did not vary substantially from the findings in the entire population.
Based on the logistic regression, we created a scoring system and assigned points as follows: 4 points for a recent hospitalization, 3 points for admission from a nursing home, 2 points for long-term hemodialysis, and 1 point for the need for ICU care, resulting in a maximum of 10 possible points. Overall, this model had moderate predictive value as demonstrated by the ROC curve. The area under the ROC curve was 0.74 (95% CI, 0.65-0.80). Based on visual inspection, we grouped patients into low-, intermediate-, and high-risk strata as a function of overall point score (Figure). Among patients with fewer than 3 points, the prevalence of resistant pathogens was less than 20%, compared with 55% in those with a score ranging from 3 to 5 points and more than 75% in those with a score of 6 points or more (P < .001). Among those with resistant infections, few had scores of less than 5 points (n = 33), indicating a low rate of false-negative findings. A score of 6 points or more was also associated with the highest area under the ROC curve when dichotomizing patients into a group at high risk for resistant infection vs non–high-risk groups. The area under the ROC curve predicting resistant infection was similar to the ROC curve with the complete point scoring system (0.72; 95% CI, 0.62-0.84).
In the subgroup analysis restricted to the presence or absence of pneumonia due to MRSA, we noted patterns seen in the broader comparisons. For example, demographic factors failed to segregate patients with MRSA from those with infection due to another organism (Table 3). Methicillin-resistant S aureus was more common in patients meeting any criteria for HCAP (OR, 3.23; 95% CI, 2.03-5.15). Again, however, the prevalence of MRSA fluctuated across components constituting HCAP. Methicillin-resistant S aureus was more often seen in those from nursing homes and patients recently hospitalized than among patients with immunosuppression or those needing long-term hemodialysis. Methicillin-resistant S aureus was also more prevalent in critically ill patients. In logistic regression (Table 4), 3 factors correlated with the presence of MRSA as the cause of pneumonia: recent hospitalization, admission from a nursing home, and the need for ICU care. In contrast to our observations in the broader cohort, long-term hemodialysis did not correlate with MRSA. In patients with any 2 characteristics independently and positively associated with MRSA pneumonia, the prevalence of MRSA approached 35%.
This retrospective analysis of culture-positive pneumonia in patients presenting to the ED confirms the high prevalence of resistant bacteria in this population. Although infections with pathogens such as MRSA and P aeruginosa are more often seen in persons with HCAP, HCAP per se as a concept has limited value in segregating subjects with potentially resistant infections from those with generally susceptible bacterial processes. Moreover, the unique factors that suggest a patient has HCAP rather than community-acquired pneumonia do not perform equally at stratifying patients as to their risk for infection with resistant organisms.
Several earlier projects have attempted to describe the epidemiology and evolution of HAIs and HCAP. In a large retrospective review of an administrative database, Kollef and colleagues1 noted that HCAP appeared to be a distinct entity separate from community-acquired pneumonia and nosocomial pneumonia. They defined HCAP somewhat differently and restricted HCAP to persons with pneumonia who were inpatients within the past 30 days, who had transferred from another health care facility, or who had been receiving long-term hemodialysis. They did not explore the components of the definition of HCAP and assumed they each had equal importance.1 Carratalà et al4 explored the epidemiology of HCAP in a secondary analysis of a prospective cohort of patients admitted to a single center. They used a characterization of HCAP similar to ours but that excluded immunosuppressed subjects. Although the specific distribution of pathogens varied from those we describe, they too noted that resistant organisms were more prevalent in HCAP. However, they failed to examine the epidemiology of HCAP as a function of the various components of this potential categorization. Neither Carratalà et al4 nor Kollef et al1 examined specific variables associated with MRSA pneumonia presenting to the ED.
Research to date in HAIs of the bloodstream has paralleled work in HCAP. For example, Friedman and coworkers2 suggest that nearly one-third of all bloodstream infections in the hospital represent HAIs of the bloodstream, whereas Vallés et al8 have documented that these HAIs have microbiologic features distinct from community-onset bloodstream infections. Neither of these analyses attempted to evaluate specific predictors of bloodstream infections with resistant infections or to explore the sensitivity or specificity of the HAI concept with respect to the identification of pathogens that necessitate initially broader antimicrobial coverage.
Our findings, therefore, are novel and expand on these prior studies. We confirm that resistant organisms are particularly important in HCAP but also document that there are reasons to be skeptical about the precise definition of HCAP as currently proposed. Some components of the HCAP concept such as immunosuppression do not seem to convey an increased risk for infection with MRSA or other resistant pathogens. Rather, the main criteria indicating resistant infection include recent hospitalization, admission from a nursing home, and severity of illness. We further show that risk factors for pneumonia due to MRSA are subtly different from those for any potentially resistant pathogen. Why does immunosuppression not necessarily increase the risk for pneumonia caused by organisms such as MRSA and P aeruginosa? It may be that the recently hospitalized patients and those arriving from nursing homes have been exposed to more extensive and recent prior antibiotic coverage that has exerted pressures selecting for resistance. This heightened chance for restraint infection in these populations may also arise because of issues with infection control in the hospital or at the nursing home. Conversely, we may have had insufficient statistical power to explore questions of resistance in immunocompromised patients. Immunosuppression may also be too heterogeneous a concept if applied broadly. Certain forms of immunosuppression potentially carry differential risks for resistant infection (eg, human immunodeficiency virus disease vs the use of cyclosporine).
Clinically, our risk stratification scoring tools provide physicians with an easy-to-use mechanism for refining the approach to determine which patients presenting with pneumonia may require broad-spectrum antibiotic coverage. The purpose of broader-spectrum coverage is to ensure that the individual receives initially appropriate antibiotic therapy, an important predictor of mortality and morbidity in pneumonia.9- 11 With a more sophisticated decision aid, those confronting patients with pneumonia can simultaneously strive to enhance their rate of administering initially appropriate antibiotic therapy while working to avoid unnecessary antibiotic coverage. Striking this balance will be crucial to efforts to curb the spread of antimicrobial resistance. Given that therapies for MRSA are often distinct from those for organisms such as P aeruginosa, our subgroup analysis in MRSA can further facilitate clinical decision making in that we have identified a cohort at low risk for MRSA pneumonia. In these persons, it may be possible to withhold anti-MRSA therapy safely, although they may technically meet the current definition for HCAP. Prospective validation of our scoring tool is required before it can be applied broadly. However, as was seen in risk scoring systems for pulmonary embolism management, physicians appreciate guidance that enhances decision making in complicated patients.12,13
The present analysis has several important limitations. First, it is retrospective and based in part on coding from the International Classification of Diseases, Ninth Revision, and therefore prone to several forms of bias. We attempted to minimize the impact of this by ensuring that all patients had microbiologic evidence of infection and by having 1 investigator determine that each person met criteria for pneumonia and had evidence of an infiltrate. Second, the data come from a single center. Thus, our findings may not be generalizable to other institutions. The importance of this concern is illustrated by the discordance in the prevalence of MRSA between our institution and the one described by Carratalà et al.4 The nursing homes surrounding the institution might have specific resistance patterns not seen elsewhere, or the extent of immunosuppression in the surrounding population might be limited. There are no specific reasons, however, to believe that the nursing homes admitting to this institution are unique. Nonetheless we urge clinicians to investigate the burden of MRSA in their own communities and EDs. Third, in restricting the analysis to patients with culture-positive pneumonia, we may have skewed our findings in that many cases of pneumonia may have negative culture results. Nevertheless, because our objective was specifically to examine resistant pathogens, we by necessity had to rely on positive findings from cultures to guide our work. Fourth, we did not include patients directly admitted to the ICU. Direct admissions at the study facility are limited, and, in general, all persons needing care on the wards undergo initial evaluation in the ED. Our focus was primarily the ED, where the issue of sorting HCAP from community-acquired pneumonia is most acute. Given the infrequency of direct ICU transfers relative to the proportion of patients with pneumonia in the ED, we believe this issue is of limited concern. Fifth, we essentially used a modified definition of HCAP. We lacked information regarding prior antibiotic exposure that is a key risk factor for infection with resistant pathogens. With better information about this, our findings might have been different. Finally, our sample size was somewhat limited. With a greater number of subjects, we could have used differing approaches to modeling the risk for resistant infection and better validated a risk-scoring tool. Specifically, with a larger sample size we could have used a split-sample validation approach or relied on classification analysis and regression tree methods for risk stratification. Future analyses should adopt these methods. With more patients, we also might have had the ability to look at risk factors within specific gram-negative pathogens such as P aeruginosa.
In conclusion, resistant pathogens such as MRSA and P aeruginosa are frequent causes of pneumonia in persons presenting to the ED. Although resistance is more common in persons meeting criteria for HCAP, not all component criteria for HCAP convey a similar risk for resistance. Simple risk stratification tools may facilitate more accurate identification of persons with pneumonia caused by resistant pathogens.
Correspondence: Andrew F. Shorr, MD, MPH, Pulmonary and Critical Care Section, Room 2A-38D, Washington Hospital Center, 110 Irving St NW, Washington, DC 20010 (firstname.lastname@example.org).
Accepted for Publication: May 5, 2008.
Author Contributions:Study concept and design: Zilberberg. Acquisition of data: Shorr, Micek, and Kollef. Analysis and interpretation of data: Shorr, Zilberberg, and Kollef. Drafting of the manuscript: Shorr and Kollef. Critical revision of the manuscript for important intellectual content: Shorr, Zilberberg, Micek, and Kollef. Statistical analysis: Shorr and Zilberberg. Obtained funding: Shorr. Administrative, technical, and material support: Shorr, Zilberberg, and Micek.
Financial Disclosure: Drs Shorr, Micek, and Kollef have served as speakers for and consultants to Ortho-McNeil-Janssen Pharmaceuticals, Inc. Dr Zilberberg has served as a consultant to Ortho-McNeil-Janssen Pharmaceuticals, Inc.
Funding/Support: This study was supported by a grant from Ortho-McNeil-Janssen Scientific Affairs, LLC.
Role of the Sponsor: The sponsor had no role in determining study design; in the conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.