Classification tree for the derivation (A) and validation (B) cohorts. AMS indicates altered mental status, defined as a Glasgow Coma Scale score lower than 14 or a designation of disoriented, stupor, or coma by a physician; BUN, blood urea nitrogen (to convert to millimoles per liter, multiply by 0.357).
Mortality rate by BAP-65 class. Results are presented as median and 2.5th percentile and 97.5th percentile based on 1000 bootstrap reiterations. BAP-65 indicates blood urea nitrogen level higher than 25 mg/dL (to convert to millimoles per liter, multiply by 0.357); altered mental status, defined as a Glasgow Coma Scale score lower than 14 or a designation of disoriented, stupor, or coma by a physician; pulse higher than 109/min; and age older than 65 years.
Mechanical ventilation (MV) within 48 hours of admission by BAP-65 class. Results are presented as median and 2.5th percentile and 97.5th percentile based on 1000 bootstrap reiterations. BAP-65 indicates blood urea nitrogen level higher than 25 mg/dL (to convert to millimoles per liter, multiply by 0.357); altered mental status, defined as a Glasgow Coma Scale score lower than 14 or a designation of disoriented, stupor, or coma by a physician; pulse higher than 109/min; and age older than 65 years.
Mortality or mechanical ventilation (MV) at any time during hospitalization by BAP-65 class. Results are presented as median and 2.5th percentile and 97.5th percentile based on 1000 bootstrap reiterations. BAP-65 indicates blood urea nitrogen level higher than 25 mg/dL (to convert to millimoles per liter, multiply by 0.357); altered mental status, defined as a Glasgow Coma Scale score lower than 14 or a designation of disoriented, stupor, or coma by a physician; pulse higher than 109/min; and age older than 65 years.
Tabak YP, Sun X, Johannes RS, Gupta V, Shorr AF. Mortality and Need for Mechanical Ventilation in Acute Exacerbations of Chronic Obstructive Pulmonary DiseaseDevelopment and Validation of a Simple Risk Score. Arch Intern Med. 2009;169(17):1595-1602. doi:10.1001/archinternmed.2009.270
Acute exacerbations of chronic obstructive pulmonary disease (AECOPDs) often require hospitalization, may necessitate mechanical ventilation, and can be fatal. We sought to develop a simple risk score to determine its severity.
We analyzed 88 074 subjects admitted with an AECOPD between 2004 and 2006. We used recursive partition to create risk classifications for in-hospital mortality. Need for mechanical ventilation served as a secondary end point. We internally validated the model via 1000 bootstrapping on half of patients and externally validated it on the remaining patients. We assessed predictive ability using the area under the receiver operating curve (AUROC).
The in-hospital mortality rate was 2%. Three variables had high discrimination of outcomes: serum urea nitrogen level greater than 25 mg/dL (to convert to millimoles per liter, multiply by 0.357); acute mental status change, and pulse greater than 109/min. For those without any of the 3 factors, age 65 years or younger further differentiated the lowest-risk group. In those with all 3 factors, the mortality rates were 13.1% (131 in 1000) and 14.6% (146 in 1000) in the derivation and validation cohorts, respectively, compared with 0.3% (3 in 1000) in both cohorts among patients without any of the 3 factors and age 65 years or younger (P < .001). The AUROC for mortality in the 2 cohorts were 0.72 (95% confidence interval [CI], 0.70-0.74) and 0.71 (95% CI, 0.70-0.73), respectively. For mechanical ventilation, the AUROCs were 0.77 (95% CI, 0.75-0.79) for both cohorts.
A simple risk class based on clinical variables easily obtained at presentation predicts mortality and need for mechanical ventilation. It may facilitate the triage and care of patients with AECOPD.
Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) represent a leading cause of morbidity and mortality.1 On average, patients with COPD experience 1 to 2 exacerbations annually, with the frequency of exacerbations increasing as lung function declines. Although more than 90% of persons with an exacerbation are treated as outpatients, a growing number of subjects require hospitalization.
For example, between 1990 and 2000, the number of annual admissions for COPD exacerbation management in the United States grew from 463 000 to 726 000.2 Hospital care for COPD remains expensive and accounts for more than 60% of the estimated $18 billion in direct costs for COPD care annually in the United States.3 Because of the aging of the population, the burden of COPD exacerbations is expected to rise.
Beyond the economic aspects of COPD care, exacerbations also substantially impair quality of life.4 Over the last decade, major improvements have been made in the care of patients admitted for an AECOPD. Both randomized trials and multiple systematic reviews have demonstrated the efficacy of antibiotics, corticosteroids, and noninvasive ventilation in this syndrome.5 Furthermore, newer guidelines have proposed staging systems and tools based on spirometric data to gauge the severity of stable patients with COPD.6 Unfortunately, these classification schemes differ significantly in their ability to categorize COPD severity accurately,7 specifically as it relates to patient outcomes. In addition, because no diagnostic gold standard exists for AECOPD, it is difficult to apply these classification schemes in the acute care setting. When assessing a subject with an AECOPD during routine practice, physicians often lack COPD-specific information such as spirometric results. Clinicians also may not be able to obtain spirometry in patients with an AECOPD either because it is simply not available or because the patients are too ill to perform the test. As a result, evaluating clinicians must use their clinical acumen to determine diagnosis and optimal treatment. Hence, clinicians faced with patients with COPD experiencing an apparent acute exacerbation lack a validated and practical means for triaging those patients and determining whether they might safely be sent home or, if admitted, require intensive care. In other disease states, the evolution of risk stratification schema has improved medical decision making and enhanced efficiency and resource use. For example, the Pneumonia Severity Index (PSI) is now endorsed in multiple national practice guidelines for community-acquired pneumonia.8 Validated severity of illness scoring instruments also foster the conduct of clinical trials and other outcome studies. They provide a uniform means for describing study patients and can help ensure that disease severity is well-balanced in trial populations.
We sought to develop and validate a severity of illness tool for AECOPD that can be applied easily and broadly at the time of admission. We hypothesized that patients with AECOPD could be stratified by their risk of death and need for mechanical ventilation (MV) based on easily obtainable clinical data available at the time of presentation to the emergency department (ED).
We examined the Cardinal Health Clinical Outcomes Research Database (Clinical Research Services, Cardinal Health, Marlborough, Massachusetts) between January 1, 2004, and December 31, 2006. This data set has been described in detail elsewhere.8- 13 Briefly, for acute care admissions to participating hospitals, the data set is comprised of demographic, diagnoses, hospital mortality, results from laboratory testing, vital signs, and other key clinical findings.
We included all persons older than 40 years (to exclude patients with potential asthma) with a principal International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge code of AECOPD (491.21, 491.22, or 496) and diagnosis related group (DRG) 88 (COPD), DRG 541, 542, 565, 566, or 475 (5 DRGs indicating that MV was performed during hospitalization). The DRG restrictions excluded approximately 1% of all patients with AECOPD hospitalized for biopsies or other uncommon procedures. The International Classification of Diseases, Ninth Revision (ICD-9) procedure codes used to identify MV were 967.0, 967.1, and 967.2. We did not include patients with a principal diagnosis of “acute respiratory failure” (ARF) (ICD-9 code 518.81) and a secondary diagnosis of AECOPD for 2 reasons. First, we intend to develop a tool that aids physicians in determining whether patients with AECOPD need inpatient care. If patients present to the ED with ARF, there is little ambiguity for admission. Second, one of our aims was to develop a tool that predicts the need for subsequent MV during the index hospitalization. For patients presenting with ARF, an immediate MV is likely required. We randomly divided the population into 2 cohorts, one for model derivation and another for model validation.
Hospital mortality served as the primary end point. Need for MV during the first 48 hours of hospitalization served as a secondary end point. The combination of mortality or need for MV any time during hospitalization represented another end point for sensitivity analysis.
Potential candidate variables were selected a priori based on their biological plausibility at explaining risk for mortality. Specifically, we explored demographic factors (age and sex), vital signs (pulse, systolic blood pressure, diastolic blood pressure, temperature, and respiratory rate), mental status, results of laboratory testings, and underlying comorbid conditions. Altered mental status was defined as a Glasgow Coma Scale score less than 14 or a physician-charted designation of “disoriented,” “stupor,” or “coma.” Overall, 85% of the study population was admitted from the ED. Of these patients, more than 99% had vital signs and mental status recorded in the ED. We noted the worst recorded value. Otherwise, the worst value on admission day was used. For laboratory testing, results from routine chemical, hematologic, blood gas, and metabolic analyses were considered as potential predictive variables. Laboratory results were generally collected on admission day and included those obtained in the ED. For patients with multiple laboratory assessments on admission day, we used the worst values noted. For patients admitted after 6:00 PM without laboratory data on admission day, the worst laboratory value collected 1 day before or after admission was used. Comorbid conditions of interest included underlying cardiopulmonary disease, chronic renal failure, history of malignant neoplasm, and other chronic conditions abstracted through medical record review or secondary ICD-9 diagnostic codes.
We used a recursive partition (RPART) approach to identify mortality risk factors with the highest discriminative power.14- 16 The RPART, also known as classification and regression tree analysis,17 has been used to derive prediction rules for acute chest pain,18 congestive heart failure,19 and other disease states.20- 22 The RPART first identifies 1 variable with the highest discrimination to partition the patient population into high and low risk for the outcome of interest (nodes). It then continues the process to partition the subsequent nodes. The goal is to identify the variables and partition point that optimally separate low- from high-risk patients.
The RPART results in a treelike algorithm consisting of multiple nodes. To make the classification tree more user friendly, we simplified the algorithm into a manageable number of risk classes based mainly on the number of risk factors present.
We used 1000 bootstrap iterations to internally validate the risk class and estimate the 95% confidence intervals (CIs). We report the median of these 1000 reiterations as the parameter estimate and the 2.5 and 97.5 percentiles as the 95% CIs.23 We relied on the area under receiver operating curves (AUROCs) to assess model discrimination and the Cochrane-Armitage trending statistic to assess whether the risk score could differentiate low-risk from high-risk patients in a fashion of graded response based on the level of risk present.
We used the risk algorithm derived from the derivation cohort to score the validation cohort. We estimated the statistical parameters with another 1000 bootstrap iterations for the validation cohort.
To test if the risk classification system could predict the need for MV, we applied the mortality risk classification to the population and assessed the concomitant need for MV within 48 hours of admission for both derivation and validation cohorts. We used the same statistical approach to estimate the AUROC and trend as we did for mortality.
We further assess the utility of the risk score in predicting either mortality or the need of MV any time during hospitalization, using same statistical approaches. The RPART software used to derive the initial rules was downloaded from a publicly accessible Web site.16 All other analyses were conducted using Statistical Analysis Software (version 9.01; SAS Institute Inc, Cary, North Carolina).
The study population contained data from 191 hospitals (41% teaching and 59% nonteaching). A total of 145 (76%) were urban hospitals and 46 (24%) were rural. There were 33 hospitals (17%) with a bed size greater than 300, 110 hospitals (58%) with a bed size between 100 and 300, and 48 hospitals (25%) with a bed size less than 100. The study population consisted of 88 074 admissions for AECOPD. The median age was 72 years (interquartile range, 63-79), and 55% of patients were female. More than half of admissions were to teaching hospitals. As Table 1 shows, tachycardia (heart rate >109/min) was common in the population, and approximately 20% of subjects had blood urea nitrogen (BUN) levels higher than 25 mg/dL (to convert to millimoles per liter, multiply by 0.357). Mechanical ventilation was required in less than 1.1% of cases. The crude in-hospital mortality rate was 1.8%.
The RPART approach initially selected a BUN level higher than 25 mg/dL as the first partition node for separating patients based on probability of hospital mortality (Figure 1A). Subsequently, it segregated patients into groups based on alterations in mental status followed by a separation as a function of the pulse higher than 109/min. Finally, age 65 years or younger further differentiated the lowest-risk group. This resulted in 9 distinct risk cohorts given the potential combinations of the 3 risk factors and age. In the derivation cohort, for those without any of the 3 main risk factors and age 65 years or younger, the mortality rate was 0.3% (3 in 1000) compared with 13.8% (138 in 1000) in persons with all 3 main factors present on admission (P < .001). We noted similar findings in the validation cohort (Figure 1B).
Because of similarities in the chance for death based on the presence of equivalent numbers of risk factors, and to enhance the ease of use, the model was collapsed into a simpler class schema. Hence, the model was compressed from 9 distinct categories into 5 level risk classes (BAP-65 class) (Table 2). In the derivation cohort, for patients who had none of the 3 main variables of interest (BUN level >25 mg/dL, altered mental status, or pulse >109/min), 17.6% were 65 years or younger (class I) and 34.4% were older than 65 years (class II). Approximately 39.6% possessed only 1 main risk factor (class III), and 7.7% possessed 2 main risk factors (class IV). Few subjects (0.7%) presented with concomitant elevations in BUN level, alterations in mental status, and tachycardia (class V). The corresponding mortality rates differed significantly as a function of the BAP-65 class (P < .001) (Figure 2). The model had a good fit, as demonstrated by the AUROC in the derivation and validation cohorts (0.72 [95% CI, 0.70-0.74] and 0.71 [95% CI, 0.70-0.73], respectively). The Cochrane-Armitage test for trend and the 95% CIs were all highly significant (P < .001) for both cohorts, confirming the graded mortality risk with an increase in the level of risk class.
Although only 1.1% of the study population required MV within the first 48 hours of admission, the simplified BAP-65 class discriminated patients based on the need for MV (Figure 3). As was seen with the probability of death, persons with 2 or 3 main risk characteristics were more likely to undergo MV than patients with fewer risk factors. In the group at high risk for death (all 3 main risk factors), 10.1% of persons underwent MV during the first 48 hours of admission. Despite accounting for only 8.4% of the entire derivation sample, more than 44% of all those needing MV had at least 2 main risk factors present. The results from the validation cohort were similar. The AUROCs were 0.77 (95% CI, 0.75-0.79) for both cohorts. The Cochrane-Armitage test for trend and the 95% CIs were all highly significant (P < .001). When the BAP-65 class was applied to the combined mortality or MV any time during hospitalization, we observed similar discrimination and trend (Figure 4).
This analysis of a large clinical database demonstrates that patients with AECOPD present a wide range of disease severity. More importantly, a simple clinical risk class easily stratifies patients by their risk of hospital mortality and need for MV early during hospitalization. Furthermore, the nexus we observed between MV and mortality reinforces the biological plausibility of our proposed severity of illness class.
For complex disease states, risk stratification tools facilitate triage assessments and enhance medical decision making. Determining hospital admission and placement have depended, in many conditions, solely on clinical judgment and acumen. This has resulted in a wide range of practice styles24 and may compromise patient care. Likewise, limited medical resources are more likely to be used efficiently if clinicians have a means for aligning need with outcomes. In that sense, risk severity tools for pneumonia,8 a condition in many ways similar to AECOPD, have proved crucial. Since its creation, the PSI has been endorsed in national practice guidelines and has become part of many hospital protocols for community-acquired pneumonia management.25 An alternative to the PSI, a simpler CURB-65 (confusion, BUN level >19.6 mg/dL, respiratory rate 30/min, low blood pressure, and age 65 years) that contains 5 variables, also has been developed to help with severity assessments in community-acquired pneumonia.26 Our efforts build on these earlier projects and extend the range of severity of illness tools to AECOPD. Our mortality prediction rule relies on easily obtainable information routinely available in the ED. This fact enhances the possible value of our tool in that it can be easily applied and does not require invasive testing, complex calculations, or expensive data collection.
Prior severity of illness tools for COPD have only focused on intermediate and longer-term mortality and require extensive assessment of many risk factors. The BODE score (body mass index, the degree of airflow obstruction and dyspnea, and exercise capacity) predicts mortality over months and requires assessment of dyspnea, exercise capacity, and other factors.27 It has never been validated for use in AECOPD. We aimed for pragmatism, as many components of the BODE score are likely not available to physicians when evaluating a patient with an AECOPD in the ED. Physicians often use biomarkers in other disease states to facilitate assessments of acute disease severity. In cardiac disease, measurement of troponin and brain natriuretic peptide levels has proven to be very useful.28,29 Unfortunately, studies of biomarkers in acute COPD exacerbations have been inconsistent. Serum brain natriuretic peptide level, for instance, may be elevated in acute COPD flares. However, it does not appear to correlate well with either in-hospital mortality or the need for MV, and its utility in risk stratification in AECOPD remains to be proven.30
Why do BUN, mental status, and pulse correlate with mortality in persons hospitalized for AECOPD? Likely, these 3 variables capture significant end-organ dysfunction. Blood urea nitrogen has consistently appeared to be an important marker of poor outcome in respiratory syndromes.8 In AECOPD, it may reflect intravascular volume depletion ensuing from poor oral intake and hyperventilation in the days prior to admission. Similarly, tachycardia may capture interactions between volume status, hypoxemia, and general distress. A decline in mental status can arise in AECOPD from hypercapnia and thus provides insight into underlying pathophysiologic mechanisms more accurately than direct measurement of the partial pressure of carbon dioxide. Because some with chronic respiratory failure tolerate elevated PCO2 levels well, the more fundamental question ought to focus on the body's response to this derangement. Alternatively, that blood gas data did not figure in our prediction rule may appear surprising. In isolation, however, the blood gas fails to integrate issues of physiologic reserve and the ability to tolerate disease progression. Likewise, it is the pathophysiological manifestations of the underlying diseases, not chronic conditions, that offered the most discriminative power in facilitating risk stratification for in-hospital mortality or need for MV. Given the short time horizon for our analysis, the underlying chronic conditions may not exert the impact they might on either intermediate or longer-term mortality. Furthermore, our goal was to create a simple risk stratification tool and not to create a complex model to predict mortality. To accomplish the latter would have required adding multiple variables which, in turn, would have increased complexity and could hinder the model's application. A similar methodological approach in hospitalizations for congestive heart failure noted that 3 variables (BUN, systolic blood pressure, and serum creatinine) had an AUROC of 0.69,19 similar to the AUROC of 0.72 for our model. Although we did not have FEV1 (forced expiratory volume in the first second of expiration) data to test whether it would add predictive ability, including FEV1 in the risk adjustment model is not very practical because of its low availability among patients with AECOPD at the time of ED presentation.
Our analysis has limitations. First, we relied on administrative data for the AECOPD diagnosis. This can be an imprecise means for identifying persons with AECOPD. However, administrative data sources have been used in other projects addressing risk stratification in diseases such as pneumonia, congestive heart failure, and pulmonary embolism.8,19,20 Recently, our approach to identifying patients with AECOPD was used in a large-scale study on quality of care for COPD.24 Like this study, we enhanced our effort to distinguish COPD from patients with asthma by restricting the population to persons older than 40 years. It should be noted that because our data set consists of all COPD admissions in 191 hospitals, our findings are less likely to be confounded as a result of being based on a predefined subgroup of patients receiving predefined treatments. Using this database afforded us the opportunity to study a large population from both teaching and nonteaching hospitals and to validate our findings both internally and externally.
Second, because we intended to develop a tool to aid ED physicians in determining the need for admission and MV after admission, we did not include patients with a principal diagnosis of ARF for whom there is little ambiguity for admission and high likelihood of immediate MV need. As such, we likely excluded subjects who had ARF and concomitantly presented with AECOPD. Hence, this may limit the generalizability of our findings. This also likely explains why the observed mortality rate was lower than what has been reported in other analyses of AECOPD, which included patients with ARF.24 We caution clinicians not to apply our findings to persons who present with ARF requiring emergent intubation and MV. However, a much larger AECOPD population tend to present at the ED without definitive “ARF” but a wide range of severity. Our tool identified 17% of AECOPD in-patients in the lowest risk class with less than 0.3% mortality. Physicians may decide whether many of these low-risk patients can be treated as outpatients. Hence, a stratification tool aiding physicians to differentiate patients at low risk from those at high risk would have value in the clinical practice and management of resource utilization for this large patient population.
Third, we lacked information on certain COPD-specific covariates, particularly FEV1. As noted earlier, spirometric data often are not available to physicians in the ED, and our goal was to create a score that could be used easily and broadly using routinely available data. More importantly, it remains unclear if and how FEV1 relates to outcomes in AECOPD. For example, a recent review of more than 2000 patients with COPD found only a weak correlation between FEV1 and both mortality and need for hospitalization.31 Consequently, some have argued that one major inadequacy of the current staging system for stable COPD is that it overemphasizes FEV1 and that we must develop tools based more on patient-centered factors.7 This was a major goal of our study. In the same vein, we could not adjust for many process-of-care variables that could affect outcomes. Specifically, we could not take use of do-not-resuscitate orders into considerations. Our previous study had shown that do-not-resuscitate was not uniformly defined or used across hospitals, which made it difficult to use for risk adjustment purpose.32 Finally, we could not evaluate the use of noninvasive ventilation. The application of noninvasive ventilation for AECOPD is gaining acceptance, and our risk stratification scheme needs to be validated for its utility in predicting the need for noninvasive ventilation in addition to MV when data become available.
In conclusion, a simple risk class stratifies patients with AECOPD by their risk for death and need for MV. Although no clinical decision rule is infallible and clinicians must always apply their best judgment, application of this risk score, if further validated in prospective studies, may enhance the care of patients presenting to the hospital with AECOPD.
Correspondence: Andrew F. Shorr, MD, MPH, Pulmonary and Critical Care Medicine, Washington Hospital Center, 110 Irving St NW, Washington, DC 20010 (firstname.lastname@example.org).
Accepted for Publication: June 4, 2009.
Author Contributions: Dr Tabak had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Tabak, Gupta, and Shorr. Acquisition of data: Tabak. Analysis and interpretation of data: Tabak, Sun, Johannes, Gupta, and Shorr. Drafting of the manuscript: Tabak, Sun, Johannes, and Shorr. Critical revision of the manuscript for important intellectual content: Tabak, Johannes, Gupta, and Shorr. Statistical analysis: Tabak, Sun, Johannes, and Shorr. Administrative, technical, and material support: Tabak and Gupta. Study supervision: Tabak and Shorr.
Financial Disclosure: Drs Tabak, Sun, Johannes, and Gupta are employees of Cardinal Health, Marlborough, Massachusetts.
Additional Contributions: Karen Derby, BS, of Cardinal Health Clinical Research Services contributed in database management, analysis, and technical support. We acknowledge helpful comments from 4 anonymous reviewers.