A Prediction Rule to Identify Low-Risk Patients With Pulmonary Embolism

Background  A simple prognostic model could help identify patients with pulmonary embolism who are at low risk of death and are candidates for outpatient treatment.

Methods  We randomly allocated 15 531 retrospectively identified inpatients who had a discharge diagnosis of pulmonary embolism from 186 Pennsylvania hospitals to derivation (67%) and internal validation (33%) samples. We derived our rule to predict 30-day mortality using classification tree analysis and patient data routinely available at initial examination as potential predictor variables. We used data from a European prospective study to externally validate the rule among 221 inpatients with pulmonary embolism. We determined mortality and nonfatal adverse medical outcomes across derivation and validation samples.

Results  Our final model consisted of 10 patient factors (age ≥70 years; history of cancer, heart failure, chronic lung disease, chronic renal disease, and cerebrovascular disease; and clinical variables of pulse rate ≥110 beats/min, systolic blood pressure <100 mm Hg, altered mental status, and arterial oxygen saturation <90%). Patients with none of these factors were defined as low risk. The 30-day mortality rates for low-risk patients were 0.6%, 1.5%, and 0% in the derivation, internal validation, and external validation samples, respectively. The rates of nonfatal adverse medical outcomes were less than 1% among low-risk patients across all study samples.

Conclusions  This simple prediction rule accurately identifies patients with pulmonary embolism who are at low risk of short-term mortality and other adverse medical outcomes. Prospective validation of this rule is important before its implementation as a decision aid for outpatient treatment.

Pulmonary embolism (PE) is a major health problem, with an estimated incidence of 23 to 69 cases per 100 000 persons annually in the United States.^1,2 Data from the National Hospital Discharge Survey³ show that 101 000 patients were hospitalized in 2002 in acute care hospitals having a primary diagnosis of PE in the United States, resulting in 676 700 inpatient days. The all-cause short-term mortality of this illness varies widely, ranging from more than 95% among patients who experience cardiorespiratory arrest to less than 2% among patients with nonmassive PE,^4-6 defined as PE without systemic hypotension, cardiogenic shock, or respiratory failure.⁷

There is growing evidence that outpatient treatment with low-molecular-weight heparin sodium is effective and safe for selected patients with nonmassive PE.^8-10 Based on this evidence, experts^11,12 and the British Thoracic Society guidelines for the management of acute PE¹³ recommend outpatient treatment for clinically stable patients. Outpatient treatment for nonmassive PE is not widely accepted because no explicit clinical criteria exist to accurately identify patients with PE who are at low risk of adverse outcomes. Therefore, we sought to develop an objective and easily applied clinical prediction rule to identify patients with PE at low risk of short-term mortality and other adverse medical outcomes who are candidates for outpatient treatment.

We identified patients with PE using the Pennsylvania Health Care Cost Containment Council database,¹⁴ which contains information on demographics, source of admission, admission and discharge dates, inpatient mortality data, and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge diagnosis and procedure codes for patients admitted to all nongovernmental acute care hospitals in Pennsylvania. Our study included inpatients 18 years or older who had a primary discharge diagnosis of acute PE between January 1, 2000, and November 30, 2002, based on the following ICD-9-CM codes: 415.1, 415.11, 415.19, and 673.20 through 673.24. To ensure that we identified the most severely ill patients with PE as the primary reason for hospitalization, we also included inpatients with a secondary ICD-9-CM code for PE and 1 of the following primary codes that may represent complications or treatments of PE: respiratory failure (ICD-9-CM code 518.81), cardiogenic shock (ICD-9-CM code 785.51), cardiac arrest (ICD-9-CM code 427.5), secondary pulmonary hypertension (ICD-9-CM code 416.8), syncope (ICD-9-CM code 780.2), thrombolysis (ICD-9-CM code 99.10), and intubation or mechanical ventilation (ICD-9-CM codes 96.04, 96.05, and 96.70-96.72). Because patients with recurrent PE may have a higher mortality than patients with a single episode,^15,16 we included all episodes of PE for the same patient within the study period to avoid potential selection bias. We did not include patients who had only a secondary ICD-9-CM code for PE or who were transferred from another health care facility, because such patients are more likely to have PE as a complication of hospitalization (eg, after surgery). Because outpatient treatment for PE was not considered usual care between 2000 and 2002, it is likely that we captured most patients having a primary diagnosis of PE in Pennsylvania during this period. This study was approved by the institutional review board of the University of Pittsburgh, Pittsburgh, Pa.

The baseline clinical variables necessary to derive our prediction rule were obtained from the Atlas database (MediQual, Marlborough, Mass).¹⁴ Clinical inpatient data from all nongovernmental acute care hospitals in Pennsylvania are represented in this proprietary database, which is compiled from patient medical records using standardized data collection instruments.

We used vital signs measured in the emergency department for all patients admitted through the emergency department; all other variables were recorded on the day of hospital admission. For patients admitted from other sources (eg, directly from a physician’s office), we abstracted all clinical variables on the day of admission. To derive our prediction rule, we used clinical variables routinely available to clinicians at the time of initial examination and previously shown to be associated with short-term mortality in patients who have PE or other acute diseases (Table 1). We did not consider other potential predictors such as right ventricular dysfunction, mean pulmonary arterial pressure, or concomitant deep vein thrombosis shown by sonography because these conditions are not routinely assessed among patients diagnosed as having PE.^17,20,31

The main study outcome used to derive our prediction rule was death from all causes within 30 days of each hospitalization. All-cause 30-day mortality is objective and clinically relevant and is a widely used outcome of prognostic models for other acute diseases or medical interventions.^32-34 Most deaths due to PE occur within this time frame.³⁵ We obtained mortality data from the National Death Index.³⁶ Using Atlas database information and discharge ICD-9-CM codes from the Pennsylvania Health Care Cost Containment Council database, we also assessed whether patients classified as low risk by our prediction rule developed nonfatal cardiogenic shock (ICD-9-CM code 785.51) or cardiorespiratory arrest, defined as cardiac arrest (ICD-9-CM code 427.5), resuscitation (ICD-9-CM codes 99.60, 99.63, and 37.91), intubation (ICD-9-CM codes 96.04 and 96.05), or mechanical ventilation (ICD-9-CM codes 96.70-90.72).

Of the 16 468 patient discharges that met our inclusion criteria, we excluded 937 because they were missing patient identifiers (n = 867) or could not be linked to the National Death Index (n = 70). Therefore, the study cohort comprised 15 531 patients who had a discharge diagnosis of PE from 186 Pennsylvania hospitals. Overall, these discharges represented 14 672 individual patients with PE; 859 discharges (6%) represented recurrent PE episodes that occurred during the study period. We randomly selected 10 354 discharges (67%) for the derivation sample and 5177 discharges (33%) for the internal validation sample.

We derived our prediction rule using classification tree analysis,³⁷ with 30-day mortality as the outcome and the demographic and clinical variables in Table 1 as predictors. Except for age, we dichotomized continuous variables using clinically meaningful cutoff points that are commonly used in clinical practice and are easily remembered by physicians (eg, systolic blood pressure <100 mm Hg and arterial oxygen saturation <90%). Unknown values were assumed to be normal, a strategy successfully used in the derivation and validation of a widely used previous prognostic model for pneumonia.³² Using S-Plus 2000 software,³⁸ we recursively partitioned our derivation sample into progressively more homogeneous subgroups by sequentially identifying predictor variables that best discriminated between patients who died and those who did not. At each step, the program automatically examined all possible splits for age and each categorical predictor to identify the variable and cutoff point that maximized goodness of fit. The splitting process continued until the subgroups were homogeneous or contained fewer than 3 deaths. Although we did not modify the automatically generated models using subjective criteria, we rounded cutoff points to the next clinically meaningful value. We explored candidate tree models with and without laboratory variables, trying to find models that identified a low-risk group with a membership of at least 20% of the total derivation sample and a 30-day mortality of less than 1%. Although no widely accepted threshold defines low risk, prognostic models for other acute diseases such as community-acquired pneumonia or heart failure defined short-term mortality rates below 1% to 2% as low risk.^14,32 Among candidate models meeting these criteria, we chose the one with the fewest predictors.

We then assessed the performance of our prediction rule in the internal validation sample by computing the proportion of patients who were classified as low vs higher risk and the proportion of patients who died within 30 days of initial examination. Because 7-day mortality may be more relevant for the hospital admission decision than 30-day mortality, we also estimated the proportion of patients in both samples who died 7 days after admission or experienced nonfatal cardiogenic shock or cardiorespiratory arrest in the hospital. We externally validated our rule using data previously collected from a prospective cohort study³⁹ that used spiral computed tomography to diagnose PE. That study enrolled patients with suspected PE from 3 emergency departments at the university hospitals of Lausanne, Geneva, and Angers between October 1, 2000, and June 30, 2002. Patients who had a contraindication to spiral computed tomography (allergy to iodine contrast agents, creatinine clearance <30 mL/min, or pregnancy), severely ill patients (massive PE with shock or expected survival <3 months), or those unable to provide signed informed consent because of cognitive impairment were excluded from that study. Baseline patient characteristics, including the predictors that comprise our rule, were collected in the emergency department. The criteria used to establish the diagnosis of PE are described elsewhere.³⁹ Death, objectively confirmed recurrent venous thromboembolism,major bleeding (defined as retroperitoneal, joint, or cerebral bleeding or any bleeding requiring transfusion), and the timing of these adverse events were documented during a 3-month follow-up. Of 1290 screened patients with suspected PE, 965 (75%) were eligible for the study.³⁹ Eligible patients were younger and had fewer comorbid conditions than excluded patients. For our external validation, we used data from 221 of 222 patients with objectively confirmed PE enrolled in that study,³⁹ excluding 1 patient who was lost to follow-up. We then estimated the proportion of patients classified as low risk by our prediction rule and the proportion of patients who died 7 days, 30 days, and 90 days after the initial examination. We also assessed whether patients developed nonfatal recurrent venous thromboembolism or had major bleeding during follow-up.

We compared the mortality rates, proportions of patients classified as low risk vs higher risk, and nonfatal adverse medical outcomes among the derivation and internal and external validation samples using logistic regression analysis with a robust variance estimator to account for the clustering of patients who were discharged more than once for PE during the study period. For comparisons involving observed zeros, we used exact χ² tests. A 2-sided P<.05 was considered statistically significant. To assess the accuracy of our rule to predict 30-day mortality, we also compared sensitivity, specificity, positive and negative predictive values, and likelihood ratios for low-risk vs higher-risk patients across derivation and validation samples.

Relative to the derivation and internal validation samples, patients in the external validation sample had a lower prevalence of most comorbid illnesses and fewer abnormal findings on physical examination, reflecting the exclusion of severely ill patients from the study³⁹ used to externally validate the rule (Table 2). Thirty-day mortality in the derivation, internal validation, and external validation samples was 9.2%, 9.5%, and 2.7%, respectively.

Among the candidate tree models, we selected a model consisting of the following 10 easily ascertained and clinically relevant patient factors: age 70 years or older, a history of 5 comorbid diseases (cancer, heart failure, chronic lung disease, chronic renal disease, and cerebrovascular disease), and the presence of 4 clinical abnormalities (pulse rate ≥110 beats/min, systolic blood pressure <100 mm Hg, altered mental status, and arterial oxygen saturation <90%) (Figure). Patients with none of these factors were defined as low risk.This model, which does not rely on any laboratory variables, identified 21.6% (95% confidence interval [CI], 20.8%-22.4%) of patients in the derivation sample as low risk, with a corresponding 30-day mortality of 0.6% (95% CI, 0.3%-1.0%). Alternative tree models that included laboratory findings did not measurably increase the proportion of patients classified as low risk or decrease the mortality rate among low-risk patients compared with this model based on history and clinical variables alone.

In the derivation sample, the 859 patients (6%) who developed recurrent PE during the study period had a slightly lower 30-day mortality (8.3%) than patients with a single PE episode (9.2%). The proportion of patients defined as low risk and the mortality of this subgroup did not change when we excluded the patients who had a discharge diagnosis of recurrent PE during the study period, nor when we excluded the 2% of patients who had primary ICD-9-CM codes for PE complications or treatments. This latter subgroup included the most severely ill patients in our study (mean 30-day mortality, 66%).

In the internal validation sample, our derived model classified 21.6% (95% CI, 20.5%-22.7%) of patients as low risk, with low-risk patients having a 30-day mortality of 1.5% (95% CI, 0.9%-2.4%) (Table 3). Although 30-day mortality was somewhat higher in the internal validation sample than in the derivation sample (P = .01), the 0.9% difference was small in absolute terms. Seven-day mortality was 0.4% (95% CI, 0.2%-0.7%) in the derivation sample and 0.9% (95% CI, 0.4%-1.6%) in the internal validation sample (P = .05). The rate of nonfatal cardiogenic shock or cardiorespiratory arrest among low-risk inpatients was 0.7% (95% CI, 0.4%-1.2%) in the derivation sample and 0.9% (95% CI, 0.4%-1.6%) in the internal validation sample (P = .58).

In the external validation sample, our prediction rule classified 33.9% (95% CI, 27.7%-40.6%) of patients as low risk, a higher proportion than in the derivation sample (P<.001) (Table 3). None of the low-risk patients in the external validation group died within 7 days or 30 days of the initial examination (P>.99 for both groups compared with the derivation sample). During the 3-month follow-up period, none of the low-risk patients in the external validation sample died, had recurrent venous thromboembolism, or experienced a major bleeding episode.

The rule had a high sensitivity (range, 97%-100%) and a high negative predictive value (range, 98%-100%) for predicting 30-day mortality (Table 4). Because the prediction rule was specifically designed to identify low-risk patients (ie, to rule out short-term mortality), the specificity (range, 23%-35%) and positive predictive value (range, 4%-12%) were low.

We developed a simple clinical prediction rule based on 10 demographic, history, and clinical findings to identify low-risk patients with PE. Among large derivation and internal validation samples of patients with PE, our rule identified more than one fifth of patients at low risk of short-term mortality and serious medical complications. In an independent validation cohort of patients with PE, we confirmed the accuracy of our prediction rule: none of the patients classified as low risk died, experienced recurrent venous thromboembolism, or had major bleeding during a 3-month follow-up. Overall, our rule had a negative predictive value for 30-day mortality of at least 98% across the derivation and 2 validation samples.

The potential clinical and economic benefit of our prediction rule can be estimated using data from a recent cost-effectiveness analysis comparing inpatient treatment with unfractionated heparin vs low-molecular-weight heparin in patients with PE.⁴⁰ Treatment with low-molecular-weight heparin was cost saving when at least 5% of patients were treated as outpatients or 8% were discharged early. Assuming a cost difference of $4500 between inpatient and outpatient treatment of PE⁴⁰ and an annual PE incidence of 101 000 cases,³ up to $91 million per year could be saved in the United States if 20% of patients were treated as outpatients. Therefore, treating patients with PE identified as low risk using our prediction rule in an ambulatory setting could result in important cost savings. However, at the initial site of treatment decision for patients with PE, it is important for physicians also to consider psychosocial contraindications to outpatient care (eg, lack of treatment adherence). Other potential barriers to outpatient treatment are the lack of outpatient systems of health care and the absence of insurance coverage for more costly low-molecular-weight heparin.

Our prediction rule consists of 10 clinical prognostic factors that are routinely available in all hospital settings and that were previously shown to be associated with adverse outcomes among patients with PE and other acute diseases.^16-20,22 Compared with a previous prognostic model for PE,^20,41 our prediction rule has distinctive strengths. First, our rule consists of clearly defined, routinely available predictors and does not require any laboratory tests or radiographic procedures not routinely performed in the management of PE. Second, the accuracy and generalizability of the rule are supported by its derivation and internal and external validation in 15 752 patients from 189 hospitals and 3 countries. Third, our study samples represent a broad disease spectrum, ranging from nonmassive PE to PE with cardiorespiratory arrest.

Investigators in a prior study⁹ successfully treated 81 (51%) of 158 patients with PE as outpatients using low-molecular-weight heparin. Patients without arterial hypotension, arterial hypoxemia, pain requiring intravenous narcotics, social contraindications to outpatient treatment, and comorbid conditions necessitating hospital treatment were eligible for that study, although the comorbid conditions requiring hospitalization were not specified. Moreover, patients enrolled in that study were younger and potentially healthier than the patients in our study samples, which may have resulted in a higher proportion of patients considered as low risk. In contrast to the unspecific eligibility criteria of the prior study,⁹ our prediction rule provides clinicians a set of explicit criteria to identify low-risk patients with PE.

However, patients classified as higher risk by our prediction rule (eg, patients with cancer) may choose to be treated as outpatients even if their short-term prognosis is worse than that in low-risk patients. Until randomized trials comparing inpatient vs outpatient treatment of PE are conducted, it remains uncertain whether the initial site of treatment affects mortality rates.

Our work has potential limitations. First, patients in our derivation and internal validation samples were identified using ICD-9-CM codes for PE rather than standardized clinical criteria and may be subject to study selection biases because of hospital coding procedures. However, prior studies^42-44 demonstrated that 94% to 96% of patients with specific ICD-9-CM codes for PE had objectively documented disease based on medical record review criteria. Second, we cannot exclude the possibility that patients who were identified using a primary ICD-9-CM code for conditions that may represent complications of PE (eg, cardiogenic shock) developed PE as a consequence of these conditions. However, the performance of our prediction rule did not change when these patients were excluded from analysis. Third, we had no information about the timeliness or type of treatments received (eg, type of heparin) among our study samples. Therefore, we could not assess whether patients who were treated differently experienced different outcomes. Fourth, we externally validated our rule using data from a prior prospective study³⁹ that was not designed to validate our prediction rule. The small sample size in that study resulted in wide 95% CIs for mortality rates. Although the healthier patients in the external validation sample may not reflect the full prognostic spectrum of patients with PE, no deaths occurred in the subgroup that was identified as low risk by our prediction rule.

We derived and validated a clinical prediction rule that accurately identifies a substantial proportion of patients with PE who are at low risk of death and other adverse outcomes and who are candidates for less costly outpatient treatment. However, before this prediction rule can be considered ready for use in clinical practice, it should be validated in a prospective study.

Correspondence: Drahomir Aujesky, MD, MSc, Service de Médecine Interne, Centre Hospitalier Universitaire Vaudois, BH 10-622, 1011 Lausanne, Switzerland (aujesky@swissonline.ch).

Accepted for Publication: June 21, 2005.

Author Contributions: Dr Aujesky had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Financial Disclosure: None.

Funding/Support: This study was funded by grant 1 R21 HL075521-01A1 from the National Heart, Lung, and Blood Institute, Bethesda, Md; by the Swiss Foundation in Medicine and Biology, Bern, and the Swiss Medical Association, Bern (Dr Aujesky); and by a K24 Career Development Award from the National Institute of Allergy and Infectious Diseases, Bethesda (Dr Fine).

Acknowledgment: We thank Kenneth J. Smith, MD, for his careful review of the manuscript and his comments on the analysis.