Simplification of the Pulmonary Embolism Severity Index for Prognostication in Patients With Acute Symptomatic Pulmonary Embolism

Background  The Pulmonary Embolism Severity Index (PESI) estimates the risk of 30-day mortality in patients with acute pulmonary embolism (PE). We constructed a simplified version of the PESI.

Methods  The study retrospectively developed a simplified PESI clinical prediction rule for estimating the risk of 30-day mortality in a derivation cohort of Spanish outpatients. Simplified and original PESI performances were compared in the derivation cohort. The simplified PESI underwent retrospective external validation in an independent multinational cohort (Registro Informatizado de la Enfermedad Tromboembólica [RIETE] cohort) of outpatients.

Results  In the derivation data set, univariate logistic regression of the original 11 PESI variables led to the removal of variables that did not reach statistical significance and subsequently produced the simplified PESI that contained the variables of age, cancer, chronic cardiopulmonary disease, heart rate, systolic blood pressure, and oxyhemoglobin saturation levels. The prognostic accuracy of the original and simplified PESI scores did not differ (area under the curve, 0.75 [95% confidence interval (CI), 0.69-0.80]). The 305 of 995 patients (30.7%) who were classified as low risk by the simplified PESI had a 30-day mortality of 1.0% (95% CI, 0.0%-2.1%) compared with 10.9% (8.5%-13.2%) in the high-risk group. In the RIETE validation cohort, 2569 of 7106 patients (36.2%) who were classified as low risk by the simplified PESI had a 30-day mortality of 1.1% (95% CI, 0.7%-1.5%) compared with 8.9% (8.1%-9.8%) in the high-risk group.

Conclusion  The simplified PESI has similar prognostic accuracy and clinical utility and greater ease of use compared with the original PESI.

In the assessment and management of patients with acute symptomatic pulmonary embolism (PE), prognostic information helps to guide therapeutic decision making, such as the need for escalation of care, admission to the intensive care unit, or administration of thrombolytic therapy.¹ Moreover, accurate and objective models of prognosis could help clinicians to determine the appropriateness of early hospital discharge or complete ambulatory treatment for patients with acute symptomatic PE.

Although several prognostic models have been derived and validated in patients with acute PE,^2-6 all of them have practical limitations.⁷The Pulmonary Embolism Severity Index (PESI) was developed to estimate 30-day mortality in patients with acute PE. The PESI used objective clinical items to produce a risk stratification score. Some investigators have used the PESI to identify patients with a low mortality risk who may be suitable for home management of their acute PE.^8-11 Use of the PESI may not be practical for routine application in busy hospital emergency departments because it requires computation of a score based on 11 different variables, and each variable has a different weight.

The purpose of this study was to derive a simplified version of the PESI in which some variables of the original score would be removed and the scoring system would be simplified. To test the hypothesis that the simplified score would retain its diagnostic accuracy and clinical usefulness, we compared the performance of the original PESI and the simplified PESI in a derivation cohort. We also performed an external validation of the simplified PESI in an independent multinational cohort of outpatients with objectively confirmed acute symptomatic PE.

The study retrospectively developed a simplified PESI clinical prediction rule for estimating the risk of 30-day mortality in a derivation cohort of outpatients with acute symptomatic PE. The performance of the simplified PESI was compared against that of the original PESI in a derivation cohort. The simplified PESI underwent retrospective external validation in an independent multinational cohort of outpatients with objectively confirmed acute symptomatic PE.

The primary outcome used to derive and validate the prediction rule was all-cause mortality 30 days after diagnosis of acute symptomatic PE.

For a prospective registry, we attempted to enroll all outpatients with a diagnosis of acute PE from January 1, 2003, through October 31, 2008. Patients were recruited from the emergency department of Hospital Ramón y Cajal. The human subjects committee of the hospital approved the study, and all enrolled patients provided informed consent for their participation in the registry and future analysis of registry data in accordance with the requirements of the ethics committee of the hospital.

Eligibility for this study required that patients have acute symptomatic PE confirmed by objective testing. A diagnosis of PE was confirmed by a high-probability ventilation-perfusion scan result (according to the criteria of the Prospective Investigation of the Pulmonary Embolism Diagnosis),¹² a lower limb venous compression ultrasonogram positive for a proximal deep vein thrombosis in patients with inconclusive or nondiagnostic findings on ventilation-perfusion scans,¹³ or acute PE diagnosed on contrast-enhanced PE-protocol helical computed tomography of the chest.¹⁴

Patients in the derivation cohort were hospitalized and treated with therapeutic doses of parenteral anticoagulants (intravenous unfractionated heparin or weight-based doses of subcutaneous low-molecular-weight heparin [enoxaparin sodium]) while therapy was converted to oral vitamin K antagonist. Thrombolytic treatment was instituted in patients with confirmed PE and hemodynamic impairment as deemed appropriate by the attending physician. After completion of the initial overlap anticoagulation period, patients continued dose-adjusted oral vitamin K antagonist therapy (acenocoumarol; target international normalized ratio, 2.5 [therapeutic range, 2.0-3.0]). The international normalized range was usually monitored daily until the therapeutic range had been achieved, then 2 or 3 times weekly for the first weeks, and then once a week to once a month depending on the stability of the results. Patients who developed contraindications to anticoagulant therapy had an inferior vena cava filter placed and discontinued the anticoagulant therapy.

Mortality was assessed in the derivation cohort by using patient or proxy interviews and/or by review of the hospital medical records. Interviews were performed by telephone and administered by local study personnel. Two investigators (D.J. and V.G.) adjudicated the cause of all deaths as (1) definite fatal PE, (2) possible fatal PE, or (3) death from other causes. Cause of death was judged to be definite fatal PE if it was confirmed by autopsy or if death followed a clinically severe PE initially or shortly after an objectively confirmed recurrent event in the absence of any alternative diagnosis. Possible fatal PE consisted of death in a patient who died suddenly or unexpectedly.

We assessed the association between variables in the original PESI and death within 30 days of follow-up by means of a univariate logistic regression. Those variables that were not significantly associated with mortality were removed from the original score. Age was a significant predictor as a continuous and categorical variable. For the simplified score, we chose to dichotomize age into categories of older than 80 years and 80 years or younger because this cutoff has been previously considered clinically meaningful.¹⁵ Heart failure and chronic pulmonary disease were combined in 1 item (chronic cardiopulmonary disease). In calculating the simplified PESI for each patient, variables were assigned 1 point if the condition was present and 0 points if the condition was absent. The total points for all variables determined the simplified PESI.

In a method similar to that for the original PESI, the simplified PESI was used to categorize patients with acute symptomatic PE as being at low and high risk for death within 30 days of follow-up. Receiver operating characteristic (ROC) analysis determined the optimal simplified PESI cutoff level to identify low-risk patients. The optimum cutoff point was established by selecting the point of test values that provided the greatest sum of sensitivity and specificity (ie, the point closest to the top left-hand corner on the ROC curve).

To assess the generalizability of the simplified PESI, we externally validated it on patient data from the Registro Informatizado de la Enfermedad Tromboembólica (RIETE). The RIETE is an ongoing international, multicenter, observational registry of consecutively enrolled patients that is designed to collect and analyze data on treatment patterns and clinical outcomes in patients with acute symptomatic venous thromboembolism. The RIETE study methods have been described elsewhere.¹⁶ The validation cohort for this study consisted of the first 7106 outpatients enrolled in the RIETE who had acute symptomatic PE and complete variables and follow-up data and had not been included in this study.

Baseline characteristics for the derivation and validation cohorts are given as mean (SD) for continuous data and counts and proportions for categorical data.

Each patient's baseline characteristics determined their risk classification according to the criteria for the original and simplified predictive PESI models. Missing values for all prognostic variables were assumed to be normal, a strategy used in the original derivation of the PESI.³ For the original PESI, a total point score for a given patient is obtained by summing the patient's age in years and the points for each predictor when present. Patients in risk classes I and II are defined as low risk, whereas risk classes III through V are defined as high risk. For the simplified PESI, a total point score for a given patient is obtained by summing the points. For each prognostic model's risk classes, the proportion of patients with 30-day all-cause mortality was determined. Proportions of patients in the original and the simplified PESI risk classes and proportions of patients with 30-day all-cause mortality among groups were compared using the χ² test with Yates correction or Fisher exact test and the McNemar test.

To assess the test and performance characteristics of each prediction rule's low-risk vs high-risk categories, we calculated the sensitivity, specificity, and positive and negative predictive values. We assessed performance of the simplified PESI by evaluating discrimination through use of the overall C statistic as described by Harrell and colleagues¹⁷ and Pencina and D’Agostino.¹⁸ The discriminative value was further examined with the method described by Pencina et al.¹⁹ This method is based on the difference between 2 models in the individual estimated probability that a case subject will be categorized as a case subject. There are 2 versions of the measure. The first version, net reclassification improvement, requires that there exist a priori meaningful risk categories (we have used low and high risk of overall 30-day mortality); only those changes in estimated prediction probabilities that imply a change from one category to another are considered. The second version, integrated discrimination improvement, considers the change in the estimated prediction probabilities as a continuous variable. We evaluated model calibration with the modified Hosmer-Lemeshow χ² statistic, in which values of less than 20 indicate good calibration.²⁰

P < .05 indicated statistical significance. Analyses were conducted with a commercially available statistical software package (SPSS, version 15.0, 2006; SPSS Inc, Chicago, Illinois).

Of the 3982 patients undergoing evaluation for possible acute symptomatic PE during the study period, 1027 (25.8%) had objectively confirmed PE. Of these, 10 (1.0%) refused to give informed consent, and the study sample consisted of 1017 patients. Because 22 patients (2.2%) were lost to follow-up, the evaluable population for the derivation cohort consisted of 995 patients (96.9%).

Univariate analysis showed that the original PESI variables of sex, respiratory rate (≥30 breaths/min vs other), temperature (<36°C vs other), and altered mental status were not significantly associated with 30-day mortality. These variables were therefore not included in the simplified PESI (Table 1). The simplified PESI included the variables of age, history of cancer, history of chronic cardiopulmonary disease, heart rate (≥110 beats/min vs other), systolic blood pressure (<100 mm Hg vs other), and oxyhemoglobin saturation level (<90% vs other).

To identify low-risk patients with PE, ROC curve analysis for the simplified PESI determined that 1 point was the optimal cutoff. Patients with a score of 0 (ie, no variables present) were categorized as low risk, and those with a score of 1 or more (any variable present) were categorized as high risk.

Compared with patients in the original PESI derivation sample, patients in this study's derivation cohort were older and more likely to be male. They more frequently had cancer and an arterial oxyhemoglobin saturation level of less than 90% and less frequently had heart failure, chronic lung disease, tachycardia, tachypnea, altered mental status, or a temperature of less than 36°C (Table 2). The proportions of patients within each PESI low- and high-risk class were significantly different in the original and simplified PESI derivation cohorts (Table 3). The simplified PESI classified a significantly lower proportion of patients in this derivation cohort as low risk (305 of 995 [30.7%]) than did the original risk score (risk class I or II) (361 of 995 [36.3%]) (P = .008).

Of the 995 patients in this study's derivation cohort, 78 (7.8% [95% confidence interval (CI), 6.2%-9.5%) died within 30 days of presentation. A similar proportion of patients died in the original PESI derivation cohort (7.8% vs 9.2%; P = .17).

In this study's derivation cohort, the original and simplified PESI models had mortality rates of less than 3% in the low-risk groups (Table 3). Patients in the simplified PESI low-risk category had a trend toward lower mortality (1.0% [95% CI, 0.0%-2.1%]) compared with patients in the original PESI low-risk category (classes I and II) (2.5% mortality [0.9%-4.1%]) (P = .25). Both prediction rules appropriately showed higher mortality in the higher risk categories. In the derivation sample, only 3 patients (1.0% [95% CI, 0.1%-2.1%]) who were classified as having low risk of death according to the simplified PESI had nonfatal recurrent venous thromboembolism or nonfatal major bleeding during follow-up. In the RIETE validation cohort, 38 of 2569 patients (1.5%) who were classified as having low risk of death according to the simplified PESI had nonfatal recurrent venous thromboembolism or nonfatal major bleeding compared with 2.7% in the high-risk group (P < .01).

The simplified PESI had a higher sensitivity, a higher negative predictive value, and a lower negative likelihood ratio than the original PESI (dichotomized as low risk vs high risk) for predicting 30-day mortality in the derivation cohort, although the 95% CI for these variables overlapped (Table 4). The simplified PESI (C statistic, 0.75 [95% CI, 0.69-0.80]) and the original PESI (0.75 [0.69-0.80]) had a similar discriminatory power to predict 30-day mortality (P = .95) (Figure). The simplified PESI was well calibrated (Hosmer-Lemeshow χ² statistic, 5.13; P = .74 for the lack of fit).

For 6 patients in this study's derivation cohort who died, reclassification was more accurate when the simplified model was used; for no patients did it become less accurate. Among the patients who did not die, 50 were reclassified into a lower risk category and 96 were reclassified into a higher risk category. The net improvement in reclassification was estimated at 0.02 (P = .40) with the simplified PESI, and the integrated discrimination improvement was estimated as −0.01 (P = .41).

Of the 7106 patients included in the RIETE validation cohort, 434 (6.1% [95% CI, 5.5%-6.7%]) died during the first month of follow-up compared with 78 of 995 patients (7.8% [6.2%-9.5%]) in this study's derivation cohort (absolute risk difference, 1.7% [0.1%-2.8%]; P = .047). The simplified PESI classified 2569 of 7106 patients (36.2%) in the RIETE validation cohort as having low risk of death, and the overall 30-day mortality of this group was 1.1% (28 of 2569 patients [95% CI, 0.7%-1.5%]) compared with 8.9% (8.1%-9.8%) in the high-risk group. In the RIETE validation cohort, the simplified PESI had a negative predictive value of 98.9% (95% CI, 98.5%-99.3%) and a negative likelihood ratio of 0.17 (0.12-0.24).

This study shows that the simplified PESI successfully predicts 30-day mortality after acute symptomatic PE. Compared with the original PESI, the simplified PESI has similar prognostic accuracy. The simplified score had good discrimination and calibration, and an external data set validated the generalizability of its predictive accuracy.

The accuracy and generalizability of the original PESI are now supported by the derivation and validation in more than 17 000 patients from more than 300 teaching and nonteaching hospitals in the United States and Europe.^8-11 The original PESI reliably and accurately identifies patients at low risk of death when evaluated during follow-up ranging from 7 to 90 days. However, the original PESI uses 11 clinical variables with different assigned weights, and its scoring depends on calculations that may be difficult to apply in the clinical setting. The simplified PESI prediction rule reduces such complexity.

In the development of a clinical prediction rule suitable for use in busy emergency departments, we sought to include variables that should correlate independently with the prognosis of PE, should be easily measurable, and should serve as a surrogate for other potentially important variables. We believe that the simplified PESI is useful because it includes 1 domain that quantifies the age of the patients, 2 domains that capture coexisting illness (cancer and chronic cardiopulmonary disease), and 3 domains that express the cardiopulmonary consequences of PE (systolic blood pressure, heart rate, and arterial oxygen saturation level). Although men had significantly higher odds of short-term mortality compared with women in one study,²¹ other studies reported comparable PE case fatality rates between men and women.^22,23 In the development of the simplified PESI, sex did not meet the criteria for inclusion in the model. Regarding descriptors of pulmonary status used in the original PESI, the simplified PESI kept arterial oxygen in the model, although the model excluded respiratory rate. A few other variables included in the original PESI also were not included in the simplified PESI. Multiple studies have not provided validation of the prognostic value of hypothermia in patients with acute symptomatic PE,²⁴ and this study did not support its use in the predictive model. Few patients (0.2%) in this study had altered mental status, and it did not meet criteria for inclusion in the model.

The overall negative predictive value and negative likelihood ratio of the simplified PESI were similar to those of the original PESI. The simplified PESI performed as well as the original PESI in the derivation and validation sets. The patients from the derivation and validation sets came from management registries rather than clinical trials. Therefore, the simplified PESI could be considered accurate for identifying low-risk patients with acute PE in real-world clinical situations.

The proportion of patients who were classified by the simplified PESI as having a low risk of death at 30 days was lower than in the original score validation studies.⁷ However, this does not have a major bearing on clinical decision making because the simplified PESI still identifies a large group of low-risk patients in whom outpatient therapy for acute PE could be considered.

In a surprising finding, the area under the ROC curve of the simplified PESI score was not lower than that of the original PESI score. Given that the original PESI score assigned different weights to the individual variables, at least some loss of predictive accuracy would have been expected in the nonweighted simplified PESI. The simplified PESI model failed to show decreased accuracy compared with the original PESI model, most likely because each variable included in the simplified PESI has already proved to be a good predictor of the outcome of PE.

Some limitations of the study methods affect the findings and interpretation of the study, and future studies could further address these issues. First, we could not estimate the potential impact of treatments on patient outcomes because this information was not consistently available. Second, although investigators prospectively collected clinical data in the derivation and validation cohorts, we retrospectively calculated the simplified PESI. Finally, the simplified score was not developed to classify patients with PE into categories of increasing risk of mortality. However, the simplified PESI may be a useful tool for identifying patients estimated to be at low risk who could be discharged early or whose PE could be managed entirely in an outpatient setting.

In summary, this study demonstrates that simplification of the PESI does not decrease the score's prognostic accuracy and clinical utility. Prospective validation of the simplified PESI in a formal outcome study would add strength to the body of evidence that supports its use.

Correspondence: David Jiménez, MD, PhD, Respiratory Department and Medicine Department, Hospital Ramón y Cajal and Alcalá de Henares University, 28034 Madrid, Spain (djc_69_98@yahoo.com).

Accepted for Publication: January 14, 2010.

Author Contributions: Dr Jiménez had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Jiménez, Lobo, Monreal, and Yusen. Acquisition of data: Jiménez, Gómez, Lobo, Uresandi, and Monreal. Analysis and interpretation of data: Jiménez, Aujesky, Moores, Otero, Muriel, and Yusen. Drafting of the manuscript: Jiménez, Muriel, and Yusen. Critical revision of the manuscript for important intellectual content: Jiménez, Aujesky, Moores, Lobo, Uresandi, Otero, Monreal, Muriel, and Yusen. Statistical analysis: Jiménez, Aujesky, Muriel, and Yusen. Obtained funding: Jiménez and Monreal. Administrative, technical, and material support: Gómez. Study supervision: Jiménez, Moores, Gómez, Lobo, Otero, and Yusen.

Financial Disclosure: None reported.

Funding/Support: This study was supported in part by grants FIS 08/0200 and SEPAR 2008. The RIETE Registry was supported by an unrestricted educational grant from Sanofi-Aventis Spain and by Bayer Schering Pharma.

Role of the Sponsor: Bayer Schering Pharma's support was limited to the international part of the RIETE (excluding patients from Spain), which accounts for 10% of the total patients included in the RIETE.

Additional Contributions: The Registry Coordinating Center, S & H Medical Science Service, provided quality control and logistic and administrative support.

Box Section Ref ID