Identification of Patients With Stable Chest Pain Deriving Minimal Value From Noninvasive Testing: The PROMISE Minimal-Risk Tool, A Secondary Analysis of a Randomized Clinical Trial

Key Points

Question  Is it possible to create a risk tool to identify intermediate-risk patients with stable chest pain unlikely to benefit from noninvasive testing?

Findings  In this secondary analysis of a randomized clinical trial, 1156 of 4632 patients (25.0%) with stable chest pain had normal coronary arteries (without atherosclerosis) and no long-term clinical events. These minimal-risk patients can be identified with good discrimination using pretest clinical characteristics alone.

Meaning  A clinical tool using readily available pretest variables discriminates such minimal-risk patients, for whom deferred testing may be considered.

Importance  Guidelines recommend noninvasive testing for patients with stable chest pain, although many subsequently have normal test results and no adverse clinical events.

Objective  To describe a risk tool developed to use only pretest clinical data to identify patients with chest pain with normal coronary arteries and no clinical events during follow-up (minimal-risk cohort).

Design, Setting, and Participants  This secondary analysis of a randomized, pragmatic comparative effectiveness trial (Prospective Multicenter Imaging Study for Evaluation of Chest Pain [PROMISE]) includes stable, symptomatic outpatients without known coronary artery disease referred for noninvasive testing at 193 sites in North America.

Interventions  Patients were randomized to receive coronary computed tomography angiography (CCTA) vs functional testing.

Main Outcomes and Measures  A low-risk tool was developed and internally validated from July 27, 2010, to September 19, 2013, in 4632 patients receiving CCTA as their initial test, with a median follow-up of 25 months. Logistic regression analysis was used to evaluate pretest variables to determine factors associated with minimal risk using a two-thirds random sample for model derivation (n = 3104) and a one-third sample for testing and validation (n = 1528). The model was then applied to the CCTA and functional testing arms, and test results and event rates were ascertained.

Results  A total of 1156 of 4632 patients (25.0%) were in the minimal-risk cohort. The final minimal-risk model included 10 clinical variables that together were correlated with normal CCTA results and no clinical events (C statistic = 0.730 for the derivation subset; 95% CI, 0.710-0.750): younger age; female sex; racial or ethnic minority; no history of hypertension, diabetes, dyslipidemia, or family history of premature coronary artery disease; never smoking; symptoms unrelated to physical or mental stress; and higher high-density lipoprotein cholesterol level. Across the entire PROMISE cohort, this model was associated with the lowest rates of severely abnormal test results (1.5% for CCTA; 4.9% for functional) and cardiovascular death or myocardial infarction (0.2% for a median of 25 months) among patients at the highest probability (10th decile) of minimal risk.

Conclusions and Relevance  In contemporary practice, more than 25% of patients with stable chest pain referred for noninvasive testing will have normal coronary arteries and no long-term clinical events. A clinical tool using readily available pretest variables discriminates such minimal-risk patients, for whom deferred testing may be considered.

Trial Registration  clinicaltrials.gov Identifier: NCT01174550

Current guidelines recommend noninvasive testing in symptomatic patients with intermediate pretest probability of obstructive coronary artery disease (CAD).¹ However, studies of outpatients with a clinical syndrome of possible ischemia found that most functional test results are normal, and more than 95% of those patients will not experience an untoward clinical event during 2 years of follow-up,^2-5 even among those thought to be at intermediate pretest likelihood of CAD by traditional risk assessment tools, such as the Diamond-Forrester risk score.^6-9 These findings have led to substantial concerns about overtesting¹⁰ and development of appropriate use criteria to guide clinical decision making¹¹ but little in the way of novel approaches to quantitative risk assessment that might better guide decision making.

Patients unlikely to have CAD, clinical events, or revascularization will derive minimal benefit and value from noninvasive testing. If such patients can be identified before testing, the need for testing and its associated costs might be reduced. Safely deferred testing even in a modest percentage of patients would be an attractive strategy in an era when health care systems are accountable for costs and outcomes.¹²

We hypothesized that the highly annotated baseline demographic and clinical pretest data available in the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) trial would allow creation of a risk model useful for identifying patients unlikely to benefit from noninvasive testing, defined as those with normal test results (including no coronary atherosclerosis) and no clinical events during the subsequent 2 years of follow-up. We then used the model to develop a clinical tool to aid health care professionals in identifying patients who may derive minimal or no value from noninvasive testing for whom deferring testing may be considered.

PROMISE was a pragmatic comparative effectiveness trial that enrolled 10 003 evaluable patients at 193 sites in North America representing community practices and academic medical centers. The PROMISE study design and primary results have been described in detail.^7,13 The study enrolled stable symptomatic outpatients without known CAD referred for noninvasive testing for further evaluation from July 27, 2010, to September 19, 2013. Patients were randomized to an initial anatomical testing strategy with coronary computed tomography angiography (CCTA) or a functional testing strategy (exercise treadmill testing, stress echocardiography, or stress nuclear imaging) and then followed up for a median of 25 months for outcome events. The study protocol was approved by the local or central institutional review board at each coordinating center and at each of the 193 enrolling sites in North America (eAppendix 1 in the Supplement). Written informed consent was obtained from all patients.

Of the 10 003 patients randomized in PROMISE, the 4633 patients who were randomized to and received CCTA as an initial noninvasive test served as the initial analytic group in this secondary analysis of PROMISE (Figure). Only patients receiving CCTA as their initial test were included in model development because the definition of normal in this cohort included those without any detectable atherosclerosis who are at the lowest risk of adverse events.¹⁴ Patients were categorized as being at minimal risk if they underwent CCTA that produced normal results and if all of the following conditions defined a priori were met: (1) coronary artery calcium score was 0 or was not obtained; (2) no evidence of atherosclerosis; (3) overall study quality was diagnostic (ie, sufficient data quality for interpretation); (4) left ventricular function was normal or not reported; (5) no wall motion abnormalities were present or not reported; and (6) no relevant cardiovascular incidental findings that could account for the patients’ symptoms (ie, aortic dissection or pulmonary embolism) were noted. All patients with normal CCTA results were included in the minimal-risk cohort in the absence of any of the following adjudicated clinical events during the median 25-month follow-up period: all-cause death, nonfatal myocardial infarction (MI), unstable angina hospitalization, or revascularization during the entire follow-up period. The final study cohort consisted of 1156 minimal-risk patients and 3476 other patients (n = 4632) (Figure).

Continuous baseline characteristics were summarized using means and SDs or medians with 25th and 75th percentiles. Categorical variables were summarized using frequencies and percentages. Group comparisons with respect to continuous baseline variables were performed using the Wilcoxon rank sum test; the Pearson χ² test or Fisher exact test was used for comparisons that involved categorical variables. Statistical significance was set at α = .05. All analyses were performed with SAS statistical software, version 9.4 or JMP Pro, version 12.1 (SAS Institute Inc).

Complete data were available for most of the baseline variables included in the a priori–specified list of interest. However, for total and high-density lipoprotein cholesterol, which had more than 5% missingness, imputation was performed with a single imputation procedure, namely, the fully conditional specification method using PROC MI in SAS, version 9.4.

Using the final study cohort, we performed a random two-thirds to one-third data split to create a model development subset (n = 3104) and a model testing and validation subset (n = 1528) (Figure). In the model development subset, multivariable logistic regression analysis was used to identify baseline (pretest) variables that were useful in discriminating the minimal-risk patients from the other patients and creating a model for identifying which patients were likely to fall into the minimal-risk category. For continuous baseline variables, we examined the shape and strength of the association between individual variables and whether patients were in the minimal-risk group using a flexible model-fitting approach that involved restricted cubic spline functions (cubic polynomials).¹⁵ Categorical variables with more than 3 levels were collapsed to 3 clinically meaningful categories. After examining univariate associations, a multivariable logistic regression model for assessing the probability of minimal risk was fit using the backward variable selection method with all the a priori–specified covariates that were significantly different at the α = .05 level between the minimal-risk and other groups or were considered clinically relevant. The model derived in the model development data set, including the variables selected and their corresponding variable estimates, was then applied to the testing and validation data set to assess the model’s performance in an independent sample.

The measure of predictive discrimination used to characterize model performance in the model development and validation samples was the area under the receiver operating characteristic curve (AUC) or C statistic.¹⁶ Calibration of the model predictions was assessed by graphic comparison of the observed proportion of minimal-risk patients to the mean model prediction across deciles of the potential probabilities of minimal risk. In addition, the Hosmer-Lemeshow statistic was calculated to statistically assess model calibration. Finally, a sensitivity analysis using bootstrapping as an alternative model-building strategy was performed as a comparison to the initial model-building approach. The mean C statistic was computed and reported based on 1000 bootstrap samples.

With the use of the final model, the potential probability of minimal risk was computed for each person in the CCTA and functional testing cohorts. For each cohort, the distribution of potential probabilities was divided into deciles, with the first decile including patients with the lowest potential probabilities of minimal risk (ie, the patients least likely to be minimal risk) and the 10th decile including patients with the highest potential probabilities (ie, the patients most likely to be minimal risk). Additional details of the model application and the comparison to other risk scores are provided in eAppendix 2 in the Supplement.

After validation of the model, the variables selected in the model development phase were refit in a final multivariable logistic regression model using the entire CCTA cohort (n = 4632) to obtain more robust variable estimates for application to other patient cohorts. The resulting C statistic was reported, as well as odds ratios and 95% CIs (as estimated by the Mann-Whitney test), for each variable in the model.

Of the 4632 patients in the CCTA arm, 1156 patients (25.0%) had a completely normal CCTA result and no clinical events during the median 25-month follow-up period; these patients were categorized as the minimal-risk cohort (Figure). In contrast, 3476 patients (75.0%) had an abnormal CCTA result and/or experienced a clinical event; they were categorized as the other cohort. Among patients in the derivation subset (two-thirds of patients, n = 3104), there were many statistically significant differences in baseline clinical characteristics between the minimal-risk (n = 775; mean [SD] age, 57.5 [7.0] years; 484 females [62.5%]; 200 racial or ethnic minority [26.0%]) and other (n = 2329) patients (Table 1). A complete list of baseline clinical characteristics for the entire CCTA study population, including all variables considered for model building, is given in eTable 1 in the Supplement.

On the basis of the multivariable analysis in the model development subset, 10 pretest clinical variables independently assessed the probability of patients being categorized as minimal risk (Table 2). The C statistic was 0.730 (95% CI, 0.710-0.750). Using the variable estimates obtained in the model development subset and applying the model to the independent validation subset, which consisted of 1528 randomly selected patients undergoing CCTA, yielded a nearly identical C statistic of 0.713 (95% CI, 0.684-0.742); when the model was applied to the entire CCTA population, the C statistic was 0.725 (95% CI, 0.709-0.741). Sensitivity analyses using bootstrapping resulted in a similar C statistic of 0.723.

Calibration of the model when applied to the independent validation data set was examined by comparing the observed proportion of minimal-risk patients with the mean potential probability of minimal risk by deciles of the potential probabilities (eFigure 1 in the Supplement). Excellent calibration is demonstrated graphically and by the Hosmer–Lemeshow calibration statistic (χ² = 5.91, P = .66).

With the variables selected, the minimal-risk model was refit in the full PROMISE CCTA cohort (n = 4632). As reported in Table 3, as the probability of minimal risk increased, the probability of a normal CCTA also increased, whereas the probability decreased of an abnormal or severely abnormal test result. As outlined in eTable 2 in the Supplement, a severely abnormal CCTA result was defined as having 1 of the following: 2-vessel disease (≥70%) or 50% or more left main artery stenosis or 70% or more proximal left anterior descending artery stenosis. Similar data were found when the model was applied to the functional testing cohort. However, compared with the CCTA cohort, at the highest probability of minimal risk, patients in the functional arm had greater rates of normal test results (84.0% vs 65.6%) and severely abnormal test results (4.9% vs 1.5%). In the entire study population (combined CCTA and functional cohorts), as the probability of minimal risk increased, the probability of cardiovascular death and MI decreased, with the lowest event rate among patients with the highest probability of minimal risk (0.2% during a median 25 months of follow-up; 10th decile or 0.1% per year).

eFigure 2 in the Supplement shows the AUCs for the PROMISE minimal-risk model compared with the Framingham and Diamond-Forrester/Coronary Artery Surgery Study (CASS) risk scores for the assessment of minimal risk when applied to the entire CCTA population. The PROMISE minimal-risk model (AUC = 0.725; 95% CI, 0.709-0.741) better characterized patients as minimal risk compared with the Framingham (AUC = 0.687; 95% CI, 0.670-0.704) and Diamond-Forrester/CASS risk scores (AUC = 0.610; 95% CI, 0.592-0.628).

As a patient example, consider a 50-year-old white woman with atypical chest pain, a medical history of hypertension, and a high-density lipoprotein cholesterol level of 70 mg/dL (to convert to millimoles per liter, multiply by 0.0259) is in the 10th decile of minimal risk. On average, patients in the 10th decile from PROMISE had a 65.6% probability of having a normal CCTA result, a 1.5% chance of a severely abnormal CCTA result, and a 0.2% risk of cardiovascular death or nonfatal MI during a 25-month follow-up period. The individual patient whose data are presented is estimated to have a probability of 0.58 of being at minimal risk (normal CCTA result and no clinical events).

Among PROMISE patients undergoing evaluation for stable chest pain with an intermediate pretest probability of CAD, 25.0% had a normal CCTA result, no clinical events, and no revascularization during a median 25 months of follow-up. A model that included only pretest demographic and clinical data identified this minimal-risk cohort with good discrimination (C statistic = 0.730) and improved existing risk scores. From that model we developed a pretest decision support tool to help clinicians identify patients with stable chest pain who are likely to derive minimal benefit and value from testing.

The ability to identify a subset of intermediate pretest probability patients with stable chest pain who might safely defer noninvasive testing is appealing given concerns about the low yield of testing in current practice and the associated costs. Studies^2-5,7 suggest that the results of most functional stress studies on outpatients with a clinical syndrome of possible ischemia are normal, and nearly all such patients will not experience an untoward clinical event. To our knowledge, few prior analyses have attempted to identify and validate a set of descriptors that could be used to help clinicians identify a subset of symptomatic, intermediate-likelihood patients who are likely to receive little or no benefit from noninvasive testing. The concept of not testing such patients is commensurate with guideline recommendations to not test unless the pretest probability of obstructive disease is greater than 10%.¹

The model used 10 pretest variables commonly available during the evaluation of a symptomatic patient with chest pain; we incorporated these variables into a simple risk assessment tool, allowing individual risk estimation. This tool provides useful information in terms of the probability of minimal risk but also the probabilities of an individual patient having any abnormal or a severely abnormal test result (CCTA and functional) or a clinical event. Although the clinical threshold to recommend noninvasive testing may vary, it may be reasonable to consider deferral of noninvasive testing for some patients. We acknowledge that approximately one-third of patients in the 10th decile (highest probability of minimal risk) still have some mild degree of coronary atherosclerosis (Table 3). However, few in this group have obstructive disease, and even fewer have a subsequent adverse event during follow-up. To put this information into context, patients in our study with the highest probability of minimal risk had a lower risk profile (0.2% risk of cardiovascular death and MI at a median 25 months of follow-up) compared with a disease-free, asymptomatic 50-year-old cohort from the Framingham Heart Study (1.4% annual risk of MI, coronary insufficiency, cardiovascular death, angina pectoris, atherothrombotic stroke, and intermittent claudication for males and 0.7% for females).¹⁷

The analysis has several advantages. Data from the patients in the CCTA arm of PROMISE were used to develop the risk tool model because CCTA can identify patients with normal coronary arteries with high accuracy,^18,19 and having no atherosclerosis represents the lowest-risk patients.¹⁴ The pragmatic nature of PROMISE well reflects contemporary community practice. The large size of the PROMISE CCTA database allowed cohorts with large sample sizes, as well as the creation of distinct derivation and validation groups. Furthermore, application of the model derived and validated within the CCTA arm to the PROMISE functional testing arm supports its generalizability, with the lowest event rates among patients with the greatest probability of minimal risk (10th decile) in both arms. Therefore, clinicians who are considering functional testing may also contemplate using this risk tool because its risk outputs are not specific to the intended test result type. Finally, the model itself has good discrimination (C statistic = 0.730), consistent validation (including using sensitivity analyses with bootstrapping), and excellent calibration. Importantly, this model identifies the minimal-risk patient cohort better than the traditional Diamond-Forrester/CASS and Framingham risk scores that are currently in clinical use.

There are some advantages of this PROMISE clinical tool compared with other risk scores. The combined Diamond-Forrester/CASS risk score, currently recommended in the 2012 American College of Cardiology/American Heart Association guidelines,¹ was not designed to identify patients unlikely to benefit from testing but rather to describe the likely findings because patients were already destined to receive noninvasive or invasive testing. The first Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicenter (CONFIRM) Registry risk score²⁰ demonstrates the incremental prognostic information derived from identification of plaque burden and stenosis on CCTA, was derived in a mixed population of patients (asymptomatic and symptomatic), requires noninvasive testing by definition as part of the score, and thus does not aid in the decision to test or defer testing. A subsequent CONFIRM risk score developed using only medical history included only symptomatic patients but is limited by not reporting cardiovascular death (only MI and all-cause mortality are reported).²¹ Unlike the PROMISE risk tool, this score did not aim to identify patients without coronary plaque (only those without a coronary stenosis ≥50%), which CONFIRM itself has shown to be prognostically important. Furthermore, our risk tool is more generalizable to a full functional testing population because neither CONFIRM score was applied to patients undergoing exercise treadmill testing alone, stress echocardiography, or nuclear perfusion with pharmacologic stress. Thus, in addition to the entirely pretest nature of the variables in our model, we evaluated a symptomatic cohort and incorporated the probability of normal test results (unlike the CONFIRM and Diamond-Forrester/CASS risk scores) and freedom from downstream events (unlike the Diamond-Forrester risk score) as end points in our model.

A method to identify patients who may derive minimal benefit from testing has several potential implications for patients, clinicians, and clinical practice in general. For patients, this process can mean a reduction in testing from which they would not benefit, thereby saving time, anxiety, and cost, as well as a potential reduction in radiation exposure and false-positive test results that could lead to more invasive, unnecessary procedures. For clinicians, this tool has the potential to help optimize office-based decision making. From a practice and societal perspective, in an era in which practitioners are increasingly held accountable for costs and quality, identifying patients unlikely to benefit from potentially expensive testing and who may be managed conservatively has many potential economic and process-of-care advantages.

This study has some limitations. It is unclear how these results might apply to a higher-risk group with a greater prevalence of disease or incidence of events or those who did not consent to participate in a clinical trial. However, the results are generated from the largest contemporary, real-world evaluation of the role of noninvasive testing among patients with stable chest pain, which strengthens its generalizability. In addition, the risk model was derived and validated using the CCTA cohort and therefore does not address other valid reasons to test, including but not limited to obtaining knowledge of functional capacity or reproduction of symptoms and their association with exertion. Although the model was found to be robust when applied to a distinct validation subset with the PROMISE trial data set, further validation of the tool within other independent data samples will provide greater generalizability. Finally, our tool is not intended to act as concrete decision support to test or not test patients, which would require a randomized clinical trial for validation. Rather, it is an aid that quantifies minimal risk and potentially identifies a subset of patients who are unlikely to benefit from proceeding directly to noninvasive testing.

More than one-quarter of stable, symptomatic outpatients with suspected CAD who currently have an indication for noninvasive testing have no coronary atherosclerosis and no revascularization procedures or clinical cardiac events during a median 2-year period. These patients can be identified using pretest clinical characteristics alone, including risk factor profile, symptom characteristics, and lipid values. These variables can be used to inform a user-friendly, point-of-care decision support tool quantifying the likelihood of having a normal test result and no clinical events. If a high likelihood of a normal test result and low clinical event rate is identified, clinicians could consider, including through shared decision making, a strategy of deferred testing for such patients because the benefit and value of testing are likely to be low.

Group Information: The Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) Investigators are listed in eAppendix 1 in the Supplement.

Accepted for Publication: November 21, 2016.

Published Online: February 15, 2017. doi:10.1001/jamacardio.2016.5501

Correction: This article was corrected on November 21, 2018, to fix pervasive errors in the data caused by statistical programming and coding errors.

Corresponding Author: James E. Udelson, MD, The CardioVascular Center, Division of Cardiology, Tufts Medical Center, 750 Washington St, Boston, MA 02111 (judelson@tuftsmedicalcenter.org).

Author Contributions: Drs Fordyce and Udelson had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Fordyce, Douglas, Hoffmann, Al-Khalidi, Patel, Granger, Lee, Udelson.

Acquisition, analysis, or interpretation of data: Fordyce, Douglas, Roberts, Hoffmann, Al-Khalidi, Patel, Kostis, Mark, Lee, Udelson.

Drafting of the manuscript: Fordyce, Roberts, Udelson.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Roberts, Al-Khalidi, Lee.

Obtained funding: Douglas, Hoffmann, Patel, Mark, Lee, Udelson.

Administrative, technical, or material support: Douglas, Udelson.

Study supervision: Hoffmann, Patel.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Fordyce reported receiving support from the Clinician Investigator Program, University of British Columbia. Dr Douglas reported serving on the data safety monitoring board for GE Healthcare and receiving research grants from HeartFlow. Dr Hoffmann reported receiving research grants from HeartFlow. Dr Patel reported receiving consultant fees or honoraria from Bayer Healthcare, Genzyme, Medscape – theheart.org, and Merck & Co and receiving research grants from Agency for Healthcare Research and Quality, AstraZeneca, Jansen, Johnson & Johnson, Maquet, National Heart, Lung, and Blood Institute, and Patient-Centered Outcomes Research Institute. Dr Granger reported receiving grants and consulting fees from GlaxoSmithKline, Boehringer Ingelheim, Bristol-Myers Squibb, Daiichi Sankyo, Pfizer, Sanofi, The Medicines Company, Janssen, and Bayer; research grants from the National Institutes of Health, the US Food and Drug Administration, Medtronic Foundation, Merck & Co, Novartis, and Armetheon; and consulting fees from Hoffmann-La Roche, Lilly, AstraZeneca, Gilead, and Medtronic. Dr Kostis reported receiving consultant fees and honoraria from ACI Clinical-Janssen Pharmaceuticals, serving on the data safety monitoring board for Palatin, and receiving other support from St Jude Medical. Dr Mark reported receiving consultant fees and honoraria from Medtronic and research grants from AGA Medical, AstraZeneca, Bayer Healthcare Pharmaceuticals, BMS, Eli Lilly, Gilead, and Merck & Co. Dr Lee reported serving on the data safety monitoring board for Amgen, AstraZeneca, CardiaQ, Edwards, and Merck & Co and receiving research grants from Medtronic. Dr Udelson reported receiving consultant fees and honoraria from Lantheus Medical Imaging; serving on the data safety monitoring board for Gilead and GSK; serving as an officer, director, trustee, or in another fiduciary role for the Heart Failure Society of American Executive Council; receiving other support from Abbott Laboratories; serving as an association editor for Circulation (American Heart Association) and as an editor for Circulation Heart Failure, Pfizer/GSK, and Sunshine Heart; and receiving research grants from the National Heart, Lung, and Blood Institute and Otsuka. No other disclosures were reported.

Funding/Support: The Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) trial was funded by grants R01 HL098237, R01 HL098236, R01 HL098305, and R01 HL098235 from the National Heart, Lung, and Blood Institute.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

Disclaimer: The views expressed in this article do not necessarily represent the official views of the National Heart, Lung, and Blood Institute.