Development and Validation of Risk Models to Select Ever-Smokers for CT Lung Cancer Screening

Importance  The US Preventive Services Task Force (USPSTF) recommends computed tomography (CT) lung cancer screening for ever-smokers aged 55 to 80 years who have smoked at least 30 pack-years with no more than 15 years since quitting. However, selecting ever-smokers for screening using individualized lung cancer risk calculations may be more effective and efficient than current USPSTF recommendations.

Objective  Comparison of modeled outcomes from risk-based CT lung-screening strategies vs USPSTF recommendations.

Design, Setting, and Participants  Empirical risk models for lung cancer incidence and death in the absence of CT screening using data on ever-smokers from the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial (PLCO; 1993-2009) control group. Covariates included age; education; sex; race; smoking intensity, duration, and quit-years; body mass index; family history of lung cancer; and self-reported emphysema. Model validation in the chest radiography groups of the PLCO and the National Lung Screening Trial (NLST; 2002-2009), with additional validation of the death model in the National Health Interview Survey (NHIS; 1997-2001), a representative sample of the United States. Models were applied to US ever-smokers aged 50 to 80 years (NHIS 2010-2012) to estimate outcomes of risk-based selection for CT lung screening, assuming screening for all ever-smokers, yield the percent changes in lung cancer detection and death observed in the NLST.

Exposures  Annual CT lung screening for 3 years beginning at age 50 years.

Main Outcomes and Measures  For model validity: calibration (number of model-predicted cases divided by number of observed cases [estimated/observed]) and discrimination (area under curve [AUC]). For modeled screening outcomes: estimated number of screen-avertable lung cancer deaths and estimated screening effectiveness (number needed to screen [NNS] to prevent 1 lung cancer death).

Results  Lung cancer incidence and death risk models were well calibrated in PLCO and NLST. The lung cancer death model calibrated and discriminated well for US ever-smokers aged 50 to 80 years (NHIS 1997-2001: estimated/observed = 0.94 [95%CI, 0.84-1.05]; AUC, 0.78 [95%CI, 0.76-0.80]). Under USPSTF recommendations, the models estimated 9.0 million US ever-smokers would qualify for lung cancer screening and 46 488 (95% CI, 43 924-49 053) lung cancer deaths were estimated as screen-avertable over 5 years (estimated NNS, 194 [95% CI, 187-201]). In contrast, risk-based selection screening of the same number of ever-smokers (9.0 million) at highest 5-year lung cancer risk (≥1.9%) was estimated to avert 20% more deaths (55 717 [95% CI, 53 033-58 400]) and was estimated to reduce the estimated NNS by 17% (NNS, 162 [95% CI, 157-166]).

Conclusions and Relevance  Among a cohort of US ever-smokers aged 50 to 80 years, application of a risk-based model for CT screening for lung cancer compared with a model based on USPSTF recommendations was estimated to be associated with a greater number of lung cancer deaths prevented over 5 years, along with a lower NNS to prevent 1 lung cancer death.

Lung cancer is the most common cause of cancer death in the United States.¹ The National Lung Screening Trial (NLST) demonstrated a 20% reduction in lung cancer mortality with 3 rounds of low-dose computed tomography (CT) screening as compared with chest radiography, over a mean follow-up of 6.4 years.² Consequently, the US Preventive Services Task Force (USPSTF) and the US Centers for Medicare & Medicaid Services now recommend annual CT screening for a risk factor–based subgroup of smokers—current and former smokers aged 55 to 80 years and 55 to 77 years, respectively, with at least 30 pack-years of smoking and, for former smokers, no more than 15 years since quitting.^3,4 These were largely based on the entry criteria for the NLST as well as microsimulation models that considered subgroups defined by age, pack-year, and quit-year criteria.^5,6

Selecting individuals at highest lung cancer risk, as determined by individual risk calculations (ie, risk-based selection) rather than by risk factor–based subgroups, might lead to more efficient screening.^3,7-10 In the NLST, 88% of CT-prevented lung cancer deaths occurred in the 60% of participants at highest risk, whereas the 20% of participants at lowest risk accounted for only 1% of CT-prevented lung cancer deaths.¹¹ The cost-effectiveness of CT screening also increased with lung cancer risk in the NLST.¹² Risk-based selection more precisely delineates the benefits and harms of screening by accommodating detailed information on all lung cancer risk factors.³ Risk-based selection also enforces consistency of screening recommendations by accommodating “equal management of people at equal risk.”¹³ However, to our knowledge, there are currently no risk tools for lung cancer that have been validated in representative samples of the US population. Likewise, empirical evidence is lacking for the superiority of risk-based lung cancer screening in the United States.

In this study, we sought to develop and validate empirical lung cancer incidence and death risk models generalizable to US smokers, as well as an empirical model for risk of false-positive findings on screening CT. Models were applied to a contemporary cohort of US ever-smokers to investigate estimated outcomes from various risk-based selection strategies vs current USPSTF recommendations for “NLST-like” screening (3 yearly CT screens) with 5-year follow-up.

Data were used from 2 lung cancer screening trials in the United States—The Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial and the NLST—as well as data from the National Health Interview Survey (NHIS), a representative sample of the US population. From 1993-2001, the PLCO trial randomized 154 901 US men and women aged 55 to 74 years to receive 4 annual posterior-anterior chest radiographs (3 in never-smokers) or the standard of care and concluded that screening using chest radiography did not reduce lung cancer mortality.¹⁴ The most recent follow-up data for PLCO were available through December 2009. From 2002-2004, the NLST randomized 53 454 US smokers aged 55 to 74 years with at least 30 pack-years of smoking and no more than 15 years since smoking cessation to receive 3 annual rounds of low-dose CT or posterior-anterior chest radiography.² The NLST data set included outcomes accrued through January 15, 2009, the latest date for censoring lung cancer death for the primary analysis. The NHIS is an annual cross-sectional, multistage probability sample of approximately 87 500 individuals representing the noninstitutionalized civilian US population.¹⁵ NHIS data collected through 2004 have been linked with the National Death Index, with follow-up through December 31, 2006.¹⁶ The National Institutes of Health Office of Human Subjects Research deemed this study exempt from institutional review board review.

Absolute risk models were developed to predict 5-year cumulative risk of lung cancer incidence and lung cancer death using data on ever-smokers within the control group of the PLCO trial. PLCO data allowed us to develop valid models for both USPSTF-eligible and USPSTF-ineligible smokers. Cox hazard ratio models on time since interview were used and accounted for life expectancy by fitting a hazard ratio model for competing causes of death.¹⁷ Compared with previous work,¹¹ each submodel (lung cancer incidence, lung cancer death, and death by other causes) now includes more self-reported demographic variables (age, sex, race, education, body mass index [BMI]), self-reported clinical variables (history of emphysema and lung cancer family history), and self-reported smoking variables (cigarettes per day, smoking duration, and smoking pack-years and quit-years). Variables and parameterization for continuous variables were selected using the Akaike information criterion. See eMethods in the Supplement for details.

Validation of the lung cancer death model for US ever-smokers aged 50 to 80 years used NHIS surveys 1997-2001 because age at smoking initiation was not systematically collected before 1997. Via National Death Index linkage through 2006, each NHIS participant had at least 5 years of follow-up for lung cancer death. Multiple imputation was used to account for the less than 2.5% missing information on BMI, race, education, or quit-years (eTable 6 in the Supplement). Because only 5.5% of participants reported having a family history of lung cancer, it can be conservatively assumed that those with missing family history information had no family history. For former smokers, the number of cigarettes smoked per day was systematically missing, for which a special imputation model was developed (eMethods in the Supplement).

Because there are no US-representative data including both lung cancer incidence and risk factors, validation of the lung cancer incidence model used data on ever-smokers in the chest radiography groups of the PLCO and the NLST. The PLCO chest radiography group experienced only slightly increased lung cancer detection.¹⁴

Model validity was assessed by calibration (the ratio of number of model-predicted cases to the number of observed cases [estimated/observed]) and discrimination (the area under curve [AUC] statistic) (eMethods in the Supplement).

A quadrinomial logistic regression model was developed to estimate an individual’s probability of 0, 1, 2, or 3 false-positive examinations over 3 rounds of CT screening as a function of their modeled lung cancer risk. The false-positive probability is the joint probability of having a positive finding on screening CT and not being diagnosed with lung cancer within 1 year after diagnostic follow-up initiated within 1 year after the positive examination (eMethods in the Supplement). This model for false-positive risk, based on the NLST data, was assumed to hold for lung screening programs in the United States.

All P values are 2-sided; P < .05 was considered statistically significant.

The empirical models were applied to a contemporary US population of ever-smokers aged 50 to 80 years (NHIS 2010-2012), to estimate potential modeled outcomes associated with different strategies for selecting ever-smokers for NLST-like CT screening. Five-year risks of lung cancer and lung cancer death (in the absence and presence of NLST-like CT screening) and risk of false-positive CT findings were estimated for each NHIS participant. These estimates used a key assumption that NLST observations (ie, 3 yearly CT screens would reduce lung cancer mortality by 20.4% and increase lung cancer detection by 12.4% over 5 years) are applicable to all US ever-smokers, regardless of their smoking history (eMethods in the Supplement).

For each selection strategy, the models were used to estimate numbers of smokers screened, lung cancer deaths averted, lung cancers detected, and false-positive CT findings. Three screening program metrics were estimated: “screening effectiveness” (defined as the number needed to screen [NNS] to prevent 1 lung cancer death), “screening efficiency” (defined as the number of CTs with false-positive findings per prevented lung cancer death), and the number of extra lung cancers diagnosed per prevented lung cancer death.

Outcomes were compared based on USPSTF eligibility vs different strategies for selecting individuals for CT screening based on lung cancer risk thresholds. For illustrative purposes, 2 risk-based strategies were considered at length: “fixed population size”—ie, choosing the lung cancer risk threshold such that the number screened matches the number of USPSTF-eligible smokers in the United States, and “modeled fixed effectiveness”—ie, choosing the lung cancer risk threshold such that the NNS matches the NNS based on screening all USPSTF-eligible smokers in the United States.

All analyses were conducted in R¹⁸ and used the survey package¹⁹ to account for survey weights (eMethods in the Supplement).

Table 1 summarizes the data sources used to fit, validate, and apply each risk model. Table 2 reports characteristics of all cohorts. Table 3 reports risk factors in the hazard ratio submodels for lung cancer incidence, lung cancer death, and competing mortality. Predictors included age, race, sex, education, BMI, family history of lung cancer, self-reported emphysema, pack-years of smoking, duration of smoking, years since smoking cessation, and packs smoked per day (Table 3). Risk of false-positive findings on screening CT increased with lung cancer risk (eTable 1 in the Supplement).

The lung cancer incidence model was validated in the chest radiography group of the NLST (estimated/observed, 1.06 [95% CI, 0.98-1.13]; AUC, 0.70 [95% CI, 0.69-0.72]), in the PLCO radiography group ever-smokers (estimated/observed, 0.94 [95% CI, 0.87-1.02]; AUC, 0.80 [95% CI, 0.78-0.81]), and within subgroups (eTable 2 in the Supplement).

The lung cancer death model was validated for US ever-smokers aged 50 to 80 years in the 1997-2001 NHIS, both overall (estimated/observed, 0.94 [95% CI, 0.84-1.05]; AUC, 0.78 [95% CI, 0.76-0.80]) and within subgroups (eTable 3 in the Supplement). The model was also validated in PLCO radiography group ever-smokers, both overall (estimated/observed, 1.08 [95% CI, 0.97-1.20]; AUC, 0.81 [95% CI, 0.79-0.83]) and within subgroups (eTable 4 in the Supplement).

Lung cancer mortality in the NLST radiography group appears to be 24% lower than that expected from the lung cancer death model that calibrated well to PLCO and to nationally representative NHIS data. Although this does not affect internal validity of the NLST, modeling lung cancer screening outcomes for the United States was affected (eResults and eTable 4 in the Supplement).

Based on NHIS 2010-2012, there were an estimated 43.4 million ever-smokers aged 50 to 80 years in the United States. Assuming that the NLST mortality reduction of 20.4% applies also to all ever-smokers independently of exposure level, an NLST-like CT screening program was modeled to prevent an estimated 82 245 (95% CI, 79 255-85 235) lung cancer deaths over 5 years (Table 4). Screening only the 9.0 million individuals (21%) eligible by USPSTF recommendations was modeled to prevent an estimated 46 488 (95% CI, 43 924-49 053) lung cancer deaths over 5 years (57% of estimated CT-preventable deaths). Instead, the risk-based fixed population size strategy of screening the 9.0 million smokers aged 50 to 80 years at highest 5-year risk of lung cancer (≥1.9%) was modeled to prevent an estimated 55 717 (95% CI, 53 033-58 400) lung cancer deaths (68% of estimated CT-preventable deaths). This was a 20% relative increase in estimated CT-preventable deaths vs USPSTF recommendations (11% absolute increase; P < .001), with the same number of smokers screened. Furthermore, compared with USPSTF recommendations, the risk-based fixed USPSTF sample–size strategy was modeled to have greater estimated screening effectiveness (NNS, 194 [95% CI, 187-201] vs 162 [95% CI, 157-166]; P < .001) and estimated screening efficiency (fewer false-positive CT screening examinations per prevented death: 133 [95% CI, 128-137] vs 116 [95% CI, 113-119]; P < .001), while maintaining the estimated ratio of extra lung cancers diagnosed per prevented death (0.93 [95% CI, 0.93-0.94] vs 0.91 [95% CI, 0.91-0.92]; P < .001) (Table 4).

The risk-based–modeled fixed-effectiveness strategy maintains the same estimated NNS (194) as modeled for the USPSTF recommendations. This strategy would select for CT screening an extra 3.1 million individuals (12.1 million total; 28% of ever-smokers aged 50-80 years) at highest 5-year lung cancer risk (≥1.7%) and was modeled to prevent an estimated 62 382 (95% CI, 59 567-65 196) lung cancer deaths (76% of CT-preventable deaths) over 5 years. Compared with the USPSTF guidelines, this was a 34% relative increase in modeled CT-preventable deaths (19% absolute increase; P < .001), while maintaining the same estimated screening efficiency (false-positive CT screening examinations per prevented death: 134 [95% CI, 131-138] vs 133 [95% CI, 128-137]; P = .50) and the same estimated ratio of extra lung cancers diagnosed per prevented death (0.92 [95% CI, 0.92-0.93] vs 0.93 [95% CI, 0.92-0.93]; P = .006) (Table 4).

The Figure shows modeled 5-year outcomes for risk-based CT screening strategies over a range of lung cancer risk thresholds. For example, preventing 90% of CT-preventable lung cancer deaths was estimated to require screening the 49% of ever-smokers at highest lung cancer risk (≥0.7% [21.2 million people]), yielding an estimated NNS of 287 (95% CI, 279-295) per prevented death and an estimated 185 (95% CI, 181-190) false-positive examinations per prevented death. Strategies below the curve, such as USPSTF and Centers for Medicare & Medicaid Services recommendations, were estimated as having less screening effectiveness than risk-based strategies.

Risk-based strategies retain the highest-risk USPSTF-eligible smokers but replace lower-risk USPTF-eligible smokers with higher-risk USPSTF-ineligible smokers. Compared with USPSTF eligibility, risk-based screening strategies preferentially include more current smokers overall, more low-intensity long-term current smokers, and more high-intensity former smokers who have quit for more than 15 years (Table 5).

In the risk-based fixed population–size strategy, 36% of the USPSTF-eligible smokers are replaced by an equal number of USPSTF-ineligible smokers at much higher lung cancer risk (average 5-year risk, 1.3% vs 3.2%) and lower NNS (647 vs 226) (eTable 5A in the Supplement). The replacements are preferentially current smokers, aged 65 to 80 years, African American, and less educated and have lower BMI, emphysema, and a family history of lung cancer (eTable 5A). The subgroup of replacements who smoked less than 30 pack-years tend to be current long-term (≥45 years) smokers, but of these 99% smoke less than 1 pack per day and 61% smoke less than half a pack per day (eTable 5A). This subgroup is also majority women and disproportionately African American. The subgroup of replacements who quit more than 15 years ago were high-intensity smokers, almost all of whom had smoked at least 30 pack-years and 53% had smoked at least 45 pack-years (eTable 5A).

Similar conclusions hold for the risk-based–modeled fixed-effectiveness strategy, which replaces 24% of the USPSTF-eligible with more USPSTF-ineligible smokers at higher lung cancer risk (average 5-year lung cancer risk, 1.1% vs 2.6%) and lower NNS (813 vs 281) (eTable 5B in the Supplement).

Empirical individual risk models were developed, validated, and applied to US health survey data to estimate the 5-year outcomes associated with NLST-like CT lung cancer screening (3 annual screens) in the United States. The risk models validate well in US research cohorts (PLCO and NLST) as well as in the US general population (NHIS), suggesting transportability of these models. The key observation from the models is that compared with selecting risk-factor–based subgroups for screening (such as current USPSTF recommendations), individual-risk–based selection of smokers was estimated to prevent more deaths, improve screening effectiveness (defined as the NNS to prevent 1 lung cancer death), and improve screening efficiency (defined as the ratio of false-positive CT screening examinations to prevented deaths).

The superior performance of risk-based screening is highlighted by the estimate that 90% of CT-preventable lung cancer deaths are possibly preventable by a risk-based strategy that screens only 49% of US ever-smokers aged 50 to 80 years. This strategy is modeled to screen 287 individuals per prevented death, which may be as comparably effective as other cancer screening programs.²⁰

Risk-based screening strategies appear superior to USPSTF recommendations because they preferentially replace the 36% of the USPSTF-eligible who are low-risk, low-benefit, ever-smokers (5-year lung cancer risk, 1.3%; NNS, 647) with USPSTF-ineligible high-risk, high-benefit, ever-smokers (5-year lung cancer risk, 3.2%; NNS, 226). These USPSTF-ineligible high-risk individuals cannot be identified by subgroups and require a risk calculation to identify, such as the 22% who smoke less than 30 pack-years,²¹ the 13% who smoke less than a half-pack per day, or the 14% who quit smoking more than 15 years ago. Conversely, 36% of USPSTF-eligible individuals are actually at low risk and thus may benefit less from screening than would be recognized without a risk calculation. Risk-based selection would also increase the number of African Americans and women selected for CT lung screening. Post hoc NLST analyses suggest possibly higher efficacy of CT lung screening for women.²²

Substantially higher effectiveness of an NLST-like CT screening program among USPSTF-eligible smokers in the United States was estimated than that observed in the NLST² (NNS, 194 vs 320, respectively). Although including individuals aged 75 to 80 years (per USPSTF recommendations) increases effectiveness, the bigger contributor is the substantially lower lung cancer mortality in the NLST than expected based on lung cancer mortality rates in the PLCO and the United States (NHIS). Consequently, the benefits of CT screening in the United States could be higher than that observed in the NLST,² that estimated using NLST rates,²³ or that estimated by microsimulation models calibrated to the NLST.²⁴ The deficit in lung cancer mortality in the NLST was not fully explained by study-specific differences in risk factors, lung cancer incidence, treatments, or histology/stage at lung cancer diagnosis. A small mortality reduction from chest radiography in the NLST, but not PLCO, cannot be ruled out. These observations demonstrate the importance of validating risk models to population-representative data, such as the NHIS, rather than to research studies, which can have healthy-volunteer effects.^25,26

Our empirical methodology has limitations. The estimates are model-based rather than directly observed outcomes. The estimates presume the implementation of screening programs in the United States with short-term performance similar to the NLST. In particular, the key assumption was that that the 20.4% reduction in lung cancer mortality and the 12.4% increase in lung cancer detection from CT screening observed in the NLST would be the same in NLST-ineligible smokers in the US population. Notably, in the NLST, the percent mortality reduction and increased percent lung cancer detection were unrelated to modeled lung cancer risk, lending support to the validity of the assumption. The NLST-based model for false-positive CT screening examinations was assumed to apply to lung screening programs in the United States. Screening performance may change with innovation, for example, if CT findings are classified by Lung-RADS rather than NLST protocols.²⁷ Our work contrasts with microsimulation-based estimates^5,6 in that, to avoid extrapolation beyond the observed NLST follow-up, only the short-term effect of an NLST-like CT screening program in the contemporary US population was considered. Since the NHIS does not collect data on cancer incidence, the lung cancer incidence model was validated only in research cohorts. In addition, there are no external data for validating the model for false-positive risk.

Risk-based selection for screening is justified only if the benefits and harms of screening primarily depend on individual cancer risk, in that 2 individuals with different combinations of risk factors but equal modeled risk would have similar outcomes. If true, this implies the principle of “equal management of people at equal risk,” which provides an intellectual framework for the development of simplified and consistent recommendations for risk-based precision medicine.¹³ This principle was adopted as the basis of current risk-based cervical cancer screening recommendations^28-30 and underlies the official risk-based decision aid.³¹ For lung cancer, however, this principle may not hold for certain high-risk subgroups that may be at increased risk of procedure-related complications, such as individuals with chronic obstructive pulmonary disease. Indeed, NLST individuals with multiple pulmonary comorbidities did not benefit from CT screening, despite having a high modeled lung cancer risk.¹¹

However, implementing risk-based screening in clinical practice poses many challenges. The models provide estimates that could be useful for justifying a cost-effective risk threshold to define screening eligibility. Risk thresholds could be based on either lung cancer incidence^32-36 or mortality, which are highly correlated. Although our incidence model and the PLCO_m2012 incidence model³⁶ were fit to data from the PLCO, the models use different predictors and are of different forms (our model is a Cox hazard ratio model accounting for competing mortality; PLCO_m2012 is a logistic regression model). In the clinic, accurate and user-friendly risk-based decision aids^37-39 are required to lay the foundation for shared decision making.⁴Although risk communication is challenging, it is an evolving field of research.⁴⁰ Much research remains to be done to develop and evaluate shared decision-making processes to make precision prevention a reality.

Although CT screening can reduce lung cancer mortality by approximately 20%, the majority of lung cancer deaths are not screen-preventable at this time. The best way for smokers to avoid lung cancer, and all smoking-related illness, remains to quit smoking as early as possible.

Among a cohort of US ever-smokers aged 50 to 80 years, application of a risk-based model for CT screening for lung cancer compared with a model based on USPSTF recommendations was estimated to be associated with a greater number of lung cancer deaths prevented over 5 years, along with a lower NNS to prevent 1 lung cancer death.

Corresponding Authors: Hormuzd A. Katki, PhD, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Dr, Room 7E606, Bethesda, MD 20892 (katkih@mail.nih.gov); Anil K. Chaturvedi, PhD, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Dr, Room 6E238, Bethesda, MD 20892 (chaturva@mail.nih.gov).

Published Online: May 15, 2016. doi:10.1001/jama.2016.6255.

Author Contributions: Dr Katki had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Katki, Kovalchik, Berg, Chaturvedi.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Katki, Kovalchik, Cheung, Chaturvedi.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Katki, Kovalchik, Cheung.

Obtained funding: Katki, Chaturvedi.

Administrative, technical, or material support: Katki, Berg, Chaturvedi.

Study supervision: Katki, Berg, Chaturvedi.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Berg reported receiving consulting fees from Medial EarlySign, a company developing algorithms from routine blood tests that may indicate an increased risk of malignancy. No other authors reported disclosures.

Funding/Support: This study was supported by the Intramural Research Program of the US National Institutes of Health (NIH)/National Cancer Institute.

Role of the Funder/Sponsor: The NIH had no role in the design and conduct of the study; collection, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.