Comparison of Observed Harms and Expected Mortality Benefit for Persons in the Veterans Health Affairs Lung Cancer Screening Demonstration Project

The Veterans Health Affairs (VHA) lung cancer screening (LCS) demonstration project identified a much higher false-positive rate following initial low-dose computed tomographic screening than did the National Lung Screening Trial (58.2% vs 26.3%).^1,2 Most false-positive results (nodules not confirmed to be lung cancer [LC] after follow-up) resulted in repeated imaging, but 2.0% of people screened also required nonbeneficial downstream diagnostic evaluation to determine these nodules were not cancer.² We sought to put these findings into context by examining how this high false-positive rate influences the harm-to-benefit ratio for higher- vs lower-risk patients.

From March 31, 2015, through June 30, 2015, 2106 patients were screened across 8 academic VAs. Screening processes and population-average outcomes for this project have been reported.² In trials, LCS’s 20% relative risk reduction (RRR) in LC mortality did not vary by baseline LC risk,³ so we estimated each patient’s absolute risk reduction (ARR) by multiplying the 20% RRR by their baseline LC mortality risk (ARR = Baseline Risk × RRR). We estimated annual baseline LC mortality risk using the Bach risk model.⁴ Unlike other models, the Bach model’s inputs are obtainable in VHA’s Corporate Data Warehouse. In addition, a recent analysis indicates it is one of the best performing models.⁵

Next, we separated patients into risk quintiles and assessed for each: number of LC cases observed; screening effectiveness (number needed to screen [NNS] per LC death prevented); and screening efficiency (number of false-positive results and downstream diagnostic procedures [eg, advanced imaging, bronchoscopies, biopsies] per LC death prevented). Following VHA policy and as part of the VA Quality Enhancement Research Initiative, this evaluation was not considered to be research and was declared to be nonresearch quality improvement activities by the VHA National Center for Health Promotion and Disease Prevention, and the Ann Arbor Veterans Affairs Medical Center institutional review board. As a quality improvement activity, patient consent was not required. Patient data were deidentified in analyses.

Patients in higher quintiles of LC risk had significantly more lung cancers diagnosed during the project, supporting the Bach model's ability to risk stratify in this population (Figure, A: 4.8 LCs per 1000 in quintile 1 vs 29.7 per 1000 in quintile 5). Initial screens were least effective for veterans in quintile 1 (lowest LC risk) (NNS of 6903) and most effective for veterans in quintile 5 (NNS of 687) (Figure). Rates of false-positive results and downstream evaluations did not differ significantly across risk quintiles (P = .52 and P = .15 for trend, respectively). That is, the overall 56.2% rate of false-positive results requiring tracking remained relatively stable across risk quintiles (95% CI, 53.1%-62.6% in quintile 1 vs 51.9%-61.5% in quintile 5), as did the overall 2.0% rate of false-positive results requiring downstream diagnostic evaluations (95% CI, 0.3%-2.6% in quintile 1 vs 1.7%-5.2%). This relationship of increasing absolute benefit and relatively stable harms enhances the favorable harm vs benefit balance for higher-risk vs lower-risk persons. The initial screen was least efficient for patients in quintile 1 (2749 false-positive results and 68 nonbeneficial diagnostic procedures per LC death prevented) and most efficient for those in quintile 5 (eg, 363 false-positive results and 22 nonbeneficial diagnostic procedures per death prevented) (Table).

The high rate of false-positive results identified in the VHA’s LCS demonstration project has caused concern about whether LCS should be implemented in this population. We reexamined these data and found that the high false-positive rate results in a more concerning harm-to-benefit ratio for those eligible persons at lower LC risk, but a much better harm-to-benefit ratio for high-risk patients (Table). We found that even given these very high false-positive rates, the overall balance of pros and cons among patients at high LC risk still surpasses those of most established cancer screening programs.

These results should be interpreted with several caveats in mind. The high rate of false-positive results found in the VA demonstration project may represent a substantial overestimate of future rates for 2 reasons: (1) initial screens likely have more false-positive results than recurrent screening, and (2) newer nodule management guidelines are showing great promise in lowering false-positive rates.⁶ Reducing the rate of false-positive findings would improve the harm-to-benefit balance for all quintiles. However, our analysis did not include all potential harms of LCS, such as overdiagnosis and psychological effects from false-positive results. In addition, effectiveness studies are still needed to confirm the extent to which the mortality benefit observed in the National Lung Screening Trial, a 20.0% reduction in lung cancer and a 6.7% reduction in all-cause mortality,¹ applies in actual practice.

These real-world findings reinforce the need to risk-stratify patients for LCS and provide support for personalized, risk-based harm-benefit estimates for all eligible persons during LCS decision-making.

Corresponding Author: Tanner J. Caverly, MD, MPH, VA Center for Clinical Management Research and University of Michigan Medical School, 2800 Plymouth Rd, Building 16, Room 321, Ann Arbor, MI 48109 (tcaverly@med.umich.edu).

Accepted for Publication: November 27, 2017.

Published Online: January 22, 2018. doi:10.1001/jamainternmed.2017.8170

Author Contributions: Dr Caverly had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Caverly, Fagerlin, Slatore, Yun, Hayward.

Acquisition, analysis, or interpretation of data: Caverly, Wiener, Tanner, Yun, Hayward.

Drafting of the manuscript: Caverly.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Caverly, Hayward.

Obtained funding: Caverly.

Administrative, technical, or material support: Caverly, Yun.

Study supervision: Caverly, Fagerlin.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding for this study was provided by the US Department of Veterans Affairs (VA) Quality Enhancement Research Initiative. Dr Caverly is coinvestigator on a research grant from Genentech’s Corporate Giving Scientific Project Support Program that is unrelated to this study and unrelated to any Genentech or Roche products. No other disclosures are reported.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: All authors were employees of the VA at the time this work was conducted. The views expressed in this article are those of the authors and do not necessarily represent the views of the VA or the US Government.

Meeting Presentation: An earlier version of this work was an oral presentation at the 2017 Veterans Affairs Health Services Research & Development (HSR&D)/Quality Improvement Enhancement Initiative (QUERI) National Conference; July 18-20, 2017; Arlington, Virginia.