Comparison of Observed Harms and Expected Mortality Benefit for Persons in the Veterans Health Affairs Lung Cancer Screening Demonstration Project | Cancer Screening, Prevention, Control | JAMA Internal Medicine | JAMA Network
[Skip to Navigation]
Sign In
Figure.  Observed Rate of Lung Cancer Diagnosis and Predicted Effectiveness With Initial Low-Dose Computed Tomography Screening
Observed Rate of Lung Cancer Diagnosis and Predicted Effectiveness With Initial Low-Dose Computed Tomography Screening

A, Observed rate of lung cancer diagnoses (per 1000 persons screened once). B, Screening effectiveness: number needed to screen (NNS) to prevent 1 lung cancer death. Error bars indicate 95% CIs.

Table.  Outcomes of Initial Low-Dose Computed Tomography Screening According to Risk Quintile
Outcomes of Initial Low-Dose Computed Tomography Screening According to Risk Quintile
1.
Aberle  DR, Adams  AM, Berg  CD,  et al; National Lung Screening Trial Research Team.  Reduced lung-cancer mortality with low-dose computed tomographic screening.  N Engl J Med. 2011;365(5):395-409.PubMedGoogle ScholarCrossref
2.
Kinsinger  LS, Anderson  C, Kim  J,  et al.  Implementation of lung cancer screening in the Veterans Health Administration.  JAMA Intern Med. 2017;177(3):399-406.PubMedGoogle ScholarCrossref
3.
Kovalchik  SA, Tammemagi  M, Berg  CD,  et al.  Targeting of low-dose CT screening according to the risk of lung-cancer death.  N Engl J Med. 2013;369(3):245-254.PubMedGoogle ScholarCrossref
4.
Bach  PB, Elkin  EB, Pastorino  U,  et al.  Benchmarking lung cancer mortality rates in current and former smokers.  Chest. 2004;126(6):1742-1749.PubMedGoogle ScholarCrossref
5.
Ten Haaf  K, Jeon  J, Tammemägi  MC,  et al.  Risk prediction models for selection of lung cancer screening candidates: a retrospective validation study.  PLoS Med. 2017;14(4):e1002277.PubMedGoogle ScholarCrossref
6.
Pinsky  PF, Gierada  DS, Black  W,  et al.  Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment.  Ann Intern Med. 2015;162(7):485-491.PubMedGoogle ScholarCrossref
Research Letter
Less Is More
March 2018

Comparison of Observed Harms and Expected Mortality Benefit for Persons in the Veterans Health Affairs Lung Cancer Screening Demonstration Project

Author Affiliations
  • 1VA Center for Clinical Management Research, Ann Arbor, Michigan
  • 2University of Michigan Medical School, Ann Arbor
  • 3Institute for Health Policy Innovation, University of Michigan, Ann Arbor
  • 4VA Salt Lake City Center for Informatics Decision Enhancement and Surveillance (IDEAS), Salt Lake City, Utah
  • 5University of Utah School of Medicine, Salt Lake City
  • 6Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial Veterans Affairs Hospital, Bedford, Massachusetts
  • 7Boston University School of Medicine, Boston, Massachusetts
  • 8VA Portland Health Care System Center to Improve Veteran Involvement in Care, Portland, Oregon
  • 9Oregon Health & Science University School of Medicine, Portland
  • 10Health Equity and Rural Outreach Innovation Center (HEROIC), Ralph H. Johnson Veterans Affairs Hospital, Charleston, South Carolina
  • 11Medical University of South Carolina, Medicine, Charleston
JAMA Intern Med. 2018;178(3):426-428. doi:10.1001/jamainternmed.2017.8170

The Veterans Health Affairs (VHA) lung cancer screening (LCS) demonstration project identified a much higher false-positive rate following initial low-dose computed tomographic screening than did the National Lung Screening Trial (58.2% vs 26.3%).1,2 Most false-positive results (nodules not confirmed to be lung cancer [LC] after follow-up) resulted in repeated imaging, but 2.0% of people screened also required nonbeneficial downstream diagnostic evaluation to determine these nodules were not cancer.2 We sought to put these findings into context by examining how this high false-positive rate influences the harm-to-benefit ratio for higher- vs lower-risk patients.

Methods

From March 31, 2015, through June 30, 2015, 2106 patients were screened across 8 academic VAs. Screening processes and population-average outcomes for this project have been reported.2 In trials, LCS’s 20% relative risk reduction (RRR) in LC mortality did not vary by baseline LC risk,3 so we estimated each patient’s absolute risk reduction (ARR) by multiplying the 20% RRR by their baseline LC mortality risk (ARR = Baseline Risk × RRR). We estimated annual baseline LC mortality risk using the Bach risk model.4 Unlike other models, the Bach model’s inputs are obtainable in VHA’s Corporate Data Warehouse. In addition, a recent analysis indicates it is one of the best performing models.5

Next, we separated patients into risk quintiles and assessed for each: number of LC cases observed; screening effectiveness (number needed to screen [NNS] per LC death prevented); and screening efficiency (number of false-positive results and downstream diagnostic procedures [eg, advanced imaging, bronchoscopies, biopsies] per LC death prevented). Following VHA policy and as part of the VA Quality Enhancement Research Initiative, this evaluation was not considered to be research and was declared to be nonresearch quality improvement activities by the VHA National Center for Health Promotion and Disease Prevention, and the Ann Arbor Veterans Affairs Medical Center institutional review board. As a quality improvement activity, patient consent was not required. Patient data were deidentified in analyses.

Results

Patients in higher quintiles of LC risk had significantly more lung cancers diagnosed during the project, supporting the Bach model's ability to risk stratify in this population (Figure, A: 4.8 LCs per 1000 in quintile 1 vs 29.7 per 1000 in quintile 5). Initial screens were least effective for veterans in quintile 1 (lowest LC risk) (NNS of 6903) and most effective for veterans in quintile 5 (NNS of 687) (Figure). Rates of false-positive results and downstream evaluations did not differ significantly across risk quintiles (P = .52 and P = .15 for trend, respectively). That is, the overall 56.2% rate of false-positive results requiring tracking remained relatively stable across risk quintiles (95% CI, 53.1%-62.6% in quintile 1 vs 51.9%-61.5% in quintile 5), as did the overall 2.0% rate of false-positive results requiring downstream diagnostic evaluations (95% CI, 0.3%-2.6% in quintile 1 vs 1.7%-5.2%). This relationship of increasing absolute benefit and relatively stable harms enhances the favorable harm vs benefit balance for higher-risk vs lower-risk persons. The initial screen was least efficient for patients in quintile 1 (2749 false-positive results and 68 nonbeneficial diagnostic procedures per LC death prevented) and most efficient for those in quintile 5 (eg, 363 false-positive results and 22 nonbeneficial diagnostic procedures per death prevented) (Table).

Discussion

The high rate of false-positive results identified in the VHA’s LCS demonstration project has caused concern about whether LCS should be implemented in this population. We reexamined these data and found that the high false-positive rate results in a more concerning harm-to-benefit ratio for those eligible persons at lower LC risk, but a much better harm-to-benefit ratio for high-risk patients (Table). We found that even given these very high false-positive rates, the overall balance of pros and cons among patients at high LC risk still surpasses those of most established cancer screening programs.

These results should be interpreted with several caveats in mind. The high rate of false-positive results found in the VA demonstration project may represent a substantial overestimate of future rates for 2 reasons: (1) initial screens likely have more false-positive results than recurrent screening, and (2) newer nodule management guidelines are showing great promise in lowering false-positive rates.6 Reducing the rate of false-positive findings would improve the harm-to-benefit balance for all quintiles. However, our analysis did not include all potential harms of LCS, such as overdiagnosis and psychological effects from false-positive results. In addition, effectiveness studies are still needed to confirm the extent to which the mortality benefit observed in the National Lung Screening Trial, a 20.0% reduction in lung cancer and a 6.7% reduction in all-cause mortality,1 applies in actual practice.

These real-world findings reinforce the need to risk-stratify patients for LCS and provide support for personalized, risk-based harm-benefit estimates for all eligible persons during LCS decision-making.

Back to top
Article Information

Corresponding Author: Tanner J. Caverly, MD, MPH, VA Center for Clinical Management Research and University of Michigan Medical School, 2800 Plymouth Rd, Building 16, Room 321, Ann Arbor, MI 48109 (tcaverly@med.umich.edu).

Accepted for Publication: November 27, 2017.

Published Online: January 22, 2018. doi:10.1001/jamainternmed.2017.8170

Author Contributions: Dr Caverly had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Caverly, Fagerlin, Slatore, Yun, Hayward.

Acquisition, analysis, or interpretation of data: Caverly, Wiener, Tanner, Yun, Hayward.

Drafting of the manuscript: Caverly.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Caverly, Hayward.

Obtained funding: Caverly.

Administrative, technical, or material support: Caverly, Yun.

Study supervision: Caverly, Fagerlin.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding for this study was provided by the US Department of Veterans Affairs (VA) Quality Enhancement Research Initiative. Dr Caverly is coinvestigator on a research grant from Genentech’s Corporate Giving Scientific Project Support Program that is unrelated to this study and unrelated to any Genentech or Roche products. No other disclosures are reported.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: All authors were employees of the VA at the time this work was conducted. The views expressed in this article are those of the authors and do not necessarily represent the views of the VA or the US Government.

Meeting Presentation: An earlier version of this work was an oral presentation at the 2017 Veterans Affairs Health Services Research & Development (HSR&D)/Quality Improvement Enhancement Initiative (QUERI) National Conference; July 18-20, 2017; Arlington, Virginia.

References
1.
Aberle  DR, Adams  AM, Berg  CD,  et al; National Lung Screening Trial Research Team.  Reduced lung-cancer mortality with low-dose computed tomographic screening.  N Engl J Med. 2011;365(5):395-409.PubMedGoogle ScholarCrossref
2.
Kinsinger  LS, Anderson  C, Kim  J,  et al.  Implementation of lung cancer screening in the Veterans Health Administration.  JAMA Intern Med. 2017;177(3):399-406.PubMedGoogle ScholarCrossref
3.
Kovalchik  SA, Tammemagi  M, Berg  CD,  et al.  Targeting of low-dose CT screening according to the risk of lung-cancer death.  N Engl J Med. 2013;369(3):245-254.PubMedGoogle ScholarCrossref
4.
Bach  PB, Elkin  EB, Pastorino  U,  et al.  Benchmarking lung cancer mortality rates in current and former smokers.  Chest. 2004;126(6):1742-1749.PubMedGoogle ScholarCrossref
5.
Ten Haaf  K, Jeon  J, Tammemägi  MC,  et al.  Risk prediction models for selection of lung cancer screening candidates: a retrospective validation study.  PLoS Med. 2017;14(4):e1002277.PubMedGoogle ScholarCrossref
6.
Pinsky  PF, Gierada  DS, Black  W,  et al.  Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment.  Ann Intern Med. 2015;162(7):485-491.PubMedGoogle ScholarCrossref
×