Assessment of Temporal Selection Bias in Genomic Testing in a Cohort of Patients With Cancer

Molecular tumor profiling is now routinely performed in cancer care and research. At scale, linked tumor profiles and clinical outcomes could enable researchers to develop rich predictors of treatment effectiveness. However, profiling may be ordered in clinical practice specifically to inform treatment for worsening cancer, introducing selection bias¹ into secondary analysis of genomic data. This would manifest not only as systematic differences between patients who entered a cohort and those who did not, but also as differences by timing of cohort entry. This analysis was conducted to evaluate such temporal selection bias in a large clinicogenomic data set.

A retrospective single-institution cohort study was conducted of patients with lung, breast, colorectal, pancreatic, or urothelial cancer (all stages) who had tumor profiling under a next-generation sequencing protocol from July 1, 2013 to December 31, 2017.² This protocol was approved by the Dana-Farber/Harvard Cancer Center Institutional Review Board; informed consent was obtained for patients who had profiling on a research basis and waived for patients who had profiling ordered as a standard clinical test. Results are presented according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

The association between time from diagnosis to genomic testing (left truncation time) and time from diagnosis to death was measured using the inverse probability weighted conditional Kendall τ statistic (Tc),^3,4 which can be interpreted as a nonparametric association between left truncation and survival times. Negative values indicate that longer times to testing correlate with survival shorter than expected in the absence of dependent left truncation; positive values indicate the opposite. Hypothesis testing evaluated the null hypothesis that left truncation and survival times were quasi-independent,³ that is, that Tc was zero. A P value of .05 (2-tailed) was considered significant. Data were evaluated from October 1, 2019, to March 27, 2020.

The cohort included 4777 patients (mean [SD] age, 62.8 [12.7] years; 1767 [36.9%] men and 3010 [63.0%] women); the distribution of cancer histology included lung (2009 [42.1%]), breast (1210 [25.3%]), colorectal (823 [17.2%]), urothelial (413 [8.6%]), and pancreatic (322 [6.7%]). The overall Tc was –0.18 (95% CI, –0.21 to –0.15; P < .001), indicating that patients undergoing genomic testing later in their disease trajectories had survival shorter than would be expected if time to testing were independent of clinical risk. In subgroup analyses, this association was observed for patients initially diagnosed with stage I (Tc, −0.18; 95% CI, −0.32 to −0.05), stage II (Tc, −0.16; 95% CI, −0.27 to −0.06), and stage III (Tc, −0.24; 95% CI, −0.32 to −0.15) disease but not stage IV disease (Tc, 0.01; 95% CI, −0.03 to 0.06) (Figure, A), consistent with a pattern of genomic testing after relapse. The association was observed across breast (Tc, −0.27; 95% CI, −0.35 to −0.19), colorectal (Tc, −0.20; 95% CI, −0.29 to −0.11), lung (Tc, −0.17; 95% CI, −0.22 to −0.13), pancreatic (Tc, −0.10; 95% CI, −0.20 to 0.00), and urothelial (Tc, −0.27; 95% CI, −0.40 to −0.15) cancer sites (Figure, B).

This analysis found an association between the timing of genomic testing and mortality risk in a clinicogenomic cohort of patients with cancer. Outcome events in such a cohort are subject not just to immortal time bias (patients who died before profiling would never have been in the cohort), but also to temporal selection bias (the timing of profiling is associated with clinical risk for outcome events). Researchers should consider these biases in genomic data analyses; studies seeking predictors of recurrence in early-stage disease may be particularly vulnerable to this problem. Limitations of this study include its single-institution design; the magnitude of bias likely varies across health systems and clinical cohorts. This analysis also did not assess for a causal relationship between genomic testing and survival.

Left truncation analyses,⁵ which consider patients at risk only after genomic testing, could eliminate immortal time bias if the timing of testing were random. However, if genomic testing is ordered because of progressive cancer, left truncation time could be undesirably associated with clinical risk, systematically excluding patients with a good prognosis at each time point. Simply restricting analyses to treatments initiated after genomic profiling may also bias a cohort toward patients considered to be high risk. Instead, researchers should measure the extent of temporal selection bias by calculating the conditional Tc for their cohort. If such bias is identified, specialized methods, including copula or transformation models,⁶ can be applied to adjust for it.

Accepted for Publication: April 1, 2020.

Published: June 8, 2020. doi:10.1001/jamanetworkopen.2020.6976

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Kehl KL et al. JAMA Network Open.

Corresponding Author: Kenneth L. Kehl, MD, MPH, Dana-Farber Cancer Institute, 450 Brookline Ave, Boston, MA 02215 (kenneth_kehl@dfci.harvard.edu).

Author Contributions: Dr Kehl had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Kehl, Hassett.

Acquisition, analysis, or interpretation of data: Kehl, Schrag, Uno.

Drafting of the manuscript: Kehl, Uno.

Critical revision of the manuscript for important intellectual content: Kehl, Schrag, Hassett.

Statistical analysis: Kehl, Uno.

Obtained funding: Kehl, Schrag.

Supervision: Schrag, Hassett.

Conflict of Interest Disclosures: Dr Kehl reported serving as a consultant/advisor to and receiving personal fees from Aetion outside the submitted work. Dr Schrag reported serving as a consultant/advisor to and receiving personal fees from Pfizer, receiving research funding from the American Association of Cancer Research, serving as an editor and receiving personal fees from JAMA, and receiving nonfinancial support from Grail outside the submitted work. Dr Schrag had a patent to PRISSMM, a phenomic data standard, pending during the course of this work. No other disclosures were reported.

Funding/Support: This study was supported by grant K99-CA245899 from the National Cancer Institute and the Simeon J. Fortin Charitable Foundation (Dr Kehl).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.