Effectiveness of a Mailed Colorectal Cancer Screening Outreach Program in Community Health Clinics: The STOP CRC Cluster Randomized Clinical Trial

Key Points

Question  How effective is a mailed fecal immunochemical test (FIT) outreach program when implemented in busy community clinic practices as part of standard care?

Findings  A mailed FIT outreach program resulted in a 3.4 percentage point increase in FIT completion in clinics that received the intervention, compared with clinics that practiced usual care. Low rates of implementation were common and were associated with low levels of effectiveness.

Meaning  Mailed FIT outreach can raise rates of colorectal cancer screening; additional strategies may be needed to support program implementation in low-resource health centers.

Importance  Approximately 24 million US individuals receive care at federally qualified health centers, which historically have low rates of colorectal cancer screening. The US Preventive Services Task Force recommends routine colorectal cancer screening for individuals aged 50 to 75 years.

Objective  To determine the effectiveness of an electronic health record (EHR)-embedded mailed fecal immunochemical test (FIT) outreach program implemented in health centers as part of standard care.

Design, Setting, and Participants  This cluster randomized pragmatic clinical trial was conducted in 26 federally qualified health center clinics, representing 8 health centers in Oregon and California, randomized to intervention (n = 13) or usual care (n = 13). All participants were overdue for colorectal cancer screening during the accrual interval (February 4, 2014 to February 3, 2015).

Interventions  Electronic health record-embedded tools to identify eligible adults and to facilitate implementation of a stepwise mailed intervention involving (1) an introductory letter, (2) a mailed FIT, and (3) a reminder letter; training, collaborative learning, and facilitation through a practice improvement process.

Main Outcomes and Measures  Effectiveness was measured as clinic-level proportions of adults who completed a FIT, and secondarily, any colorectal cancer screening within 12 months of accrual or by August 3, 2015. Implementation was measured as clinic-level proportions of adults who were mailed an introductory letter and ordered a FIT.

Results  Twenty-six clinics with 41 193 adults (mean [SD] age, 58.5 [6.3] years; 22 994 women) were randomized to receive the direct mail colorectal screening intervention (13 clinics; 21 134 patients) or usual care (13 clinics; 20 059 patients). Compared with usual care clinics, intervention clinics had significantly higher adjusted clinic-level proportion of participants who completed a FIT (13.9% vs 10.4%; difference, 3.4 percentage points; 95% CI, 0.1%-6.8%) and any colorectal cancer screening (18.3% vs 14.5%; difference, 3.8 percentage points; 95% CI, 0.6%-7.0%). We observed large variation across health centers in effectiveness (FIT completion differences range, −7.4 percentage points to 17.6 percentage points) and implementation (proportion who were mailed a FIT range, 6.5% to 68.2%). The number needed to mail to achieve a completed FIT was 4.8 overall, and 4.0 in clinics that mailed a FIT reminder.

Conclusions and Relevance  An EHR-embedded mailed FIT outreach intervention significantly improved rates of FIT completion and rates of any colorectal cancer screening. Higher rates of colorectal cancer screening occurred in clinics that successfully implemented the mailed outreach program.

Trial Registration  ClinicalTrials.gov identifier: NCT01742065

For more than 20 years, we have known that colorectal cancer (CRC) screening reduces cancer incidence and mortality.^1-3 The US Preventive Services Task Force has recommended CRC screening for adults ages 50 to 75 years since 2008.⁴ Nonetheless, screening rates remain low, particularly among underserved groups, including those served by federally qualified health centers (FQHCs).⁵ In FQHCs nationally, only 40% of eligible adults were up-to-date for CRC screening in 2016.⁶ Research is needed on how to realize the full potential of routine screening to detect and prevent CRC.

Studies have shown that mailing fecal immunochemical test (FIT) kits to individuals who are due for CRC screening can improve screening rates,^7,8 with increases in screening uptake of 22% to 45% in FQHC patient populations.^9-13 However, these studies have been small in scope or designed to determine efficacy under ideal circumstances (eg, interventions delivered by researchers rather than clinic staff).^13,14 Little is known about the large-scale effectiveness of mailed FIT outreach interventions in FQHC settings.

We conducted the Strategies and Opportunities to STOP Colon Cancer in Priority Populations (STOP CRC) study to evaluate the effectiveness of a mailed FIT intervention delivered by clinic staff at FQHCs. This pragmatic study provided 13 clinics with electronic health record (EHR) tools to identify and contact patients who were due for screening; trained clinic staff to use the tools; and compared results with 13 clinics practicing usual care. This paper presents primary outcomes from the study: we assess the effectiveness of a large-scale mailed FIT outreach approach to CRC screening and evaluate levels of implementation across a range of FQHC clinics.

Our clinic recruitment methods have been reported previously (Figure 1).¹⁵ Briefly, we included 7 FQHCs (representing 24 clinics) and 1 FQHC (representing 2 clinics) affiliated with an academic medical center. All clinics operated in Oregon and California and served similar low-income populations. All health centers were affiliated with OCHIN (formerly Oregon Community Health Information Network) and used the same Epic EHR (Vernona, WI). Health center eligibility criteria included willingness to randomize clinics and use a single fecal test across all participating clinics; having an electronic interface with the laboratory that processed the kits; and having sufficient capacity for follow-up colonoscopy, among other factors.¹⁵ The study was approved by the Institutional Review Board of Kaiser Permanente Northwest (Protocol # 4364), with ceding agreements from Kaiser Permanente Washington Research Institute and OCHIN. We obtained a waiver of informed consent because the study was embedded into standard care and the risk to patients was considered minimal. The trial protocol is available in Supplement 1.

The project statistician stratified allocation assignments by health center to balance clinic-level characteristics between study arms. Participating clinics were randomized to either usual care (n = 13) or an EHR-embedded intervention (n = 13) on February 4, 2014. Neither researchers nor clinic staff were blinded to randomization assignment. The intervention was delivered by clinic staff; eligible adults were unaware of their study participation.

Eligible adults were aged 50 to 74 years, had a clinic visit during the 12 months prior to accrual, and were due for CRC screening based on the following established criteria¹⁶: (1) no EHR evidence of completing a fecal test during the past 11 months, flexible sigmoidoscopy during the past 4 years, or colonoscopy during the past 9 years; (2) no evidence of an order for a fecal test in the past 6 months or a referral for a sigmoidoscopy or colonoscopy in the past year. We excluded adults with evidence of relevant health conditions (eg, colorectal cancer, colon disease, end-stage renal failure). Patients were identified using real-time EHR tools that were updated daily; patients were accrued into the analytic data set if they appeared on the list even for a single day. This process led to the identification of 21 134 individuals in intervention clinics and 20 059 individuals in usual care clinics over the accrual interval.

Usual care clinics continued their standard processes for CRC screening, which typically consisted of providing information and ordering tests during routine clinical encounters. These clinics were offered training and intervention materials at the end of the follow-up period.

The intervention consisted of customized EHR-embedded tools, training in the use of these tools, a collaborative environment for continued learning after initial training, and a standardized practice improvement process.¹⁷ The design of the STOP CRC EHR tools has been described previously.¹⁸ Briefly, the research team collaborated with OCHIN to customize EHR tools using Reporting Workbench in Epic. The tools identified adults who were due for CRC screening and allowed clinic staff to generate mailing lists and materials (eg, letters and labels) for 3 sequential mailings: (1) an introductory letter; (2) a FIT kit packet that included wordless instructions on how to complete the test¹⁹; and (3) a reminder letter. The tools automatically removed patients from lists if they were found to have an invalid address, reported prior CRC screening, or were excluded by a clinic.

The EHR tools were provided to staff at each intervention clinic during the initial training, along with templates for introductory letters, FIT kit inserts, and reminder letters. We encouraged clinics to customize the templates as needed. Clinic staff were responsible for integrating the intervention into their care processes. As part of the collaborative learning model, EHR site specialists from all health centers were invited to monthly meetings to discuss implementation of the intervention, and health center leadership were invited to quarterly meetings with an advisory board. In addition, we used a well-validated practice improvement approach that included Plan-Do-Study-Act (PDSA) cycles and provided a trained facilitator who offered PDSA booster training at each center.²⁰ The PDSA cycles were conducted at each health center 4 to 6 months after initial training. Each health center selected its own target for process improvement (eg, improving workflows, increasing return rates).¹⁷

To activate the EHR tools, clinic staff needed to link the EHR tools to a printer for printing labels; assure laboratory interfaces were in place, import customized introductory letters, and assign appropriate codes for laboratory orders, among other steps. Staff then used patient lists to carry out each step of the intervention. Initial eligibility lists facilitated batch-printing, stuffing, and mailing of introductory letters. Lists of patients eligible for a mailed FIT facilitated placement of FIT laboratory orders and labeling and mailing of FITs. Lists of patients due for reminders were used to batch-print and mail reminder letters. Clinic staff could choose the mailing schedule (monthly, quarterly, 1-time).

The primary outcome was clinic-level proportion of eligible adults (accrued between February 4, 2014 and February 3, 2015) who completed FIT within 12 months or through August 3, 2015 (when usual care clinics received access to study tools). The evaluation interval was February 4, 2014 to August 3, 2015. A secondary outcome was clinic-level proportion of eligible adults completing any CRC screening (fecal test, sigmoidoscopy, or colonoscopy) during the evaluation interval. We calculated implementation as the clinic-level proportion of participants who were mailed an introductory letter and were subsequently ordered a FIT (orders were placed at the time of FIT mailing) during the evaluation interval, based on EHR data. This allowed us to distinguish between mailed FITs and those distributed in-clinic.

To assess effectiveness, we fit generalized estimating equations (GEE) with a logistic link to model patient-level data. We weighted patient data so that each clinic’s data had an equal weight. Models were adjusted for age, sex, and health center. They used robust variance estimators and independent correlation structures. We specified clinic as a clustering variable to account for intraclinic correlation; the intraclass correlation coefficient was 0.05 after covariable adjustment. We report effectiveness as the absolute difference between intervention and usual care clinics in adjusted probabilities calculated using mean values for all covariates; reported P values are based on the corresponding adjusted odds ratio and account for reduced degrees of freedom owing to clustering.

We used this analytic approach both for our primary outcome (FIT completion) and our secondary outcome (any CRC screening). All analyses were 2-tailed and considered significant if P ≤ .05. The analyses were conducted from May 10, 2016 to June 5, 2018, using SAS (version 9.4, SAS Institute) and STATA statistical software (version 15.1, StataCorp). Sensitivity analysis included GEE models without weighting (in this case with an exchangeable correlation structure); GEE models that also adjusted for insurance status; GEE models that did not adjust for patient-level covariates; unweighted random effects models; and models that excluded the 2 academic medical center clinics (these clinics did not have shared governance, and the intervention clinic, but not the usual care clinic, experienced leadership turnover). Sample size calculations have been reported previously.²¹

Although our planned analyses included all individuals accrued after EHR tools were provided to clinics on February 4, 2014, no clinic began printing letters until at least June of 2014; some did not begin until spring of 2015. To account for this implementation delay, we repeated all analyses using a data set that only included individuals accrued between June 4, 2014 and February 3, 2015 (8-month interval). As with the primary data set, outcomes in this secondary analysis were assessed through August 3, 2015.

We conducted a descriptive analysis comparing FIT completion rates and proportions of patients who were mailed a FIT in cross-sectional samples of newly eligible patients in the intervention clinics during years 1 and 2 following randomization (accrual and evaluation intervals: February 5, 2014 to February 3, 2015, and February 4, 2015 to February 3, 2016). We were unable to conduct a formal assessment of effectiveness in year 2 because the intervention was implemented in usual care clinics in August of 2015.

Across all 26 clinics, 25 888 adults were age-eligible and not up-to-date with screening at the time of randomization and 15 305 became eligible during the accrual interval. These 41 193 adults comprise our primary analytic sample. Intervention and usual care clinics were similar in size (Figure 1). The baseline (2014) CRC screening rate was 34% in both intervention (range, 13.4%-58.2%) and usual care (range, 17.5%-59.3%) clinics. Baseline rates of fecal testing in the past year were 13.0% (range, 3.7%-25.3%) in intervention clinics and 12.6% (range, 5.3%-20.7%) in usual care clinics.

Patient characteristics in each study arm are provided in Table 1. Median clinic-level proportions of adults aged 50 to 64 years were 80% in intervention clinics and 83% in usual care clinics. Participants in median intervention and usual care clinics mainly preferred speaking English (33 547 [87%]), were white (34 879 [92%]), and were enrolled in Medicaid or Medicare (15 326 [35%] and 10 754 [23%], respectively).

Table 2 presents clinic-level FIT completion proportions for both the primary data set and for the data set that accounted for implementation delays. For the primary data set, adjusted FIT completion proportions were 3.4 percentage points higher for intervention clinics (13.9%) than usual care clinics (10.4%); a statistically significant difference (95% CI, 0.1%-6.8%; P = .05). Net differences in FIT completion ranged from −7.4 percentage points to 17.6 percentage points across health centers (Figure 2).

The adjusted clinic-level proportion of participants completing any CRC screening was 3.8 percentage points higher for intervention clinics (18.3%) than for usual care clinics (14.5%) and was statistically significant (95% CI, 0.6%-7.0%; P = .02).

In analyses that accounted for implementation delays, difference in adjusted FIT completion between study arms was 4.7 percentage points (17.6% vs 12.8%) and was statistically significant (95% CI, 0.9%-8.6%; P = .02). The difference in any CRC screening between arms was slightly greater than in the primary data set (4.7 percentage points, 21.0% vs 16.3%) and was again statistically significant (95% CI, 1.1%-8.2%; P = .01).

A total of 8295 and 4794 patients in intervention clinics were newly eligible for study outreach in years 1 and 2, respectively. The FIT completion rates in these patients were 18.7% in year 1 and 19.1% in year 2, suggesting maintenance of the effect over time; similarly, proportions of participants who were mailed a FIT were 39% in year 1 and 34% in year 2.

Of eligible adults who were mailed a FIT test, the overall FIT return rate was 1304 of 6308 (21%). Rates were higher in clinics that consistently delivered a reminder letter (25%), compared with those who did so inconsistently (14%) or not at all (6%). This simple per-protocol analysis suggests an intervention effect of 15 percentage points between clinics consistently delivering reminder letters and usual care clinics (10%) (Table 2). The number needed to mail to achieve a completed FIT was 4.8 overall, and 4.0 in clinics that mailed a FIT reminder.

During the evaluation interval, 5305 individuals across study arms had EHR evidence of completing a FIT; 721 (13.6%) had a positive FIT result. Of those, 427 (59.2%) completed a colonoscopy within 12 months (based on medical chart abstraction); of which 75 (17.6%) were found to have an advanced adenoma (67 [15.7%]) or cancer (8 [1.9%]).

The proportion of eligible intervention participants who were mailed a FIT ranged from 6.5% to 68.2% across health centers (Figure 2). Of the 21 134 intervention participants in the primary data set, 6925 (33%) were mailed an introductory letter. Of these, 6308 (91%) were ordered a FIT, and 4074 (59%) were mailed a reminder letter (Table 3). Among the 15 763 intervention participants in the data set that accounted for implementation delays, 7370 (47%) were mailed an introductory letter. Of these, 6628 (90%) were ordered a FIT, and 4033 (55%) were mailed a reminder letter. Clinic-level distributions of delivered intervention activities are provided in the eTable in Supplement 2. Across health centers, levels of implementation were strongly correlated with net differences in FIT completion (primary data set correlation, 0.89; lagged data set correlation, 0.87).

Providing clinics with EHR-embedded tools and staff training to deliver mailed FIT outreach to eligible patients resulted in significantly higher rates of FIT completion (3.4 percentage points) and any CRC screening (3.8 percentage points) compared with usual care. Data that accounted for implementation delays showed somewhat larger effects. These increases in CRC screening occurred despite relatively low implementation of the program.

Although these results show promise, they represent smaller intervention effects than previous efficacy studies. Our pilot FIT outreach program resulted in a 38% boost in FIT completion rates.²² The smaller effect in this full trial may reflect the efficacy-effectiveness gap, where the ideal effect of an intervention is attenuated by conditions in the actual health care system.²³ In our pilot study, we performed a 1-time data pull to identify eligible adults and ensured that all eligible adults received the intervention. In the present pragmatic study, only one-third of eligible patients were mailed a FIT. Indeed, FIT return rates among eligible adults who were mailed a FIT were consistent with other studies, suggesting that low implementation rates account for the lower completion rates we observed.^9,10

Low implementation may have occurred because the intervention competed with other clinic priorities.²⁴ In qualitative interviews with health center leaders²⁴; we found that primary challenges were time burden on clinic staff, limited organizational capacity, EHR issues, and difficulties using Reporting Workbench (which was newly implemented for the study). Participating health centers experienced substantial leadership turnover during the study course, which likely created implementation challenges.²⁴ In addition, for most participating health centers, STOP CRC represented the first time EHR tools were used to deliver cancer screening services outside the clinic. Implementation might have increased with experience.

Another factor that may have influenced implementation rates is that our automated EHR tools identified eligible adults in real time and were updated daily. These real-time tools were preferred by clinic leaders because patients’ information, including screening status, continually changes and because they could integrate with day-to-day care and be adapted to address other preventive health care needs. However, many adults dropped off the list because they no longer had visited the clinic in the past year or for other reasons before clinic staff could begin the intervention: clinics could not reach 100% of eligible patients unless staff acted on their lists daily. Although this real-time design may make the EHR tools more sustainable, it may also limit implementation, and therefore effectiveness, within a given intervention window.

Several changes to the health care context during the study period may have influenced our results. First, the Affordable Care Act’s Medicaid expansion offered insurance for many previously uninsured individuals in both Oregon and California, many of whom were age-eligible for CRC screening. Second, in 2013, CRC screening became an incentivized metric for Oregon’s Coordinated Care Organizations (including FQHCs). These policies and other external factors may have influenced FIT completion rates in usual care clinics during the study period, potentially diluting the impact of our intervention. Indeed, screening rates in all STOP CRC clinics increased from 33.2% to 46.8% from 2013 to 2015.

Some additional limitations should be noted. Although FIT testing was our primary outcome, some health centers were unable to process completed FIT samples that were missing collection dates, leading to potential underreporting.¹⁷ In addition, our secondary outcome, any CRC screening, is subject to well-documented under-capture of colonoscopy in primary care records,²⁵ and we could not distinguish between screening and diagnostic colonoscopy (though indication was unlikely to differ by study arm). In addition, we used a proxy for having been mailed a FIT (mailed letter followed by a FIT order), potentially leading to some implementation misclassification. There are also known downstream problems that we did not evaluate (eg, low rates of repeated testing, low rates of follow-up colonoscopy). Nevertheless, our year 2 findings suggest maintenance of the intervention effect over time. Finally, because our study was pragmatic, we provided clinics with resources (eg, clinic impact fees and collaborative learning environment) that may be unavailable to clinics seeking to adopt the program.

Our study also has several notable strengths, including a large and diverse population that typically has CRC screening at low rates, and the inclusion of 8 different FQHCs, allowing for greater generalization than previous work. Because all clinics used the same EHR, variation in data quality were minimized.

This work demonstrates that mailed FIT outreach programs can have clinical impact when integrated into clinical workflows but emphasizes the need to identify additional strategies to support program implementation in low-resource health centers.

Corresponding Author: Gloria D. Coronado, PhD, Kaiser Permanente Center for Health Research, 3800 N Interstate Ave, Portland, OR 97227 (gloria.d.coronado@kpchr.org).

Accepted for Publication: June 11, 2018.

Published Online: August 6, 2018. doi:10.1001/jamainternmed.2018.3629

Correction: This article was corrected on June 3, 2019, to fix an error in Table 1.

Author Contributions: Dr Coronado had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Vollmer conducted all statistical analyses presented in this manuscript and maintains responsibility for its accuracy.

Study concept and design: Coronado, Petrik, Vollmer, Taplin, Green.

Acquisition, analysis, or interpretation of data: Coronado, Petrik, Vollmer, Keast, Fields, Green.

Drafting of the manuscript: Coronado, Petrik, Taplin, Fields, Green.

Critical revision of the manuscript for important intellectual content: Coronado, Vollmer, Keast, Green.

Statistical analysis: Coronado, Petrik, Vollmer, Keast.

Obtained funding: Coronado, Green.

Administrative, technical, or material support: Coronado, Petrik, Fields, Green.

Study supervision: Coronado, Petrik, Taplin, Green.

Conflict of Interest Disclosures: From November 2014 to August 2015, Dr Coronado served as a coinvestigator on an industry-funded study to evaluate patient adherence to an experimental blood test for colorectal cancer. The study was funded by EpiGenomics. From September 2017 to June 2019, Dr Coronado has served as the principal investigator on an industry-funded study to compare the clinical performance of an experimental FIT to an US Food and Drug Administration-approved FIT. The study is funded by Quidel Corporation. No other disclosures are reported.

Funding/Support: Research reported in this publication was supported by the National Institutes of Health grants UH2AT007782 and 4UH3CA18864002.

Role of the Funder/Sponsor: The National Institutes of Health contributed to the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The information presented does not reflect the views of the National Institutes of Health.