Effectiveness of a Mailed Colorectal Cancer Screening Outreach Program in Community Health Clinics: The STOP CRC Cluster Randomized Clinical Trial | Cancer Screening, Prevention, Control | JAMA Internal Medicine | JAMA Network
[Skip to Navigation]
Figure 1.  CONSORT Flowchart
CONSORT Flowchart
Figure 2.  Implementation and Effectiveness of the STOP CRC Intervention, by Health Center
Implementation and Effectiveness of the STOP CRC Intervention, by Health Center

FIT indicates fecal immunochemical test; HC, health center; STOP CRC, the Strategies and Opportunities to STOP Colon Cancer in Priority Populations study.

Table 1.  Baseline Clinic-Level Characteristics of 41 193 Eligible Adults in Analysis Sample
Baseline Clinic-Level Characteristics of 41 193 Eligible Adults in Analysis Sample
Table 2.  Colorectal Cancer Screening Completion, by Intervention and Usual Care Arm
Colorectal Cancer Screening Completion, by Intervention and Usual Care Arm
Table 3.  Intervention Activities Among Eligible Adults in Intervention Clinics, in Primary and Lagged Data Sets
Intervention Activities Among Eligible Adults in Intervention Clinics, in Primary and Lagged Data Sets
1.
Mandel  JS, Bond  JH, Church  TR,  et al.  Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study.  N Engl J Med. 1993;328(19):1365-1371. doi:10.1056/NEJM199305133281901PubMedGoogle ScholarCrossref
2.
Scholefield  JH, Moss  S, Sufi  F, Mangham  CM, Hardcastle  JD.  Effect of faecal occult blood screening on mortality from colorectal cancer: results from a randomised controlled trial.  Gut. 2002;50(6):840-844. doi:10.1136/gut.50.6.840PubMedGoogle ScholarCrossref
3.
Zauber  AG, Lansdorp-Vogelaar  I, Knudsen  AB, Wilschut  J, van Ballegooijen  M, Kuntz  KM.  Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force.  Ann Intern Med. 2008;149(9):659-669. doi:10.7326/0003-4819-149-9-200811040-00244PubMedGoogle ScholarCrossref
4.
Whitlock  EP, Lin  J, Liles  E,  et al.  US Preventive Services Task Force Evidence Syntheses, formerly Systematic Evidence Reviews. Screening for Colorectal Cancer: An Updated Systematic Review. Rockville (MD). US: Agency for Healthcare Research and Quality; 2008.
5.
Centers for Disease Control and Prevention (CDC).  Vital signs: colorectal cancer screening test use--United States, 2012.  MMWR Morb Mortal Wkly Rep. 2013;62(44):881-888.PubMedGoogle Scholar
6.
US Department of Health and Human Services Health Resources and Services Administration. Colorectal Cancer Screening. http://www.hrsa.gov/quality/toolbox/measures/colorectalcancer/index.html. Accessed June 9, 2016.
7.
Levin  TR.  Colorectal cancer screening: 80% by 2018. Colonoscopists simply cannot do it alone.  Gastrointest Endosc. 2016;83(3):552-554. doi:10.1016/j.gie.2015.08.023PubMedGoogle ScholarCrossref
8.
Klabunde  C, Blom  J, Bulliard  JL,  et al.  Participation rates for organized colorectal cancer screening programmes: an international comparison.  J Med Screen. 2015;22(3):119-126. doi:10.1177/0969141315584694PubMedGoogle ScholarCrossref
9.
Baker  DW, Brown  T, Buchanan  DR,  et al.  Comparative effectiveness of a multifaceted intervention to improve adherence to annual colorectal cancer screening in community health centers: a randomized clinical trial.  JAMA Intern Med. 2014;174(8):1235-1241. doi:10.1001/jamainternmed.2014.2352PubMedGoogle ScholarCrossref
10.
Coronado  GD, Golovaty  I, Longton  G, Levy  L, Jimenez  R.  Effectiveness of a clinic-based colorectal cancer screening promotion program for underserved Hispanics.  Cancer. 2011;117(8):1745-1754. doi:10.1002/cncr.25730PubMedGoogle ScholarCrossref
11.
Goldman  SN, Liss  DT, Brown  T,  et al.  Comparative effectiveness of multifaceted outreach to initiate colorectal cancer screening in community health centers: a randomized controlled trial.  J Gen Intern Med. 2015;30(8):1178-1184. doi:10.1007/s11606-015-3234-5PubMedGoogle ScholarCrossref
12.
Jean-Jacques  M, Kaleba  EO, Gatta  JL, Gracia  G, Ryan  ER, Choucair  BN.  Program to improve colorectal cancer screening in a low-income, racially diverse population: a randomized controlled trial.  Ann Fam Med. 2012;10(5):412-417. doi:10.1370/afm.1381PubMedGoogle ScholarCrossref
13.
Singal  AG, Gupta  S, Tiro  JA,  et al.  Outreach invitations for FIT and colonoscopy improve colorectal cancer screening rates: A randomized controlled trial in a safety-net health system.  Cancer. 2016;122(3):456-463. doi:10.1002/cncr.29770PubMedGoogle ScholarCrossref
14.
Levy  BT, Daly  JM, Xu  Y, Ely  JW.  Mailed fecal immunochemical tests plus educational materials to improve colon cancer screening rates in Iowa Research Network (IRENE) practices.  J Am Board Fam Med. 2012;25(1):73-82. doi:10.3122/jabfm.2012.01.110055PubMedGoogle ScholarCrossref
15.
Coronado  GD, Retecki  S, Schneider  J, Taplin  SH, Burdick  T, Green  BB.  Recruiting community health centers into pragmatic research: findings from STOP CRC.  Clin Trials. 2016;13(2):214-222. doi:10.1177/1740774515608122PubMedGoogle ScholarCrossref
16.
Green  BB, Wang  CY, Horner  K,  et al.  Systems of support to increase colorectal cancer screening and follow-up rates (SOS): design, challenges, and baseline characteristics of trial participants.  Contemp Clin Trials. 2010;31(6):589-603. doi:10.1016/j.cct.2010.07.012PubMedGoogle ScholarCrossref
17.
Coury  J, Schneider  JL, Rivelli  JS,  et al.  Applying the Plan-Do-Study-Act (PDSA) approach to a large pragmatic study involving safety net clinics.  BMC Health Serv Res. 2017;17(1):411. doi:10.1186/s12913-017-2364-3PubMedGoogle ScholarCrossref
18.
Coronado  GD, Burdick  T, Petrik  A, Kapka  T, Retecki  S, Green  B.  Using an automated data-driven, EHR-embedded program for mailing FIT kits: lessons from the STOP CRC pilot study.  J Gen Pract (Los Angel). 2014;2:2.PubMedGoogle Scholar
19.
Coronado  GD, Sanchez  J, Petrik  A, Kapka  T, DeVoe  J, Green  B.  Advantages of wordless instructions on how to complete a fecal immunochemical test: lessons from patient advisory council members of a federally qualified health center.  J Cancer Educ. 2014;29(1):86-90. doi:10.1007/s13187-013-0551-4PubMedGoogle ScholarCrossref
20.
Moule  P, Evans  D, Pollard  K.  Using the Plan-Do-Study-Act model: pacesetters experiences.  Int J Health Care Qual Assur. 2013;26(7):593-600. doi:10.1108/IJHCQA-09-2011-0053PubMedGoogle ScholarCrossref
21.
Coronado  GD, Vollmer  WM, Petrik  A,  et al.  Strategies and opportunities to STOP colon cancer in priority populations: design of a cluster-randomized pragmatic trial.  Contemp Clin Trials. 2014;38(2):344-349. doi:10.1016/j.cct.2014.06.006PubMedGoogle ScholarCrossref
22.
Coronado  GD, Vollmer  WM, Petrik  A,  et al.  Strategies and opportunities to STOP colon cancer in priority populations: pragmatic pilot study design and outcomes.  BMC Cancer. 2014;14:55. doi:10.1186/1471-2407-14-55PubMedGoogle ScholarCrossref
23.
Nordon  C, Karcher  H, Groenwold  RH,  et al; GetReal consortium.  The “efficacy-effectiveness gap”: historical background and current conceptualization.  Value Health. 2016;19(1):75-81. doi:10.1016/j.jval.2015.09.2938PubMedGoogle ScholarCrossref
24.
Coronado  GD, Schneider  JL, Petrik  A, Rivelli  J, Taplin  S, Green  BB.  Implementation successes and challenges in participating in a pragmatic study to improve colon cancer screening: perspectives of health center leaders.  Transl Behav Med. 2017;7(3):557-566. doi:10.1007/s13142-016-0461-1PubMedGoogle ScholarCrossref
25.
Petrik  AF, Green  BB, Vollmer  WM,  et al.  The validation of electronic health records in accurately identifying patients eligible for colorectal cancer screening in safety net clinics.  Fam Pract. 2016;33(6):639-643. doi:10.1093/fampra/cmw065PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Patient relevant outcomes - missing?
    Nir Tsabar, MD/DSc | Clalit Health Services, North District, Israel
    Large scale interventions (such as screening-programs) should be evaluated not only for efficiency but also for outcomes such as overall survival.
    CONFLICT OF INTEREST: None Reported
    Original Investigation
    August 6, 2018

    Effectiveness of a Mailed Colorectal Cancer Screening Outreach Program in Community Health Clinics: The STOP CRC Cluster Randomized Clinical Trial

    Author Affiliations
    • 1Kaiser Permanente Center for Health Research, Portland, Oregon
    • 2Center for Global Health, National Cancer Institute, Rockville, Maryland
    • 3OCHIN, Portland, Oregon
    • 4Kaiser Permanente Washington Health Research Institute, Seattle, Washington
    JAMA Intern Med. 2018;178(9):1174-1181. doi:10.1001/jamainternmed.2018.3629
    Key Points

    Question  How effective is a mailed fecal immunochemical test (FIT) outreach program when implemented in busy community clinic practices as part of standard care?

    Findings  A mailed FIT outreach program resulted in a 3.4 percentage point increase in FIT completion in clinics that received the intervention, compared with clinics that practiced usual care. Low rates of implementation were common and were associated with low levels of effectiveness.

    Meaning  Mailed FIT outreach can raise rates of colorectal cancer screening; additional strategies may be needed to support program implementation in low-resource health centers.

    Abstract

    Importance  Approximately 24 million US individuals receive care at federally qualified health centers, which historically have low rates of colorectal cancer screening. The US Preventive Services Task Force recommends routine colorectal cancer screening for individuals aged 50 to 75 years.

    Objective  To determine the effectiveness of an electronic health record (EHR)-embedded mailed fecal immunochemical test (FIT) outreach program implemented in health centers as part of standard care.

    Design, Setting, and Participants  This cluster randomized pragmatic clinical trial was conducted in 26 federally qualified health center clinics, representing 8 health centers in Oregon and California, randomized to intervention (n = 13) or usual care (n = 13). All participants were overdue for colorectal cancer screening during the accrual interval (February 4, 2014 to February 3, 2015).

    Interventions  Electronic health record-embedded tools to identify eligible adults and to facilitate implementation of a stepwise mailed intervention involving (1) an introductory letter, (2) a mailed FIT, and (3) a reminder letter; training, collaborative learning, and facilitation through a practice improvement process.

    Main Outcomes and Measures  Effectiveness was measured as clinic-level proportions of adults who completed a FIT, and secondarily, any colorectal cancer screening within 12 months of accrual or by August 3, 2015. Implementation was measured as clinic-level proportions of adults who were mailed an introductory letter and ordered a FIT.

    Results  Twenty-six clinics with 41 193 adults (mean [SD] age, 58.5 [6.3] years; 22 994 women) were randomized to receive the direct mail colorectal screening intervention (13 clinics; 21 134 patients) or usual care (13 clinics; 20 059 patients). Compared with usual care clinics, intervention clinics had significantly higher adjusted clinic-level proportion of participants who completed a FIT (13.9% vs 10.4%; difference, 3.4 percentage points; 95% CI, 0.1%-6.8%) and any colorectal cancer screening (18.3% vs 14.5%; difference, 3.8 percentage points; 95% CI, 0.6%-7.0%). We observed large variation across health centers in effectiveness (FIT completion differences range, −7.4 percentage points to 17.6 percentage points) and implementation (proportion who were mailed a FIT range, 6.5% to 68.2%). The number needed to mail to achieve a completed FIT was 4.8 overall, and 4.0 in clinics that mailed a FIT reminder.

    Conclusions and Relevance  An EHR-embedded mailed FIT outreach intervention significantly improved rates of FIT completion and rates of any colorectal cancer screening. Higher rates of colorectal cancer screening occurred in clinics that successfully implemented the mailed outreach program.

    Trial Registration  ClinicalTrials.gov identifier: NCT01742065

    Introduction

    For more than 20 years, we have known that colorectal cancer (CRC) screening reduces cancer incidence and mortality.1-3 The US Preventive Services Task Force has recommended CRC screening for adults ages 50 to 75 years since 2008.4 Nonetheless, screening rates remain low, particularly among underserved groups, including those served by federally qualified health centers (FQHCs).5 In FQHCs nationally, only 40% of eligible adults were up-to-date for CRC screening in 2016.6 Research is needed on how to realize the full potential of routine screening to detect and prevent CRC.

    Studies have shown that mailing fecal immunochemical test (FIT) kits to individuals who are due for CRC screening can improve screening rates,7,8 with increases in screening uptake of 22% to 45% in FQHC patient populations.9-13 However, these studies have been small in scope or designed to determine efficacy under ideal circumstances (eg, interventions delivered by researchers rather than clinic staff).13,14 Little is known about the large-scale effectiveness of mailed FIT outreach interventions in FQHC settings.

    We conducted the Strategies and Opportunities to STOP Colon Cancer in Priority Populations (STOP CRC) study to evaluate the effectiveness of a mailed FIT intervention delivered by clinic staff at FQHCs. This pragmatic study provided 13 clinics with electronic health record (EHR) tools to identify and contact patients who were due for screening; trained clinic staff to use the tools; and compared results with 13 clinics practicing usual care. This paper presents primary outcomes from the study: we assess the effectiveness of a large-scale mailed FIT outreach approach to CRC screening and evaluate levels of implementation across a range of FQHC clinics.

    Methods
    Health Center Recruitment

    Our clinic recruitment methods have been reported previously (Figure 1).15 Briefly, we included 7 FQHCs (representing 24 clinics) and 1 FQHC (representing 2 clinics) affiliated with an academic medical center. All clinics operated in Oregon and California and served similar low-income populations. All health centers were affiliated with OCHIN (formerly Oregon Community Health Information Network) and used the same Epic EHR (Vernona, WI). Health center eligibility criteria included willingness to randomize clinics and use a single fecal test across all participating clinics; having an electronic interface with the laboratory that processed the kits; and having sufficient capacity for follow-up colonoscopy, among other factors.15 The study was approved by the Institutional Review Board of Kaiser Permanente Northwest (Protocol # 4364), with ceding agreements from Kaiser Permanente Washington Research Institute and OCHIN. We obtained a waiver of informed consent because the study was embedded into standard care and the risk to patients was considered minimal. The trial protocol is available in Supplement 1.

    Clinic Randomization

    The project statistician stratified allocation assignments by health center to balance clinic-level characteristics between study arms. Participating clinics were randomized to either usual care (n = 13) or an EHR-embedded intervention (n = 13) on February 4, 2014. Neither researchers nor clinic staff were blinded to randomization assignment. The intervention was delivered by clinic staff; eligible adults were unaware of their study participation.

    Patient Eligibility

    Eligible adults were aged 50 to 74 years, had a clinic visit during the 12 months prior to accrual, and were due for CRC screening based on the following established criteria16: (1) no EHR evidence of completing a fecal test during the past 11 months, flexible sigmoidoscopy during the past 4 years, or colonoscopy during the past 9 years; (2) no evidence of an order for a fecal test in the past 6 months or a referral for a sigmoidoscopy or colonoscopy in the past year. We excluded adults with evidence of relevant health conditions (eg, colorectal cancer, colon disease, end-stage renal failure). Patients were identified using real-time EHR tools that were updated daily; patients were accrued into the analytic data set if they appeared on the list even for a single day. This process led to the identification of 21 134 individuals in intervention clinics and 20 059 individuals in usual care clinics over the accrual interval.

    Treatment Arms
    Usual Care Arm

    Usual care clinics continued their standard processes for CRC screening, which typically consisted of providing information and ordering tests during routine clinical encounters. These clinics were offered training and intervention materials at the end of the follow-up period.

    Intervention Arm

    The intervention consisted of customized EHR-embedded tools, training in the use of these tools, a collaborative environment for continued learning after initial training, and a standardized practice improvement process.17 The design of the STOP CRC EHR tools has been described previously.18 Briefly, the research team collaborated with OCHIN to customize EHR tools using Reporting Workbench in Epic. The tools identified adults who were due for CRC screening and allowed clinic staff to generate mailing lists and materials (eg, letters and labels) for 3 sequential mailings: (1) an introductory letter; (2) a FIT kit packet that included wordless instructions on how to complete the test19; and (3) a reminder letter. The tools automatically removed patients from lists if they were found to have an invalid address, reported prior CRC screening, or were excluded by a clinic.

    The EHR tools were provided to staff at each intervention clinic during the initial training, along with templates for introductory letters, FIT kit inserts, and reminder letters. We encouraged clinics to customize the templates as needed. Clinic staff were responsible for integrating the intervention into their care processes. As part of the collaborative learning model, EHR site specialists from all health centers were invited to monthly meetings to discuss implementation of the intervention, and health center leadership were invited to quarterly meetings with an advisory board. In addition, we used a well-validated practice improvement approach that included Plan-Do-Study-Act (PDSA) cycles and provided a trained facilitator who offered PDSA booster training at each center.20 The PDSA cycles were conducted at each health center 4 to 6 months after initial training. Each health center selected its own target for process improvement (eg, improving workflows, increasing return rates).17

    To activate the EHR tools, clinic staff needed to link the EHR tools to a printer for printing labels; assure laboratory interfaces were in place, import customized introductory letters, and assign appropriate codes for laboratory orders, among other steps. Staff then used patient lists to carry out each step of the intervention. Initial eligibility lists facilitated batch-printing, stuffing, and mailing of introductory letters. Lists of patients eligible for a mailed FIT facilitated placement of FIT laboratory orders and labeling and mailing of FITs. Lists of patients due for reminders were used to batch-print and mail reminder letters. Clinic staff could choose the mailing schedule (monthly, quarterly, 1-time).

    Study Outcomes

    The primary outcome was clinic-level proportion of eligible adults (accrued between February 4, 2014 and February 3, 2015) who completed FIT within 12 months or through August 3, 2015 (when usual care clinics received access to study tools). The evaluation interval was February 4, 2014 to August 3, 2015. A secondary outcome was clinic-level proportion of eligible adults completing any CRC screening (fecal test, sigmoidoscopy, or colonoscopy) during the evaluation interval. We calculated implementation as the clinic-level proportion of participants who were mailed an introductory letter and were subsequently ordered a FIT (orders were placed at the time of FIT mailing) during the evaluation interval, based on EHR data. This allowed us to distinguish between mailed FITs and those distributed in-clinic.

    Statistical Methods
    Primary Analyses

    To assess effectiveness, we fit generalized estimating equations (GEE) with a logistic link to model patient-level data. We weighted patient data so that each clinic’s data had an equal weight. Models were adjusted for age, sex, and health center. They used robust variance estimators and independent correlation structures. We specified clinic as a clustering variable to account for intraclinic correlation; the intraclass correlation coefficient was 0.05 after covariable adjustment. We report effectiveness as the absolute difference between intervention and usual care clinics in adjusted probabilities calculated using mean values for all covariates; reported P values are based on the corresponding adjusted odds ratio and account for reduced degrees of freedom owing to clustering.

    We used this analytic approach both for our primary outcome (FIT completion) and our secondary outcome (any CRC screening). All analyses were 2-tailed and considered significant if P ≤ .05. The analyses were conducted from May 10, 2016 to June 5, 2018, using SAS (version 9.4, SAS Institute) and STATA statistical software (version 15.1, StataCorp). Sensitivity analysis included GEE models without weighting (in this case with an exchangeable correlation structure); GEE models that also adjusted for insurance status; GEE models that did not adjust for patient-level covariates; unweighted random effects models; and models that excluded the 2 academic medical center clinics (these clinics did not have shared governance, and the intervention clinic, but not the usual care clinic, experienced leadership turnover). Sample size calculations have been reported previously.21

    Analyses That Accounted for Implementation Delays

    Although our planned analyses included all individuals accrued after EHR tools were provided to clinics on February 4, 2014, no clinic began printing letters until at least June of 2014; some did not begin until spring of 2015. To account for this implementation delay, we repeated all analyses using a data set that only included individuals accrued between June 4, 2014 and February 3, 2015 (8-month interval). As with the primary data set, outcomes in this secondary analysis were assessed through August 3, 2015.

    Clinic-Level Maintenance

    We conducted a descriptive analysis comparing FIT completion rates and proportions of patients who were mailed a FIT in cross-sectional samples of newly eligible patients in the intervention clinics during years 1 and 2 following randomization (accrual and evaluation intervals: February 5, 2014 to February 3, 2015, and February 4, 2015 to February 3, 2016). We were unable to conduct a formal assessment of effectiveness in year 2 because the intervention was implemented in usual care clinics in August of 2015.

    Results
    Clinic Characteristics

    Across all 26 clinics, 25 888 adults were age-eligible and not up-to-date with screening at the time of randomization and 15 305 became eligible during the accrual interval. These 41 193 adults comprise our primary analytic sample. Intervention and usual care clinics were similar in size (Figure 1). The baseline (2014) CRC screening rate was 34% in both intervention (range, 13.4%-58.2%) and usual care (range, 17.5%-59.3%) clinics. Baseline rates of fecal testing in the past year were 13.0% (range, 3.7%-25.3%) in intervention clinics and 12.6% (range, 5.3%-20.7%) in usual care clinics.

    Patient Characteristics

    Patient characteristics in each study arm are provided in Table 1. Median clinic-level proportions of adults aged 50 to 64 years were 80% in intervention clinics and 83% in usual care clinics. Participants in median intervention and usual care clinics mainly preferred speaking English (33 547 [87%]), were white (34 879 [92%]), and were enrolled in Medicaid or Medicare (15 326 [35%] and 10 754 [23%], respectively).

    Primary Outcome

    Table 2 presents clinic-level FIT completion proportions for both the primary data set and for the data set that accounted for implementation delays. For the primary data set, adjusted FIT completion proportions were 3.4 percentage points higher for intervention clinics (13.9%) than usual care clinics (10.4%); a statistically significant difference (95% CI, 0.1%-6.8%; P = .05). Net differences in FIT completion ranged from −7.4 percentage points to 17.6 percentage points across health centers (Figure 2).

    Secondary Outcomes

    The adjusted clinic-level proportion of participants completing any CRC screening was 3.8 percentage points higher for intervention clinics (18.3%) than for usual care clinics (14.5%) and was statistically significant (95% CI, 0.6%-7.0%; P = .02).

    In analyses that accounted for implementation delays, difference in adjusted FIT completion between study arms was 4.7 percentage points (17.6% vs 12.8%) and was statistically significant (95% CI, 0.9%-8.6%; P = .02). The difference in any CRC screening between arms was slightly greater than in the primary data set (4.7 percentage points, 21.0% vs 16.3%) and was again statistically significant (95% CI, 1.1%-8.2%; P = .01).

    A total of 8295 and 4794 patients in intervention clinics were newly eligible for study outreach in years 1 and 2, respectively. The FIT completion rates in these patients were 18.7% in year 1 and 19.1% in year 2, suggesting maintenance of the effect over time; similarly, proportions of participants who were mailed a FIT were 39% in year 1 and 34% in year 2.

    Per-Protocol Analyses

    Of eligible adults who were mailed a FIT test, the overall FIT return rate was 1304 of 6308 (21%). Rates were higher in clinics that consistently delivered a reminder letter (25%), compared with those who did so inconsistently (14%) or not at all (6%). This simple per-protocol analysis suggests an intervention effect of 15 percentage points between clinics consistently delivering reminder letters and usual care clinics (10%) (Table 2). The number needed to mail to achieve a completed FIT was 4.8 overall, and 4.0 in clinics that mailed a FIT reminder.

    During the evaluation interval, 5305 individuals across study arms had EHR evidence of completing a FIT; 721 (13.6%) had a positive FIT result. Of those, 427 (59.2%) completed a colonoscopy within 12 months (based on medical chart abstraction); of which 75 (17.6%) were found to have an advanced adenoma (67 [15.7%]) or cancer (8 [1.9%]).

    Implementation

    The proportion of eligible intervention participants who were mailed a FIT ranged from 6.5% to 68.2% across health centers (Figure 2). Of the 21 134 intervention participants in the primary data set, 6925 (33%) were mailed an introductory letter. Of these, 6308 (91%) were ordered a FIT, and 4074 (59%) were mailed a reminder letter (Table 3). Among the 15 763 intervention participants in the data set that accounted for implementation delays, 7370 (47%) were mailed an introductory letter. Of these, 6628 (90%) were ordered a FIT, and 4033 (55%) were mailed a reminder letter. Clinic-level distributions of delivered intervention activities are provided in the eTable in Supplement 2. Across health centers, levels of implementation were strongly correlated with net differences in FIT completion (primary data set correlation, 0.89; lagged data set correlation, 0.87).

    Discussion

    Providing clinics with EHR-embedded tools and staff training to deliver mailed FIT outreach to eligible patients resulted in significantly higher rates of FIT completion (3.4 percentage points) and any CRC screening (3.8 percentage points) compared with usual care. Data that accounted for implementation delays showed somewhat larger effects. These increases in CRC screening occurred despite relatively low implementation of the program.

    Although these results show promise, they represent smaller intervention effects than previous efficacy studies. Our pilot FIT outreach program resulted in a 38% boost in FIT completion rates.22 The smaller effect in this full trial may reflect the efficacy-effectiveness gap, where the ideal effect of an intervention is attenuated by conditions in the actual health care system.23 In our pilot study, we performed a 1-time data pull to identify eligible adults and ensured that all eligible adults received the intervention. In the present pragmatic study, only one-third of eligible patients were mailed a FIT. Indeed, FIT return rates among eligible adults who were mailed a FIT were consistent with other studies, suggesting that low implementation rates account for the lower completion rates we observed.9,10

    Low implementation may have occurred because the intervention competed with other clinic priorities.24 In qualitative interviews with health center leaders24; we found that primary challenges were time burden on clinic staff, limited organizational capacity, EHR issues, and difficulties using Reporting Workbench (which was newly implemented for the study). Participating health centers experienced substantial leadership turnover during the study course, which likely created implementation challenges.24 In addition, for most participating health centers, STOP CRC represented the first time EHR tools were used to deliver cancer screening services outside the clinic. Implementation might have increased with experience.

    Another factor that may have influenced implementation rates is that our automated EHR tools identified eligible adults in real time and were updated daily. These real-time tools were preferred by clinic leaders because patients’ information, including screening status, continually changes and because they could integrate with day-to-day care and be adapted to address other preventive health care needs. However, many adults dropped off the list because they no longer had visited the clinic in the past year or for other reasons before clinic staff could begin the intervention: clinics could not reach 100% of eligible patients unless staff acted on their lists daily. Although this real-time design may make the EHR tools more sustainable, it may also limit implementation, and therefore effectiveness, within a given intervention window.

    Several changes to the health care context during the study period may have influenced our results. First, the Affordable Care Act’s Medicaid expansion offered insurance for many previously uninsured individuals in both Oregon and California, many of whom were age-eligible for CRC screening. Second, in 2013, CRC screening became an incentivized metric for Oregon’s Coordinated Care Organizations (including FQHCs). These policies and other external factors may have influenced FIT completion rates in usual care clinics during the study period, potentially diluting the impact of our intervention. Indeed, screening rates in all STOP CRC clinics increased from 33.2% to 46.8% from 2013 to 2015.

    Strengths and Limitations

    Some additional limitations should be noted. Although FIT testing was our primary outcome, some health centers were unable to process completed FIT samples that were missing collection dates, leading to potential underreporting.17 In addition, our secondary outcome, any CRC screening, is subject to well-documented under-capture of colonoscopy in primary care records,25 and we could not distinguish between screening and diagnostic colonoscopy (though indication was unlikely to differ by study arm). In addition, we used a proxy for having been mailed a FIT (mailed letter followed by a FIT order), potentially leading to some implementation misclassification. There are also known downstream problems that we did not evaluate (eg, low rates of repeated testing, low rates of follow-up colonoscopy). Nevertheless, our year 2 findings suggest maintenance of the intervention effect over time. Finally, because our study was pragmatic, we provided clinics with resources (eg, clinic impact fees and collaborative learning environment) that may be unavailable to clinics seeking to adopt the program.

    Our study also has several notable strengths, including a large and diverse population that typically has CRC screening at low rates, and the inclusion of 8 different FQHCs, allowing for greater generalization than previous work. Because all clinics used the same EHR, variation in data quality were minimized.

    Conclusions

    This work demonstrates that mailed FIT outreach programs can have clinical impact when integrated into clinical workflows but emphasizes the need to identify additional strategies to support program implementation in low-resource health centers.

    Back to top
    Article Information

    Corresponding Author: Gloria D. Coronado, PhD, Kaiser Permanente Center for Health Research, 3800 N Interstate Ave, Portland, OR 97227 (gloria.d.coronado@kpchr.org).

    Accepted for Publication: June 11, 2018.

    Published Online: August 6, 2018. doi:10.1001/jamainternmed.2018.3629

    Correction: This article was corrected on June 3, 2019, to fix an error in Table 1.

    Author Contributions: Dr Coronado had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Vollmer conducted all statistical analyses presented in this manuscript and maintains responsibility for its accuracy.

    Study concept and design: Coronado, Petrik, Vollmer, Taplin, Green.

    Acquisition, analysis, or interpretation of data: Coronado, Petrik, Vollmer, Keast, Fields, Green.

    Drafting of the manuscript: Coronado, Petrik, Taplin, Fields, Green.

    Critical revision of the manuscript for important intellectual content: Coronado, Vollmer, Keast, Green.

    Statistical analysis: Coronado, Petrik, Vollmer, Keast.

    Obtained funding: Coronado, Green.

    Administrative, technical, or material support: Coronado, Petrik, Fields, Green.

    Study supervision: Coronado, Petrik, Taplin, Green.

    Conflict of Interest Disclosures: From November 2014 to August 2015, Dr Coronado served as a coinvestigator on an industry-funded study to evaluate patient adherence to an experimental blood test for colorectal cancer. The study was funded by EpiGenomics. From September 2017 to June 2019, Dr Coronado has served as the principal investigator on an industry-funded study to compare the clinical performance of an experimental FIT to an US Food and Drug Administration-approved FIT. The study is funded by Quidel Corporation. No other disclosures are reported.

    Funding/Support: Research reported in this publication was supported by the National Institutes of Health grants UH2AT007782 and 4UH3CA18864002.

    Role of the Funder/Sponsor: The National Institutes of Health contributed to the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The information presented does not reflect the views of the National Institutes of Health.

    References
    1.
    Mandel  JS, Bond  JH, Church  TR,  et al.  Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study.  N Engl J Med. 1993;328(19):1365-1371. doi:10.1056/NEJM199305133281901PubMedGoogle ScholarCrossref
    2.
    Scholefield  JH, Moss  S, Sufi  F, Mangham  CM, Hardcastle  JD.  Effect of faecal occult blood screening on mortality from colorectal cancer: results from a randomised controlled trial.  Gut. 2002;50(6):840-844. doi:10.1136/gut.50.6.840PubMedGoogle ScholarCrossref
    3.
    Zauber  AG, Lansdorp-Vogelaar  I, Knudsen  AB, Wilschut  J, van Ballegooijen  M, Kuntz  KM.  Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force.  Ann Intern Med. 2008;149(9):659-669. doi:10.7326/0003-4819-149-9-200811040-00244PubMedGoogle ScholarCrossref
    4.
    Whitlock  EP, Lin  J, Liles  E,  et al.  US Preventive Services Task Force Evidence Syntheses, formerly Systematic Evidence Reviews. Screening for Colorectal Cancer: An Updated Systematic Review. Rockville (MD). US: Agency for Healthcare Research and Quality; 2008.
    5.
    Centers for Disease Control and Prevention (CDC).  Vital signs: colorectal cancer screening test use--United States, 2012.  MMWR Morb Mortal Wkly Rep. 2013;62(44):881-888.PubMedGoogle Scholar
    6.
    US Department of Health and Human Services Health Resources and Services Administration. Colorectal Cancer Screening. http://www.hrsa.gov/quality/toolbox/measures/colorectalcancer/index.html. Accessed June 9, 2016.
    7.
    Levin  TR.  Colorectal cancer screening: 80% by 2018. Colonoscopists simply cannot do it alone.  Gastrointest Endosc. 2016;83(3):552-554. doi:10.1016/j.gie.2015.08.023PubMedGoogle ScholarCrossref
    8.
    Klabunde  C, Blom  J, Bulliard  JL,  et al.  Participation rates for organized colorectal cancer screening programmes: an international comparison.  J Med Screen. 2015;22(3):119-126. doi:10.1177/0969141315584694PubMedGoogle ScholarCrossref
    9.
    Baker  DW, Brown  T, Buchanan  DR,  et al.  Comparative effectiveness of a multifaceted intervention to improve adherence to annual colorectal cancer screening in community health centers: a randomized clinical trial.  JAMA Intern Med. 2014;174(8):1235-1241. doi:10.1001/jamainternmed.2014.2352PubMedGoogle ScholarCrossref
    10.
    Coronado  GD, Golovaty  I, Longton  G, Levy  L, Jimenez  R.  Effectiveness of a clinic-based colorectal cancer screening promotion program for underserved Hispanics.  Cancer. 2011;117(8):1745-1754. doi:10.1002/cncr.25730PubMedGoogle ScholarCrossref
    11.
    Goldman  SN, Liss  DT, Brown  T,  et al.  Comparative effectiveness of multifaceted outreach to initiate colorectal cancer screening in community health centers: a randomized controlled trial.  J Gen Intern Med. 2015;30(8):1178-1184. doi:10.1007/s11606-015-3234-5PubMedGoogle ScholarCrossref
    12.
    Jean-Jacques  M, Kaleba  EO, Gatta  JL, Gracia  G, Ryan  ER, Choucair  BN.  Program to improve colorectal cancer screening in a low-income, racially diverse population: a randomized controlled trial.  Ann Fam Med. 2012;10(5):412-417. doi:10.1370/afm.1381PubMedGoogle ScholarCrossref
    13.
    Singal  AG, Gupta  S, Tiro  JA,  et al.  Outreach invitations for FIT and colonoscopy improve colorectal cancer screening rates: A randomized controlled trial in a safety-net health system.  Cancer. 2016;122(3):456-463. doi:10.1002/cncr.29770PubMedGoogle ScholarCrossref
    14.
    Levy  BT, Daly  JM, Xu  Y, Ely  JW.  Mailed fecal immunochemical tests plus educational materials to improve colon cancer screening rates in Iowa Research Network (IRENE) practices.  J Am Board Fam Med. 2012;25(1):73-82. doi:10.3122/jabfm.2012.01.110055PubMedGoogle ScholarCrossref
    15.
    Coronado  GD, Retecki  S, Schneider  J, Taplin  SH, Burdick  T, Green  BB.  Recruiting community health centers into pragmatic research: findings from STOP CRC.  Clin Trials. 2016;13(2):214-222. doi:10.1177/1740774515608122PubMedGoogle ScholarCrossref
    16.
    Green  BB, Wang  CY, Horner  K,  et al.  Systems of support to increase colorectal cancer screening and follow-up rates (SOS): design, challenges, and baseline characteristics of trial participants.  Contemp Clin Trials. 2010;31(6):589-603. doi:10.1016/j.cct.2010.07.012PubMedGoogle ScholarCrossref
    17.
    Coury  J, Schneider  JL, Rivelli  JS,  et al.  Applying the Plan-Do-Study-Act (PDSA) approach to a large pragmatic study involving safety net clinics.  BMC Health Serv Res. 2017;17(1):411. doi:10.1186/s12913-017-2364-3PubMedGoogle ScholarCrossref
    18.
    Coronado  GD, Burdick  T, Petrik  A, Kapka  T, Retecki  S, Green  B.  Using an automated data-driven, EHR-embedded program for mailing FIT kits: lessons from the STOP CRC pilot study.  J Gen Pract (Los Angel). 2014;2:2.PubMedGoogle Scholar
    19.
    Coronado  GD, Sanchez  J, Petrik  A, Kapka  T, DeVoe  J, Green  B.  Advantages of wordless instructions on how to complete a fecal immunochemical test: lessons from patient advisory council members of a federally qualified health center.  J Cancer Educ. 2014;29(1):86-90. doi:10.1007/s13187-013-0551-4PubMedGoogle ScholarCrossref
    20.
    Moule  P, Evans  D, Pollard  K.  Using the Plan-Do-Study-Act model: pacesetters experiences.  Int J Health Care Qual Assur. 2013;26(7):593-600. doi:10.1108/IJHCQA-09-2011-0053PubMedGoogle ScholarCrossref
    21.
    Coronado  GD, Vollmer  WM, Petrik  A,  et al.  Strategies and opportunities to STOP colon cancer in priority populations: design of a cluster-randomized pragmatic trial.  Contemp Clin Trials. 2014;38(2):344-349. doi:10.1016/j.cct.2014.06.006PubMedGoogle ScholarCrossref
    22.
    Coronado  GD, Vollmer  WM, Petrik  A,  et al.  Strategies and opportunities to STOP colon cancer in priority populations: pragmatic pilot study design and outcomes.  BMC Cancer. 2014;14:55. doi:10.1186/1471-2407-14-55PubMedGoogle ScholarCrossref
    23.
    Nordon  C, Karcher  H, Groenwold  RH,  et al; GetReal consortium.  The “efficacy-effectiveness gap”: historical background and current conceptualization.  Value Health. 2016;19(1):75-81. doi:10.1016/j.jval.2015.09.2938PubMedGoogle ScholarCrossref
    24.
    Coronado  GD, Schneider  JL, Petrik  A, Rivelli  J, Taplin  S, Green  BB.  Implementation successes and challenges in participating in a pragmatic study to improve colon cancer screening: perspectives of health center leaders.  Transl Behav Med. 2017;7(3):557-566. doi:10.1007/s13142-016-0461-1PubMedGoogle ScholarCrossref
    25.
    Petrik  AF, Green  BB, Vollmer  WM,  et al.  The validation of electronic health records in accurately identifying patients eligible for colorectal cancer screening in safety net clinics.  Fam Pract. 2016;33(6):639-643. doi:10.1093/fampra/cmw065PubMedGoogle ScholarCrossref
    ×