Effect of Targeted Messaging on Return to In-Person Visits During the COVID-19 Pandemic: A Randomized Clinical Trial | Public Health | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure.  Incidence of Outcomes by Communication Type
Incidence of Outcomes by Communication Type

For each outcome, the control group (no letter) was compared with all of the intervention groups combined, with tailored and standard letters groups, and with mailed and electronic delivery methods. Comparisons by χ2 analysis were performed first. When P was <.05 by χ2 analysis, pairwise comparisons were performed.

aP < .05 for pairwise comparison with the no letter group, adjusted via the Holm method.

bP < .05 for pairwise comparison within the letter group, adjusted via the Holm method.

Table.  Incidence of Scheduled Visits by Communication Type and Patient Characteristics
Incidence of Scheduled Visits by Communication Type and Patient Characteristics
1.
Czeisler  MÉ, Marynak  K, Clarke  KEN,  et al.  Delay or avoidance of medical care because of COVID-19–related concerns—United States, June 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(36):1250-1257. doi:10.15585/mmwr.mm6936a4PubMedGoogle ScholarCrossref
2.
Findling  MG, Blendon  RJ, Benson  JM. Delayed care with harmful health consequences—reported experiences from national surveys during coronavirus disease 2019. JAMA Health Forum. Published online December 14, 2020.
3.
Jeffery  MM, D’Onofrio  G, Paek  H,  et al.  Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 pandemic in the US.   JAMA Intern Med. 2020;180(10):1328-1333. doi:10.1001/jamainternmed.2020.3288PubMedGoogle ScholarCrossref
4.
ReviveHealth. COVID-19 Consumer Survey Update, April 2020. Accessed September 20, 2020. https://www.thinkrevivehealth.com/covid-19/survey-findings-part-3
5.
Fishbein  M, Cappella  JN.  The role of theory in developing effective health communications.   J Commun. 2006 Aug 1;56:S1–17. doi:10.1111/j.1460-2466.2006.00280.xGoogle ScholarCrossref
6.
Lee  SJ, Brennan  E, Gibson  LA,  et al.  Predictive validity of an empirical approach for selecting promising message topics: a randomized-controlled study.   J Commun. 2016;66(3):433-453. doi:10.1111/jcom.12227PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 1,985
    Citations 0
    Research Letter
    Public Health
    June 30, 2021

    Effect of Targeted Messaging on Return to In-Person Visits During the COVID-19 Pandemic: A Randomized Clinical Trial

    Author Affiliations
    • 1Division of Endocrinology, Diabetes, and Metabolism, University of Pennsylvania Perelman School of Medicine, Philadelphia
    • 2Data Analytics Center, University of Pennsylvania Health System, Philadelphia
    • 3Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia
    • 4Information Services, University of Pennsylvania Health System, Philadelphia
    • 5Marketing, University of Pennsylvania Health System, Philadelphia
    • 6Annenberg School for Communication, University of Pennsylvania, Philadelphia
    JAMA Netw Open. 2021;4(6):e2115211. doi:10.1001/jamanetworkopen.2021.15211
    Introduction

    During the COVID-19 pandemic, many patients have delayed care.1-4 We performed a randomized clinical trial to determine whether targeted messaging could improve the return to in-person visits.

    Methods

    A randomized, double-blind, clinical trial (1:9 no letter vs letter; 1:1 tailored vs standard letter, 1:1 electronic vs mail) was conducted at 3 hospitals in the University of Pennsylvania Health System (eFigure and eAppendix in the Supplement 1). The University of Pennsylvania institutional review board deemed the study exempt and waived the need for consent because they found minimal risk to patient privacy. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

    We identified adult patients aged 18 years or older who had canceled in-person appointments, procedures, or surgical procedures from March 9 through June 7, 2020, and had not rescheduled with clinicians in the Heart and Vascular Service Line and Penn Orthopaedics. Patients were randomized to groups that received no letter, a tailored letter, or a standard letter inviting them to reschedule canceled appointments. The tailored letter used an inoculation strategy in which patients’ perceived threats to safety were acknowledged and addressed early in the message,5,6 with age specificity for patients 65 years or older and informational and word count equivalence between letters. The protocol and letter copies are included in Supplement 2. Messages were sent to patients on June 12 through June 17, 2020, with a final follow-up on July 17, 2020.

    Patients self-reported their race based on the following predefined categories: White, Black, Asian, Pacific Islander, American Indian, or other. Only data from White patients and Black patients were included because of the small number of patients who identified with the other categories listed for race. Race was included in the study because prior data suggested that there were differences in access to health care between Black individuals and White individuals during the COVID-19 pandemic.

    The primary outcome was the percentage of patients who had an in-person visit, procedure, or surgery with a clinician in the service line within 1 month of sending the letter. Secondary outcomes included the percentage of patients who had a telemedicine visit within 1 month or who scheduled a visit (in-person or telemedicine) within 1 month for any future date.

    We estimated a 90% power to detect a 2% difference between letter types and a 4% difference between delivery methods. The control group was compared with all groups combined, with letter types, and with delivery methods for each outcome. A χ2 analysis was performed, with pairwise comparisons adjusted by the Holm method. The percentage of patients who scheduled a future visit were compared by letter type and by delivery type overall and stratified by age, sex, race, median household income, insurance type, and service line, with testing for interactions. Analyses were conducted using R, version 3.63 (R Project for Statistical Computing). P values were 2-sided, and statistical significance set at P < .05.

    Results

    Of 38 493 patients with cancellations, 27 373 rescheduled and 11 120 (28.9%) were randomized (Figure). The median age was 59.7 (range, 18-100) years, 6074 (55%) were women, and 6830 (61%) were White patients.

    In-person return visits within 1 month were low and not significantly different between patients who did and did not receive a letter (503 [5.0%] vs 45 [4.1%]; P = .18) (Figure). Patients receiving any letter vs no letter had a higher percentage of telemedicine visits and future scheduled visits within 1 month (telemedicine visits: 132 [1.3%] vs 4 [0.4%]; P = .006; future visits: 1439 [14.4%] vs 130 [11.7%], P = .02).

    Patients receiving the tailored letter vs standard letter scheduled more visits, achieving statistical significance overall (768 [15.3%] vs 671 [13.4%]; P = .006) and for individuals younger than 65 years, women, Black patients, White patients, patients with commercial insurance, patients with lower household income, and patients receiving orthopedic services (Table). For the subset randomized by delivery type, patients with a household income below the median were more likely to schedule a visit when mailed a letter vs receiving an electronic letter, with a statistically significant interaction by household income (230 [16.6%] vs 177 [13.0%]).

    Discussion

    In this randomized clinical trial, we found that a large proportion of patients who canceled visits and procedures early in the COVID-19 pandemic did not reschedule once reopening occurred. A single message targeted directly to these patients did not affect the return to in-person visits within 1 month, but it resulted in a small increase in reengagement through telemedicine and rescheduling of future visits. Study limitations include limited outcomes and unknown generalizability to other locations and times. Strategies to improve patient engagement, with attention to message framing and delivery mode, are needed to encourage continuity of health care in the era of COVID-19.

    Back to top
    Article Information

    Accepted for Publication: April 27, 2021.

    Published: June 30, 2021. doi:10.1001/jamanetworkopen.2021.15211

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Cappola AR et al. JAMA Network Open.

    Corresponding Author: Anne R. Cappola, MD, ScM, Division of Endocrinology, Diabetes, and Metabolism, Perelman School of Medicine at the University of Pennsylvania, 3400 Civic Center Blvd, Bldg 421, 12th Flr, Philadelphia, PA 19104-5160 (acappola@pennmedicine.upenn.edu).

    Author Contributions: Dr Cappola had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Cappola, Ives, Gonzales, Cappella.

    Acquisition, analysis, or interpretation of data: Cappola, Schriver, Mowery, Wollack.

    Drafting of the manuscript: Cappola, Mowery, Cappella.

    Critical revision of the manuscript for important intellectual content: Cappola, Schriver, Wollack, Ives, Gonzales, Cappella.

    Statistical analysis: Schriver.

    Obtained funding: Ives, Gonzales.

    Administrative, technical, or material support: Mowery, Wollack, Ives, Gonzales.

    Supervision: Mowery.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: This study received funding from the University of Pennsylvania Health System.

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Data Sharing Statement: See Supplement 3.

    Additional Information: ClinicalTrials.gov Identifier: NCT04418947.

    References
    1.
    Czeisler  MÉ, Marynak  K, Clarke  KEN,  et al.  Delay or avoidance of medical care because of COVID-19–related concerns—United States, June 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(36):1250-1257. doi:10.15585/mmwr.mm6936a4PubMedGoogle ScholarCrossref
    2.
    Findling  MG, Blendon  RJ, Benson  JM. Delayed care with harmful health consequences—reported experiences from national surveys during coronavirus disease 2019. JAMA Health Forum. Published online December 14, 2020.
    3.
    Jeffery  MM, D’Onofrio  G, Paek  H,  et al.  Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 pandemic in the US.   JAMA Intern Med. 2020;180(10):1328-1333. doi:10.1001/jamainternmed.2020.3288PubMedGoogle ScholarCrossref
    4.
    ReviveHealth. COVID-19 Consumer Survey Update, April 2020. Accessed September 20, 2020. https://www.thinkrevivehealth.com/covid-19/survey-findings-part-3
    5.
    Fishbein  M, Cappella  JN.  The role of theory in developing effective health communications.   J Commun. 2006 Aug 1;56:S1–17. doi:10.1111/j.1460-2466.2006.00280.xGoogle ScholarCrossref
    6.
    Lee  SJ, Brennan  E, Gibson  LA,  et al.  Predictive validity of an empirical approach for selecting promising message topics: a randomized-controlled study.   J Commun. 2016;66(3):433-453. doi:10.1111/jcom.12227PubMedGoogle ScholarCrossref
    ×