Effect of Physician-Delivered COVID-19 Public Health Messages and Messages Acknowledging Racial Inequity on Black and White Adults’ Knowledge, Beliefs, and Practices Related to COVID-19: A Randomized Clinical Trial | Health Disparities | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Enrollment and Randomization of Participants
Enrollment and Randomization of Participants
Figure 2.  Distribution of the Knowledge Gap Score in the Control and Intervention Groups
Distribution of the Knowledge Gap Score in the Control and Intervention Groups

Whiskers indicate 95% CIs.

Table 1.  Summary of Participant Characteristicsa
Summary of Participant Characteristicsa
Table 2.  Impact of Any Video Message Intervention vs Control: Incidence Ratea
Impact of Any Video Message Intervention vs Control: Incidence Ratea
Table 3.  Impact of Tailoring Messages: Incidence Rate Ratiosa
Impact of Tailoring Messages: Incidence Rate Ratiosa
1.
Zhao  J, Lee  M, Ghader  S, Younes  H, Darzi  A, Xiong  C, Zhang  L. Quarantine fatigue: first-ever decrease in social distancing measures after the COVID-19 pandemic outbreak before reopening United States. Preprint updated June 11, 2020. Accessed June 8, 2021. https://arxiv.org/abs/2006.03716
2.
McDonnell Nieto del Rio  G. Doctors plead with Americans to take the virus surge seriously. The New York Times. Accessed June 8, 2021. https://www.nytimes.com/live/2020/11/15/world/covid-19-coronavirus#doctors-plead-with-americans-to-take-the-virus-surge-seriously
3.
Alsan  MS, Stanford  FC, Banerjee  A,  et al.  Comparison of knowledge and information-seeking behavior after general COVID-19 public health messages and messages tailored for Black and Latinx communities: a randomized controlled trial.   Ann Intern Med. 2021;174(4):484-492.PubMedGoogle ScholarCrossref
4.
US Centers for Disease Control and Prevention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity. Updated May 26, 2021. Accessed January 21, 2021. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/hospitalization-death-by-race-ethnicity.html
5.
American Medical Association. AMA Board of Trustees pledges action against racism, police brutality. June 7, 2020. Accessed June 8, 2021. https://www.ama-assn.org/press-center/ama-statements/ama-board-trustees-pledges-action-against-racism-police-brutality
6.
US Census Bureau. 2018 Population estimates by age, sex, race and Hispanic origin. June 19, 2019. Accessed June 19, 2020. https://www.census.gov/newsroom/press-kits/2019/detailed-estimates.html
7.
American Medical Association. AMA applauds congressional action on prescription drug transparency. April 9, 2019. Accessed June 8, 2021. https://www.ama-assn.org/press-center/ama-statements/ama-applauds-congressional-action-prescription-drug-transparency
8.
Wertenbroch  K, Skiera  B.  Measuring consumers’ willingness to pay at the point of purchase.   J Marketing Res. 2002;39(2):228-241. doi:10.1509/jmkr.39.2.228.19086Google ScholarCrossref
9.
Hainmueller  J.  Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies.   Political Analysis. 2012;20(1):25-46. doi:10.1093/pan/mpr025Google ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Infectious Diseases
    July 14, 2021

    Effect of Physician-Delivered COVID-19 Public Health Messages and Messages Acknowledging Racial Inequity on Black and White Adults’ Knowledge, Beliefs, and Practices Related to COVID-19: A Randomized Clinical Trial

    Author Affiliations
    • 1Harvard Medical School, Boston, Massachusetts
    • 2Department of Pediatrics–General Pediatrics, Massachusetts General Hospital for Children, Boston
    • 3Department of Psychiatry, Massachusetts General Hospital, Boston
    • 4Department of Psychiatry, McLean Hospital, Belmont, Massachusetts
    • 5Harvard Kennedy School of Government, Cambridge, Massachusetts
    • 6Department of Medicine–Neuroendocrine Unit, Department of Pediatrics–Endocrinology, Massachusetts General Hospital, Boston
    • 7Department of Economics, Massachusetts Institute of Technology, Cambridge
    • 8Department of Economics, Harvard University, Cambridge, Massachusetts
    • 9Department of Economics, Stanford University, Stanford, California
    • 10Department of Economics, Ludwig Maximilian University of Munich, Munich, Germany
    • 11Paris School of Economics, Paris, France
    • 12Yale School of Management, New Haven, Connecticut
    • 13Clinical Translational Epidemiology Unit, Massachusetts General Hospital, Boston
    JAMA Netw Open. 2021;4(7):e2117115. doi:10.1001/jamanetworkopen.2021.17115
    Key Points

    Question  Do messages delivered by physicians increase COVID-19 knowledge and improve preventive behaviors among White and Black individuals?

    Findings  In this randomized clinical trial of 18 223 White and Black adults, a message delivered by a physician increased COVID-19 knowledge and shifted information-seeking and self-protective behaviors. Effects did not differ by race, and tailoring messages to specific communities did not exhibit a differential effect on knowledge or individual behavior.

    Meaning  These findings suggest that physician messaging campaigns may be effective in persuading members of society from a broad range of backgrounds to seek information and adopt preventive behaviors to combat COVID-19.

    Abstract

    Importance  Social distancing is critical to the control of COVID-19, which has disproportionately affected the Black community. Physician-delivered messages may increase adherence to these behaviors.

    Objectives  To determine whether messages delivered by physicians improve COVID-19 knowledge and preventive behaviors and to assess the differential effectiveness of messages tailored to the Black community.

    Design, Setting, and Participants  This randomized clinical trial of self-identified White and Black adults with less than a college education was conducted from August 7 to September 6, 2020. Of 44 743 volunteers screened, 30 174 were eligible, 5534 did not consent or failed attention checks, and 4163 left the survey before randomization. The final sample had 20 460 individuals (participation rate, 68%). Participants were randomly assigned to receive video messages on COVID-19 or other health topics.

    Interventions  Participants saw video messages delivered either by a Black or a White study physician. In the control groups, participants saw 3 placebo videos with generic health topics. In the treatment group, they saw 3 videos on COVID-19, recorded by several physicians of varied age, gender, and race. Video 1 discussed common symptoms. Video 2 highlighted case numbers; in one group, the unequal burden of the disease by race was discussed. Video 3 described US Centers for Disease Control and Prevention social distancing guidelines. Participants in both the control and intervention groups were also randomly assigned to see 1 of 2 American Medical Association statements, one on structural racism and the other on drug price transparency.

    Main Outcomes and Measures  Knowledge, beliefs, and practices related to COVID-19, demand for information, willingness to pay for masks, and self-reported behavior.

    Results  Overall, 18 223 participants (9168 Black; 9055 White) completed the survey (9980 [55.9%] women, mean [SD] age, 40.2 [17.8] years). Overall, 6303 Black participants (34.6%) and 7842 White participants (43.0%) were assigned to the intervention group, and 1576 Black participants (8.6%) and 1968 White participants (10.8%) were assigned to the control group. Compared with the control group, the intervention group had smaller gaps in COVID-19 knowledge (incidence rate ratio [IRR], 0.89 [95% CI, 0.87-0.91]) and greater demand for COVID-19 information (IRR, 1.05 [95% CI, 1.01-1.11]), willingness to pay for a mask (difference, $0.50 [95% CI, $0.15-$0.85]). Self-reported safety behavior improved, although the difference was not statistically significant (IRR, 0.96 [95% CI, 0.92-1.01]; P = .08). Effects did not differ by race (F = 0.0112; P > .99) or in different intervention groups (F = 0.324; P > .99).

    Conclusions and Relevance  In this study, a physician messaging campaign was effective in increasing COVID-19 knowledge, information-seeking, and self-reported protective behaviors among diverse groups. Studies implemented at scale are needed to confirm clinical importance.

    Trial Registration  ClinicalTrials.gov Identifier: NCT04502056

    Introduction

    Physical distancing and mask wearing remain essential to the control of COVID-19, yet vigilance has decreased over time.1 To address fatigue, health care professionals have used social media to spread public health messages.2 There is evidence that these messages improve knowledge, but there are less data on whether they change behavior.3

    Black US residents have been disproportionately affected by the pandemic.4 This reflects the cumulative impact of systemic racism, acknowledged as a public health threat by the American Medical Association (AMA) in a June 2020 statement.5 This raises the question on whether the effectiveness of public health messages regarding COVID-19 would be enhanced if tailored to the Black community. The focus of this study was to identify whether messages delivered by physicians increase COVID-19 knowledge and improve preventive behaviors for White and Black individuals and to assess whether various ways of increasing the relevance of messages to the Black community (ie, physician race, AMA acknowledgments of racial injustices, or information about the disproportionate burden of COVID-19 on the Black community) affects their impact on both White and Black participants.

    Methods
    Trial Design and Oversight

    The trial flowchart (Figure 1; eFigure 1 in Supplement 1) describes the factorial design and the allocation of participants to each intervention arm. The design was approved by the ethical review boards of Massachusetts Institute of Technology (MIT) and Stanford, with Massachusetts General Hospital, Yale, and Harvard ceding authority to MIT. All participants provided written informed consent. The trial and the outcomes were registered on ClinicalTrials.gov (NCT04502056). Planned analyses were published on the American Economic Association trial registry (AEARCTR-0006177). The pre-analysis plan and institutional review board–approved protocol are available in Supplement 2. This study followed the Consolidated Standards of Reporting Trials (CONSORT) and American Association for Public Research (AAPOR) reporting guidelines.

    Participants

    Individuals were recruited online throughout the United States by the survey company Lucid from August 7, 2020, to September 6, 2020. Lucid recruits survey participants by advertising surveys to third-party suppliers, including double opt-in panels, publishing networks, social media, and other types of online communities. Participants aged 18 years or older, self-identifying as White or Black, and without a college degree were eligible. We focused on these 2 groups because we were interested in tailoring messages toward the Black community as well as the reaction of the White community to the tailoring of those messages. Latinx individuals were included in our previous study,3 along with specific tailoring toward this community. Recruitment used quotas to match the 2018 population estimates by age, sex, and race issued by the US Census Bureau.6

    Interventions
    Intervention vs Control Messages

    After reading the informed consent form (approved by the institutional review board) (eAppendix 1 in Supplement 1) and agreeing to participate, all individuals answered sociodemographic questions, saw 3 videos, and then completed the outcome survey questions. In the control groups, participants saw 3 placebo videos with generic health topics, including fitness guidelines, recommended sugar intake, and the importance of adequate sleep. In the treatment group, they saw 3 videos on COVID-19, recorded by several physicians of varied age, gender, and race. Participants in each group (placebo and intervention) saw video messages delivered either by a Black or a White study physician (including L.O.-N., M.A., F.C.S., and E.W.).

    Video 1 defined COVID-19 and discussed common symptoms associated with COVID-19 as well as asymptomatic transmission. Video 2 reminded the viewer that COVID-19 was actively circulating in the United States. Video 3 described the Centers for Disease Control and Prevention social distancing guidelines (complete scripts appear in eAppendix 1 in Supplement 1).

    Unequal Burden of COVID-19

    Video 2 in the COVID-19 intervention had 2 randomized variants. Script 1 emphasized the number of new cases in the week of July 6, 2020. Script 2 added that, controlling for age, Black individuals were 3 times as likely to become infected as White individuals and 4 times as likely to die from it. These 2 variants of video 2 were cross-randomized with the intervention.

    AMA Antiracism or Placebo Statement

    At the beginning of the study, all participants saw a video of an actor delivering the script of a public statement by the AMA. The AMA antiracism statement, issued on June 7, 2020, “recognizes that racism in its systemic, structural, institutional, and interpersonal forms is an urgent threat to public health, the advancement of health equity, and a barrier to excellence in the delivery of medical care.”5 The AMA placebo intervention was an AMA statement on drug pricing.7 The race and gender of the person reading the statement were randomized to each recipient.

    Outcomes

    Most outcomes were measured online immediately following the intervention or the placebo. The prespecified primary outcomes were knowledge, beliefs, and practices related to COVID-19, measured immediately after the intervention; intended behavior, measured immediately after the intervention; and knowledge and behavior, measured a few days after the intervention.

    eAppendix 2 in Supplement 1 describes the outcome measurement in detail. Primary outcomes presented in the main text include 5 outcomes. First, knowledge gap, which measures knowledge and beliefs. Participants were asked to identify ways to prevent COVID-19 spread and identify 4 common symptoms. The knowledge gap outcome is an integer that can have values from 0 (no error) to 10 (10 errors). Second, information seeking was measured by offering participants the option of requesting additional information on COVID-19–related resources by clicking on up to 5 links that included more content. We measured information-seeking behavior as the number of links in which participants expressed interest, a count variable between 0 (lowest information-seeking behavior) and 5 (greatest information-seeking behavior). Third, self-reported safety behavior was measured a few days after the initial intervention among a subsample that was eligible for follow-up and could be tracked. Participants were asked about how often they engaged in 4 behaviors of interest: (1) whether they wore a mask indoors; (2) whether they wore a mask outdoors; (3) whether they washed their hands; and (4) whether they followed social distancing guidelines. The safety gap index had values of 0 (if a participant reported that they always practiced the 4 behaviors of interest) to 4 (participant reported that they practiced none of the behaviors). Fourth, at the end of the survey, each participant was asked the price they would be willing to pay for high-quality masks. Each participant was entered into a draw to receive either a coupon for masks or a gift card to an online retailer. When a participant was selected by the draw, a price was then randomly drawn for the coupon. If their reported willingness to pay was greater than the price, they would receive the masks; otherwise, they would receive the gift card. Therefore, it was in participants’ best interest to report their true willingness to pay for the masks. This type of procedure has been shown to lead to truthful reporting.8

    We collected data on 3 secondary outcomes specified in our pre-analysis plan (Supplement 2). First, we asked participants to report their judgment of how well federal and state policies balanced opening the economy and limiting the health impacts of COVID-19. Second, we measured how participants prioritized COVID-19 protection vs other issues by asking the participants how they would want to allocate a donation of $1000 (which the research team would fund) between 2 charities: Give a Mask or the Alzheimer Association. Third, we asked whether they would prioritize a COVID-19 relief donation to a Black-focused charity (the BET COVID-19 relief fund) or a general charity (the Give Directly Project 100+ relief program).

    Randomization

    Randomization into intervention vs control and between interventions was stratified according to sex, age (≥45 years), race, and self-identified partisan affiliation (Republican vs other affiliation). The flowchart (Figure 1) summarizes the randomization; a more detailed version is available as eFigure 1 in Supplement 1.

    Participants were first randomized into the AMA antiracism or placebo statements, with equal probability. They were then randomly assigned to intervention or control. One of 5 participants was assigned to control; the remainder were assigned to an intervention.

    Within each group, participants were randomized into Black or White physician groups with equal probability. Intervention participants were further randomized, with equal probability, into 1 the 2 arms for video 2: they either received the information about the unequal burden of disease or did not. Randomization was performed using the Qualtrics platform, using a randomizer block within each stratum with the option to evenly present elements.

    Statistical Analysis

    We determined that a sample of 20 000 individuals (10 000 Black and 10 000 White) would provide 85% power to detect effect sizes of 0.05 SDs for intervention relative to control and for effects of specific variations in message content. These are small effect sizes that would justify scale up of this inexpensive intervention.

    The analysis was performed by original assigned group, and it included all participants who completed the survey. Multivariable regression models include the stratifying variables (age × sex × race × Republican identity dummies).

    Effect of Any Video Message Intervention Relative to Control

    To analyze the effect of seeing any video message on the knowledge gap, information-seeking behavior, and safety behavior outcomes, we fit the following negative binomial regression model for the count outcome:

    log(μi) = β0 + β1interventioni + βT2stratumi,

    where μi is the estimated mean outcome value (knowledge gap count, count of demanded links, or safety behavior count), interventioni is an indicator that equals 1 if the individual received the intervention videos and 0 if they received the placebo videos, and stratumi is a vector of indicator variables. Similar models are estimated for binary regressions (using the logistic regression equation) and continuous variable (using ordinary least squares) (eAppendix 2 in Supplement 1). Because there were multiple outcomes, we provide P values and q values adjusted for false discovery rates.

    Effect of Variation in the Message Framing

    To analyze the impact of different arms, we fit a negative binomial regression model to the count data:

    log(μi) = β0 + β1Black physiciani + β2AMA antiracismi + β3interventioni + (β4Black physiciani × interventioni) + (β5AMA antracismi × interventioni) + (β6mortality differencei × interventioni) + βT7stratumi,

    where Black physiciani is an indicator that equals 1 if the physician was a Black individual and 0 otherwise; AMA antiracismi is an indicator for the AMA statement featuring the antiracism message (rather than the drug pricing message); mortality differencei is an indicator for video 2 mentioning the excess mortality from COVID-19. Similar specifications are fitted with logistic regression for binary variables and ordinary least squares for continuous variables.

    To address possible bias stemming from nonrandom attrition for the follow-up survey, we weighted the follow-up data using Hainmueller entropy weights,9 which ensures that the observed baseline characteristics of the follow-up sample matches the original sample as closely as possible (eAppendix 2 in Supplement 1). eAppendix 3 in Supplement 1 describes further robustness checks and subgroups analysis.

    Analyses were performed using R version 4.0.3 (R Project for Statistical Computing). Statistical significance was set at P < .05, and all tests were 2-tailed.

    Results
    Trial Sample

    The trial sample was enrolled from August 7 through September 6, 2020. Of 44 743 screened, 30 174 were eligible for participation, 5534 individuals did not consent or failed 2 simple attention checks, 4163 left the survey before randomization, and 17 were excluded from the analysis due to unknown race or multiple survey completion. The final sample at randomization had 20 460 individuals, for a participation rate of 68%. After attrition, 18 223 individuals were included in the study.

    Summary statistics (Table 1) were computed on the sample that was randomized and that completed all main baseline variables. Our sample included 9880 (55.9%) women. The mean (SD) age was 40.2 (17.8) years, 4206 (23.8%) reported household incomes greater than $60 000, 4228 Black participants (53.7%) and 2749 White participants (28.0%) identified as members of the Democratic party, 12 106 (68.4%) reported always wearing a mask indoors (outside the home), and 5517 (31.2%) reported always wearing a mask when outdoors. Black participants were twice as likely as White participants to report always wearing a mask outdoors (3513 [44.6%] vs 2004 [20.4%]) and were also more likely to report practicing hand hygiene (5084 [64.5%] vs 5695 [58.1%]) and physical distancing (4587 [58.2%] vs 4874 [49.7%]). Baseline covariates were balanced between intervention and control at all stages (eAppendix 5 and eTable 1A in Supplement 1).

    Attrition

    Due to the online nature of the survey, participants could exit at any point after watching the video messages without finishing the survey. To include as many participants as possible in the analysis, we included everyone who answered knowledge questions for the knowledge outcome. Overall, 18 762 participants included in the initial randomization were included at that stage (9445 Black; 9317 White). For other outcomes, except follow-up outcomes, we included 18 223 participants (9168 Black; 9055 White) who completed the survey. Attrition was similar in all groups (eTable 1B in Supplement 1).

    The adherence to safety behavior was collected a few days later among a smaller follow-up sample that experienced more attrition. Overall, 12 591 individuals were included in the follow-up sample to track; only people who had given permission to Lucid to be recontacted were included in this sample. Of those, we successfully contacted 6217. Attrition at this stage was 51.8% in the treatment group and 51.0% in the control group. A systematic analysis of attrition at all 3 stages (eTable 1B in Supplement 1) revealed no systematic difference between the characteristic of attritors in the treatment and comparison groups. At the realized sample size, the power was 0.055 SDs for the knowledge sample (18 762 respondents), 0.056 SDs for all other outcomes in the first survey, and 0.095 SDs for the follow up sample (6217 respondents).

    Effects of Any Video Message, Intervention vs Control

    Receiving any COVID-19 video improved knowledge of COVID-19 and adherence to preventive practices. Table 2 shows main outcomes overall and by racial group.

    Main Sample

    The knowledge gap incidence rate was 0.241 (95% CI, 0.235-0.246) in the control group and 0.214 (95% CI, 0.211-0.217) in intervention group. The intervention had a significant effect on reducing knowledge gaps relative to the control group (estimated incidence rate ratio [IRR], 0.89 [95% CI, 0.87-0.91]; P < .001; q < .001). In the control group, 315 participants (8.4%) had no gap in knowledge (Figure 2). The proportion increased to 13.0% (1915 participants) in the intervention group; 1352 participants (35.9%) in the control group had a 1-point knowledge gap compared with 6636 (44.2%) in the intervention group. The incidence rate of information seeking behavior was 0.320 in the control group and 0.338 in the intervention group (estimated IRR, 1.05 [95% CI, 1.01-1.11]; P = .03; q = .04). The willingness to pay for a mask increased from $14.07 in the control group to $14.58 in the intervention group (difference, $0.50 [95% CI, $0.15-$0.85]; P = .005; q = .013).

    The intervention was impactful for both Black and White participants. It was more impactful for White participants vs Black participants on knowledge (IRR, 0.80 [95% CI, 0.76-0.83] vs 0.94 [95% CI 0.91-0.97]; P for difference < .001), but equally impactful for all the other measures (eTable 2C in Supplement 1). The F statistics for the hypothesis that the coefficients across all outcomes are jointly different for both races was 0.0112 (P > .99).

    Follow-up Survey

    At the follow-up survey (Table 2), which was realized on a smaller sample with larger attrition, the safety gap index incidence rate was 0.47 (95% CI, 0.45-0.48) in the control group and 0.45 (95% CI, 0.44-0.46) in the treatment group (IRR, 0.96 [95% CI 0.92-1.01]; P = .08, q = .08). Overall, 244 participants (20.1%) and 218 participants (18.0%) in the control group and 1040 participants (21.6%) and 837 participants (17.4%) in the intervention group reported respecting all and none, respectively, of 4 safety practices (eFigure 2 in Supplement 1).

    Supplemental Outcomes

    eTable 3 in Supplement 1 reports effects on secondary outcomes. One noteworthy result is that we found significant effects of the intervention on donations both to a COVID-19 charity (vs Alzheimer) and to a COVID-19 relief charity specific to Black US residents (vs COVID-19 economic relief for everyone).

    Effects of the Framing of the Videos

    None of the efforts to make the messages more relevant to the Black community had a differential effect on knowledge or individual behavior (Table 3). An F statistic that all effects for Black physician, AMA antiracism statement, and disproportionate burden statement are jointly different from 0 was 0.324 (P > .99).

    The only outcome that was affected by some of these variants was the desired donation to a Black-specific COVID-19 charity (eTable 4 in Supplement 1). eTable 5 in Supplement 1 shows that the combination of the AMA antiracism statement, a Black physician, and a video acknowledging racial disparities significantly increased how White and Black participants allocated donations to a charity focused on Black communities (ordinary least squares coefficient, $30.60 [95% CI, $10.93-$50.27]; P = .002).

    Heterogeneity by Sex, Education, Income, and Political Affiliation

    Across all conditions, there were no statistically significant differences by sex or political affiliation (eTable 2A and eTable 2E in Supplement 1). The effect of intervention relative to control on knowledge gaps was more pronounced for participants with a high school education or more (eTable 2B in Supplement 1). However, there was no significant difference for the demand for links and willingness to pay for a mask. The intervention was more impactful among participants with lower incomes (ie, <$60 000).

    Discussion

    Exposure to public health video messages about COVID-19 recorded by a diverse set of physicians decreased knowledge gaps on COVID-19 symptoms, preventive behaviors, and transmission among Black and White participants with modest incomes, relative to a control condition that saw placebo videos. The effect on knowledge was substantial and clear. This replicates the results of our prior study conducted in May 20203 and extends it to White participants. New to this study, we also found a modest but statistically significant increase in the demand for more information, the willingness to pay for high quality masks, and self-reported behavior at follow-up.

    Despite the heightened awareness of racial justice issues during the period of this intervention and increased polarization in the political discourse in the run-up to the presidential election, effects are remarkably similar across racial and political lines. These results suggest that physicians still have the ability to inform and persuade members of society from a broad range of backgrounds.

    Our results also indicate that tailoring the message to specific communities did not affect its impact on behavior. Both White and Black physicians were able to effectively convey the importance of masking and social distancing to Black and White participants (unlike the previous study, in which concordance was important to change behavior3). The AMA antiracism statement did not affect participants’ attentiveness to the message delivered, for Black or White respondents. Acknowledgment of structural racism remains important, but it may not be sufficient to increase the level of trust from the Black community.

    Importantly, the intervention made both Black and White participants more willing to focus resources both toward COVID-19 in general and toward the Black community in particular. Highlighting health conditions that disproportionately affect the Black community is one step toward increasing public consciousness of structural racism.

    Limitations

    There are several limitations of the study. First, it was conducted online, and the participants may not be representative of the population with less than a college degree, given that they have access to the internet and are used to participating in online studies. Second, although information-seeking behavior and willingness to pay for masks were objectively measured, participants’ preventive health behaviors were not directly observed. Outcomes were self-reported. Third, outcomes might be subject to social desirability bias. Fourth, there may be bias due to attrition, particularly for the self-reported safety behavior, given that only a small fraction of the sample could be followed up a few days after the initial intervention. While the observable variables remain balanced, the unobservable may not be. Furthermore, while we found consistent effects on knowledge, information seeking, the willingness to pay for masks, and self-reported behavior, the final clinical significance of these findings is uncertain because effects on all were quantitatively small.

    Conclusions

    These results suggest physician messaging campaigns may be effective and trust in Black and White physicians is equally high. There is no evidence of preexisting bias that would have led the intervention to have a negative effect. Because it is inexpensive, it could be a promising way to encourage behavior at scale. However, future studies implemented at a large scale are needed to confirm whether these kinds of interventions can change behavior in a way that will affect clinical outcomes. In ongoing work, we will study scale up messaging by doctors using social media.

    Back to top
    Article Information

    Accepted for Publication: May 9, 2021.

    Published: July 14, 2021. doi:10.1001/jamanetworkopen.2021.17115

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Torres C et al. JAMA Network Open.

    Corresponding Author: Esther Duflo, PhD, Department of Economics, Massachusetts Institute of Technology, 77 Massachusetts Ave, Building E52-544, Cambridge, MA 02139 (eduflo@mit.edu).

    Author Contributions: Dr Duflo had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Torres, Ogbu-Nwobodo, Alsan, Banerjee, Breza, Chandrasekhar, Eichmeyer, Goldsmith-Pinkham, Olken, Vautrey, Duflo.

    Acquisition, analysis, or interpretation of data: Ogbu-Nwobodo, Stanford, Banerjee, Breza, Chandrasekhar, Karnani, Loisel, Goldsmith-Pinkham, Olken, Vautrey, Warner, Duflo.

    Drafting of the manuscript: Ogbu-Nwobodo, Alsan, Stanford, Chandrasekhar, Loisel, Olken, Vautrey, Warner, Duflo.

    Critical revision of the manuscript for important intellectual content: Torres, Ogbu-Nwobodo, Stanford, Banerjee, Breza, Eichmeyer, Karnani, Goldsmith-Pinkham, Olken, Warner.

    Statistical analysis: Breza, Chandrasekhar, Karnani, Loisel, Goldsmith-Pinkham, Olken, Vautrey, Duflo.

    Obtained funding: Breza, Goldsmith-Pinkham, Duflo.

    Administrative, technical, or material support: Torres, Alsan, Stanford, Breza, Karnani, Goldsmith-Pinkham, Warner.

    Supervision: Torres, Ogbu-Nwobodo, Breza, Vautrey, Duflo.

    Conflict of Interest Disclosures: Dr Olken reported receiving ad credits from Facebook outside the submitted work. No other disclosures were reported.

    Funding/Support: This work was supported by grant 2029880 from the National Science Foundation to Drs Alsan and Duflo, the Physician/Scientist Development Award from the Executive Committee on Research at Massachusetts General Hospital to Dr Stanford, grant P30 DK040561from the National Institutes of Health to Dr Stanford, grant L30 DK118710 from the National Institutes of Health to Dr Stanford, and the Clinician-Teacher Development Award from the Massachusetts General Physicians Organization at Massachusetts General Hospital to Dr Torres.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Group Members: The members of the COVID-19 Working Group appear in Supplement 3.

    Disclaimer: The findings and conclusions expressed are solely those of the authors and do not represent the views of their funders.

    Data Sharing Statement: See Supplement 4.

    Additional Contributions: We thank the Center for Diversity and Inclusion, especially Sandra Pena Ordonez, BS, and Elena Olson, JD for their assistance in this project. We thank Minjeong Joyce Kim, BS (Stanford University), and Sirena Yu (Massachusetts Institute of Technology), for their research assistance. These individuals were not compensated for their time.

    References
    1.
    Zhao  J, Lee  M, Ghader  S, Younes  H, Darzi  A, Xiong  C, Zhang  L. Quarantine fatigue: first-ever decrease in social distancing measures after the COVID-19 pandemic outbreak before reopening United States. Preprint updated June 11, 2020. Accessed June 8, 2021. https://arxiv.org/abs/2006.03716
    2.
    McDonnell Nieto del Rio  G. Doctors plead with Americans to take the virus surge seriously. The New York Times. Accessed June 8, 2021. https://www.nytimes.com/live/2020/11/15/world/covid-19-coronavirus#doctors-plead-with-americans-to-take-the-virus-surge-seriously
    3.
    Alsan  MS, Stanford  FC, Banerjee  A,  et al.  Comparison of knowledge and information-seeking behavior after general COVID-19 public health messages and messages tailored for Black and Latinx communities: a randomized controlled trial.   Ann Intern Med. 2021;174(4):484-492.PubMedGoogle ScholarCrossref
    4.
    US Centers for Disease Control and Prevention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity. Updated May 26, 2021. Accessed January 21, 2021. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/hospitalization-death-by-race-ethnicity.html
    5.
    American Medical Association. AMA Board of Trustees pledges action against racism, police brutality. June 7, 2020. Accessed June 8, 2021. https://www.ama-assn.org/press-center/ama-statements/ama-board-trustees-pledges-action-against-racism-police-brutality
    6.
    US Census Bureau. 2018 Population estimates by age, sex, race and Hispanic origin. June 19, 2019. Accessed June 19, 2020. https://www.census.gov/newsroom/press-kits/2019/detailed-estimates.html
    7.
    American Medical Association. AMA applauds congressional action on prescription drug transparency. April 9, 2019. Accessed June 8, 2021. https://www.ama-assn.org/press-center/ama-statements/ama-applauds-congressional-action-prescription-drug-transparency
    8.
    Wertenbroch  K, Skiera  B.  Measuring consumers’ willingness to pay at the point of purchase.   J Marketing Res. 2002;39(2):228-241. doi:10.1509/jmkr.39.2.228.19086Google ScholarCrossref
    9.
    Hainmueller  J.  Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies.   Political Analysis. 2012;20(1):25-46. doi:10.1093/pan/mpr025Google ScholarCrossref
    ×