Effect of Physician-Delivered COVID-19 Public Health Messages and Messages Acknowledging Racial Inequity on Black and White Adults’ Knowledge, Beliefs, and Practices Related to COVID-19

Key Points Question Do messages delivered by physicians increase COVID-19 knowledge and improve preventive behaviors among White and Black individuals? Findings In this randomized clinical trial of 18 223 White and Black adults, a message delivered by a physician increased COVID-19 knowledge and shifted information-seeking and self-protective behaviors. Effects did not differ by race, and tailoring messages to specific communities did not exhibit a differential effect on knowledge or individual behavior. Meaning These findings suggest that physician messaging campaigns may be effective in persuading members of society from a broad range of backgrounds to seek information and adopt preventive behaviors to combat COVID-19.


Introduction
Physical distancing and mask wearing remain essential to the control of COVID-19, yet vigilance has decreased over time. 1 To address fatigue, health care professionals have used social media to spread public health messages. 2There is evidence that these messages improve knowledge, but there are less data on whether they change behavior. 3ack US residents have been disproportionately affected by the pandemic. 4This reflects the cumulative impact of systemic racism, acknowledged as a public health threat by the American Medical Association (AMA) in a June 2020 statement. 5This raises the question on whether the effectiveness of public health messages regarding COVID-19 would be enhanced if tailored to the Black community.The focus of this study was to identify whether messages delivered by physicians increase COVID-19 knowledge and improve preventive behaviors for White and Black individuals and to assess whether various ways of increasing the relevance of messages to the Black community (ie, physician race, AMA acknowledgments of racial injustices, or information about the disproportionate burden of COVID-19 on the Black community) affects their impact on both White and Black participants.

Methods Trial Design and Oversight
The trial flowchart (Figure 1; eFigure 1 in Supplement 1) describes the factorial design and the allocation of participants to each intervention arm.The design was approved by the ethical review boards of Massachusetts Institute of Technology (MIT) and Stanford, with Massachusetts General Hospital, Yale, and Harvard ceding authority to MIT.All participants provided written informed consent.The trial and the outcomes were registered on ClinicalTrials.gov(NCT04502056).Planned analyses were published on the American Economic Association trial registry (AEARCTR-0006177).
The pre-analysis plan and institutional review board-approved protocol are available in Supplement 2. This study followed the Consolidated Standards of Reporting Trials (CONSORT) and American Association for Public Research (AAPOR) reporting guidelines.

Participants
Individuals were recruited online throughout the United States by the survey company Lucid from August 7, 2020, to September 6, 2020.Lucid recruits survey participants by advertising surveys to third-party suppliers, including double opt-in panels, publishing networks, social media, and other types of online communities.Participants aged 18 years or older, self-identifying as White or Black, and without a college degree were eligible.We focused on these 2 groups because we were interested in tailoring messages toward the Black community as well as the reaction of the White

Unequal Burden of COVID-19
Video 2 in the COVID-19 intervention had 2 randomized variants.Script 1 emphasized the number of new cases in the week of July 6, 2020.Script 2 added that, controlling for age, Black individuals were 3 times as likely to become infected as White individuals and 4 times as likely to die from it.These 2 variants of video 2 were cross-randomized with the intervention.

AMA Antiracism or Placebo Statement
At the beginning of the study, all participants saw a video of an actor delivering the script of a public statement by the AMA.The AMA antiracism statement, issued on June 7, 2020, "recognizes that racism in its systemic, structural, institutional, and interpersonal forms is an urgent threat to public health, the advancement of health equity, and a barrier to excellence in the delivery of medical care." 5 The AMA placebo intervention was an AMA statement on drug pricing. 7The race and gender of the person reading the statement were randomized to each recipient.

Outcomes
Most outcomes were measured online immediately following the intervention or the placebo.The prespecified primary outcomes were knowledge, beliefs, and practices related to COVID-19, measured immediately after the intervention; intended behavior, measured immediately after the intervention; and knowledge and behavior, measured a few days after the intervention.to 10 (10 errors).Second, information seeking was measured by offering participants the option of requesting additional information on COVID-19-related resources by clicking on up to 5 links that included more content.We measured information-seeking behavior as the number of links in which participants expressed interest, a count variable between 0 (lowest information-seeking behavior) and 5 (greatest information-seeking behavior).Third, self-reported safety behavior was measured a few days after the initial intervention among a subsample that was eligible for follow-up and could be tracked.Participants were asked about how often they engaged in 4 behaviors of interest: (1)   whether they wore a mask indoors; (2) whether they wore a mask outdoors; (3) whether they washed their hands; and (4) whether they followed social distancing guidelines.The safety gap index had values of 0 (if a participant reported that they always practiced the 4 behaviors of interest) to 4 (participant reported that they practiced none of the behaviors).Fourth, at the end of the survey, each participant was asked the price they would be willing to pay for high-quality masks.Each participant was entered into a draw to receive either a coupon for masks or a gift card to an online retailer.When a participant was selected by the draw, a price was then randomly drawn for the coupon.If their reported willingness to pay was greater than the price, they would receive the masks; otherwise, they would receive the gift card.Therefore, it was in participants' best interest to report their true willingness to pay for the masks.This type of procedure has been shown to lead to truthful reporting. 8 collected data on 3 secondary outcomes specified in our pre-analysis plan (Supplement 2).
First, we asked participants to report their judgment of how well federal and state policies balanced opening the economy and limiting the health impacts of COVID-19.Second, we measured how participants prioritized COVID-19 protection vs other issues by asking the participants how they would want to allocate a donation of $1000 (which the research team would fund) between 2

Randomization
Randomization into intervention vs control and between interventions was stratified according to sex, age (Ն45 years), race, and self-identified partisan affiliation (Republican vs other affiliation).The flowchart (Figure 1) summarizes the randomization; a more detailed version is available as eFigure 1 in Supplement 1.
Participants were first randomized into the AMA antiracism or placebo statements, with equal probability.They were then randomly assigned to intervention or control.One of 5 participants was assigned to control; the remainder were assigned to an intervention.
Within each group, participants were randomized into Black or White physician groups with equal probability.Intervention participants were further randomized, with equal probability, into 1 the 2 arms for video 2: they either received the information about the unequal burden of disease or did not.Randomization was performed using the Qualtrics platform, using a randomizer block within each stratum with the option to evenly present elements.

Statistical Analysis
We determined that a sample of 20 000 individuals (10 000 Black and 10 000 White) would provide 85% power to detect effect sizes of 0.05 SDs for intervention relative to control and for effects of specific variations in message content.These are small effect sizes that would justify scale up of this inexpensive intervention.
The analysis was performed by original assigned group, and it included all participants who completed the survey.Multivariable regression models include the stratifying variables (age × sex × race × Republican identity dummies).

Effect of Any Video Message Intervention Relative to Control
To analyze the effect of seeing any video message on the knowledge gap, information-seeking behavior, and safety behavior outcomes, we fit the following negative binomial regression model for the count outcome: where μ i is the estimated mean outcome value (knowledge gap count, count of demanded links, or safety behavior count), intervention i is an indicator that equals 1 if the individual received the intervention videos and 0 if they received the placebo videos, and stratum i is a vector of indicator variables.Similar models are estimated for binary regressions (using the logistic regression equation) and continuous variable (using ordinary least squares) (eAppendix 2 in Supplement 1).Because there were multiple outcomes, we provide P values and q values adjusted for false discovery rates.

Effect of Variation in the Message Framing
To analyze the impact of different arms, we fit a negative binomial regression model to the count data: where Black physician i is an indicator that equals 1 if the physician was a Black individual and 0 otherwise; AMA antiracism i is an indicator for the AMA statement featuring the antiracism message (rather than the drug pricing message); mortality difference i is an indicator for video 2 mentioning To address possible bias stemming from nonrandom attrition for the follow-up survey, we weighted the follow-up data using Hainmueller entropy weights, 9 which ensures that the observed baseline characteristics of the follow-up sample matches the original sample as closely as possible (eAppendix 2 in Supplement 1).eAppendix 3 in Supplement 1 describes further robustness checks and subgroups analysis.

JAMA Network Open | Infectious Diseases
Analyses were performed using R version 4.0.3(R Project for Statistical Computing).Statistical significance was set at P < .05,and all tests were 2-tailed.

Trial Sample
The trial sample was enrolled from August 7 through September b The preventive practices variables refer to the question: "What fraction of the time would you say that you engage in the following behaviors?"Mask in (always) is equal to 1 if the respondent answered "always" to "Wearing a mask when you go inside buildings that are not your home / take public transportation," otherwise it is 0. Mask out (always) is equal to 1 if the respondent answered "always" to "Wearing a mask outside," otherwise it is 0. Wash hands (always) is equal to 1 if the respondent answered "always" to "Washing your hands with soap and water right away when you come home after going out." Distance (always) is equal to 1 if the respondent answered "always" to "Staying at least 6 feet away from people who are not part of your household."Incidence rate for knowledge gaps is the count of knowledge gaps divided by the maximum possible count (10).
Incidence rate for interest in links is the count of links demanded divided by the maximum possible count (5).
Incidence rate for safety gaps is the count of safety gaps divided by the maximum possible count (4).IRR (or coefficients) compare the any intervention with the control group.IRRs for safety gap score were estimated by fitting a negative binomial regression model with units weighted following Hainmueller entropy-based weighting 9 to account for imbalances due to attrition for the follow-up outcomes.q values are reported accounting for the different outcomes and coefficients in each panel. b The F statistic for a test equality of the coefficients for the Black participants and White participants (obtained by estimating all outcomes in a joint system) was 0.0112 (P > .99).The test statistics for the hypothesis that all the interaction coefficients are jointly 0 across equations was 0.324 (P > .99).Estimates in each row came from a single negative binomial regression (or ordinary least squares regression for WTP masks) following the second equation in the main text.IRRs for follow-up outcomes were calculated from estimates obtained by fitting a negative binomial regression model with units reweighted following Hainmueller entropy-based reweighting 9 to account for imbalances due to attrition.

Discussion
Exposure to public health video messages about COVID-19 recorded by a diverse set of physicians decreased knowledge gaps on COVID-19 symptoms, preventive behaviors, and transmission among Black and White participants with modest incomes, relative to a control condition that saw placebo videos.The effect on knowledge was substantial and clear.This replicates the results of our prior study conducted in May 2020 3 and extends it to White participants.New to this study, we also found a modest but statistically significant increase in the demand for more information, the willingness to pay for high quality masks, and self-reported behavior at follow-up.
Despite the heightened awareness of racial justice issues during the period of this intervention and increased polarization in the political discourse in the run-up to the presidential election, effects are remarkably similar across racial and political lines.These results suggest that physicians still have the ability to inform and persuade members of society from a broad range of backgrounds.
Our results also indicate that tailoring the message to specific communities did not affect its impact on behavior.Both White and Black physicians were able to effectively convey the importance of masking and social distancing to Black and White participants (unlike the previous study, in which concordance was important to change behavior 3 ).The AMA antiracism statement did not affect participants' attentiveness to the message delivered, for Black or White respondents.
Acknowledgment of structural racism remains important, but it may not be sufficient to increase the level of trust from the Black community.
Importantly, the intervention made both Black and White participants more willing to focus resources both toward COVID-19 in general and toward the Black community in particular.
Highlighting health conditions that disproportionately affect the Black community is one step toward increasing public consciousness of structural racism.

Limitations
There are several limitations of the study.First, it was conducted online, and the participants may not be representative of the population with less than a college degree, given that they have access to the internet and are used to participating in online studies.Second, although information-seeking behavior and willingness to pay for masks were objectively measured, participants' preventive health behaviors were not directly observed.Outcomes were self-reported.Third, outcomes might be subject to social desirability bias.Fourth, there may be bias due to attrition, particularly for the selfreported safety behavior, given that only a small fraction of the sample could be followed up a few days after the initial intervention.While the observable variables remain balanced, the unobservable may not be.Furthermore, while we found consistent effects on knowledge, information seeking, the willingness to pay for masks, and self-reported behavior, the final clinical significance of these findings is uncertain because effects on all were quantitatively small.

Conclusions
These results suggest physician messaging campaigns may be effective and trust in Black and White physicians is equally high.There is no evidence of preexisting bias that would have led the intervention to have a negative effect.Because it is inexpensive, it could be a promising way to encourage behavior at scale.However, future studies implemented at a large scale are needed to confirm whether these kinds of interventions can change behavior in a way that will affect clinical outcomes.In ongoing work, we will study scale up messaging by doctors using social media.

eAppendix 2 in
Supplement 1 describes the outcome measurement in detail.Primary outcomes presented in the main text include 5 outcomes.First, knowledge gap, which measures knowledge and beliefs.Participants were asked to identify ways to prevent COVID-19 spread and identify 4 common symptoms.The knowledge gap outcome is an integer that can have values from 0 (no error) Effect of Physician-Delivered Messages on Adults' COVID-19 Knowledge, Beliefs, and Practices JAMA Network Open.2021;4(7):e2117115.doi:10.1001/jamanetworkopen.2021.17115(Reprinted) July 14, 2021 2/13 Downloaded From: https://jamanetwork.com/ on 09/23/2023 community to the tailoring of those messages.Latinx individuals were included in our previous study, 3 6long with specific tailoring toward this community.Recruitment used quotas to match the 2018 population estimates by age, sex, and race issued by the US Census Bureau.6 Effect of Physician-Delivered Messages on Adults' COVID-19 Knowledge, Beliefs, and Practices charities: Give a Mask or the Alzheimer Association.Third, we asked whether they would prioritize a COVID-19 relief donation to a Black-focused charity (the BET COVID-19 relief fund) or a general charity (the Give Directly Project 100+ relief program).

Table 1 )
6, 2020.Of 44 743 screened, 30 174were eligible for participation, 5534 individuals did not consent or failed 2 simple attention checks, 4163 left the survey before randomization, and 17 were excluded from the analysis due to unknown race or multiple survey completion.The final sample at randomization had 20 460 individuals, for a participation rate of 68%.After attrition, 18 223 individuals were included in the study.were computed on the sample that was randomized and that

Table 1 .
Summary of Participant Characteristics a a This table presents summary statistics on baseline variables for our main sample of individuals who completed all baseline variables.

Table 2 .
Impact of Any Video Message Intervention vs Control: Incidence Rate a a

Table 3 .
Impact of Tailoring Messages: Incidence Rate Ratios a a

SUPPLEMENT 2. Pre-analysis Plan and Trial Protocol SUPPLEMENT 3. Nonauthor Contributors SUPPLEMENT 4. Data Sharing Statement
Distribution of the Safety Gap Score in the Control and Intervention Groups eTable 1. Balance and Attrition eTable 2. Outcomes by Subgroup eTable 3. Effects of Any Message Intervention: Effects on Additional Outcomes eTable 4. Effects of Tailoring Messages on Additional Outcomes eTable 5. Effects of All Black Treatments on Outcomes Diseases SUPPLEMENT 1. eAppendix 1. Survey Design and Videos eAppendix 2. Supplementary Methods eAppendix 3. Robustness Checks and Subgroup Analysis eFigure 1. Full Study Flowchart eFigure 2.