Consumer Views on Using Digital Data for COVID-19 Control in the United States | Public Health | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Approval for COVID-19 Digital Data Use Scenarios
Approval for COVID-19 Digital Data Use Scenarios

The weighted distribution of responses to the 9 COVID-19 digital data use scenarios is shown.

Figure 2.  Support for Apple-Google Contact Tracing Program
Support for Apple-Google Contact Tracing Program

Adjusted probabilities of support for 3 possible variations of the Apple-Google digital contact tracing program are shown.

Table 1.  Characteristics of the 3389 Participants in the Survey Studya
Characteristics of the 3389 Participants in the Survey Studya
Table 2.  Factors Associated With Support for 9 Public Health Digital Data Use Scenarios
Factors Associated With Support for 9 Public Health Digital Data Use Scenarios
Table 3.  Factor Analysis for COVID-19 Public Health Digital Data Use Scenariosa
Factor Analysis for COVID-19 Public Health Digital Data Use Scenariosa
1.
Tau  B. Government tracking how people move around in coronavirus pandemic. Wall Street Journal. Published March 28, 2020. Accessed March 29, 2020. https://www.wsj.com/articles/government-tracking-how-people-move-around-in-coronavirus-pandemic-11585393202
2.
Wang  CJ, Ng  CY, Brook  RH.  Response to COVID-19 in Taiwan: big data analytics, new technology, and proactive testing.   JAMA. 2020;323(14):1341-1342. doi:10.1001/jama.2020.3151PubMedGoogle ScholarCrossref
3.
Onishi  N, Méheut  C. France weighs its love of liberty in fight against coronavirus. New York Times. Published April 17, 2020. Accessed April 19, 2020. https://www.nytimes.com/2020/04/17/world/europe/coronavirus-france-digital-tracking.html
4.
Mozur  P, Zhong  R, Krolik  A. In coronavirus fight, China gives citizens a color code, with red flags. New York Times. Published March 2, 2020. Accessed April 19, 2020. https://www.nytimes.com/2020/03/01/business/china-coronavirus-surveillance.html
5.
Sonn  JW. Coronavirus: South Korea’s success in controlling disease is due to its acceptance of surveillance. The Conversation. Published March 19, 2020. Accessed April 16, 2020. https://theconversation.com/coronavirus-south-koreas-success-in-controlling-disease-is-due-to-its-acceptance-of-surveillance-134068
6.
McNeil DG Jr. Can smart thermometers track the spread of the coronavirus? New York Times. Published March 18, 2020. Accessed April 16, 2020. https://www.nytimes.com/2020/03/18/health/coronavirus-fever-thermometers.html
7.
Moreland  A, Herlihy  C, Tynan  MA,  et al; CDC Public Health Law Program; CDC COVID-19 Response Team, Mitigation Policy Analysis Unit.  Timing of state and territorial COVID-19 stay-at-home orders and changes in population movement—United States, March 1-May 31, 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(35):1198-1203. doi:10.15585/mmwr.mm6935a2PubMedGoogle ScholarCrossref
8.
Couture  V, Dingel  JI, Green  AE, Handbury  J, Williams  KR. Measuring movement and social contact with smartphone data: a real-time application to COVID-19. National Bureau of Economic Research Working Paper 27560. Published July 2020. Accessed March 23, 2020. https://www.nber.org/system/files/working_papers/w27560/w27560.pdf
9.
Schechner S, Winkler R. Here’s how Apple and Google plan to track the coronavirus through your phone. Wall Street Journal. Published April 11, 2020. Accessed April 13, 2020. https://www.wsj.com/articles/heres-how-apple-and-google-plan-to-track-the-coronavirus-through-your-phone-11586618075
10.
Bengio  Y, Janda  R, Yu  YW,  et al.  The need for privacy with public digital contact tracing during the COVID-19 pandemic.   Lancet Digit Health. 2020;2(7):e342-e344. doi:10.1016/S2589-7500(20)30133-3PubMedGoogle ScholarCrossref
11.
Mehta  SJ, Troxel  AB, Marcus  N,  et al.  Participation rates with opt-out enrollment in a remote monitoring intervention for patients with myocardial infarction.   JAMA Cardiol. 2016;1(7):847-848. doi:10.1001/jamacardio.2016.2374PubMedGoogle ScholarCrossref
12.
Hinch  R, Probert  W, Nurtay  A,  et al. Effective configurations of a digital contact tracing app: a report to NHSX. Accessed May 2, 2020. https://cdn.theconversation.com/static_files/files/1009/Report_-_Effective_App_Configurations.pdf?1587531217
13.
Ferretti  L, Wymant  C, Kendall  M,  et al.  Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing.   Science. 2020;368(6491):eabb6936. doi:10.1126/science.abb6936PubMedGoogle Scholar
14.
Ting  DSW, Carin  L, Dzau  V, Wong  TY.  Digital technology and COVID-19.   Nat Med. 2020;26(4):459-461. doi:10.1038/s41591-020-0824-5PubMedGoogle ScholarCrossref
15.
Cho  H, Ippolito  D, Yu  YW. Contact tracing mobile apps for COVID-19: privacy considerations and related trade-offs. arXiv. Preprint posted online March 30, 2020. arXiv:2003.11511v2
16.
Valentino-DeVries  J, Singer  N, Krolik  A. A scramble for virus apps that do no harm. New York Times. Published April 29, 2020. Accessed April 30, 2020. https://www.nytimes.com/2020/04/29/business/coronavirus-cellphone-apps-contact-tracing.html
17.
Ienca  M, Vayena  E.  On the responsible use of digital data to tackle the COVID-19 pandemic.   Nat Med. 2020;26(4):463-464. doi:10.1038/s41591-020-0832-5PubMedGoogle ScholarCrossref
18.
Lopez L, Hamel L. Public attitudes towards the coronavirus outbreak in highly affected counties. Kaiser Family Foundation. Published April 29, 2020. Accessed May 11, 2020. https://www.kff.org/policy-watch/public-attitudes-towards-coronavirus-outbreak-highly-affected-counties/
19.
Ipsos. KnowledgePanel: a methodological overview. Accessed November 4, 2020. https://www.ipsos.com/sites/default/files/ipsosknowledgepanelmethodology.pdf
20.
Dimitropoulos L, Patel V, Scheffler SA, Posnack S. Public attitudes toward health information exchange: perceived benefits and concerns. Am J Manag Care. 2011;17(12):SP111-SP116.
21.
Patel  VN, Dhopeshwarkar  RV, Edwards  A,  et al.  Low-income, ethnically diverse consumers’ perspective on health information exchange and personal health records.   Inform Health Soc Care. 2011;36(4):233-252.doi:10.3109/17538157.2011.554930PubMedGoogle ScholarCrossref
22.
US Centers for Disease Control and Prevention. Health equity considerations and racial and ethnic minority groups. Published April 30, 2020. Updated February 12, 2021. Accessed February 13, 2021. https://www.cdc.gov/coronavirus/2019-ncov/community/health-equity/race-ethnicity.html
23.
Alcendor  DJ.  Racial disparities-associated COVID-19 mortality among minority populations in the US.   J Clin Med. 2020;9(8):2442. doi:10.3390/jcm9082442PubMedGoogle ScholarCrossref
24.
Buckee  CO, Balsari  S, Chan  J,  et al.  Aggregated mobility data could help fight COVID-19.   Science. 2020;368(6487):145-146. doi:10.1126/science.abb8021PubMedGoogle Scholar
25.
Chen  E, Lerman  K, Ferrara  E.  Tracking social media discourse about the COVID-19 pandemic: development of a public coronavirus Twitter data set.   JMIR Public Health Surveill. 2020;6(2):e19273. doi:10.2196/19273PubMedGoogle Scholar
26.
Open Government Partnership. Collecting open government approaches to COVID-19. Accessed April 16, 2020. https://www.opengovpartnership.org/collecting-open-government-approaches-to-covid-19/
27.
Callegaro M, DiSogra C.  Computing response metrics for online panels.   Public Opin Q. 2008;72(5):1008–1032. Google ScholarCrossref
28.
Auxier  B. How Americans see digital privacy issues amid the COVID-19 outbreak. Pew Research Center. Published May 4, 2020. Accessed November 4, 2020. https://www.pewresearch.org/fact-tank/2020/05/04/how-americans-see-digital-privacy-issues-amid-the-covid-19-outbreak/
29.
Beidas  RS, Buttenheim  AM, Feuerstein-Simon  R,  et al.  Optimizing and implementing contact tracing through behavioral economics.   NEJM Catal. Published online June 23, 2020.Google Scholar
30.
Patel  MS, Volpp  KG, Asch  DA.  Nudge units to improve the delivery of health care.   N Engl J Med. 2018;378(3):214-216. doi:10.1056/NEJMp1712984PubMedGoogle ScholarCrossref
31.
Finkle  J. Solid support for Apple in iPhone encryption fight: poll. Reuters. Published February 24, 2016. Accessed December 5, 2020. https://www.reuters.com/article/us-apple-encryption-poll-idUSKCN0VX159
32.
Turow  J, Hennessy  M, Draper  N, Ankabi  O, Virgilio D. Divided we feel: partisan politics drive Americans’ emotions regarding surveillance of low-income populations. Annenberg School for Communication, University of Pennsylvania. Published 2018. Accessed April 3, 2020. https://repository.upenn.edu/asc_papers/543/
33.
Doherty  C. Balancing act: national security and civil liberties in post-9/11 era. Pew Research Center. Published June 7, 2013. Accessed April 20, 2020. https://www.pewresearch.org/fact-tank/2013/06/07/balancing-act-national-security-and-civil-liberties-in-post-911-era/
34.
Calvillo  DP, Ross  BJ, Garcia  RJB, Smelter  TJ, Rutchick  AM.  Political ideology predicts perceptions of the threat of COVID-19 (and susceptibility to fake news about it).   Soc Psychol Personal Sci. 2020;11(8):1119-1128. doi:10.1177/1948550620940539Google ScholarCrossref
35.
Hamel  L, Kearney  A, Kirzinger  A, Lopes  L, Muñana  C, Brodie  M. KFF health tracking poll—May 2020. Kaiser Family Foundation. Published May 27, 2020. Accessed June 2, 2020. https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-may-2020/
36.
Madden  M. Privacy, security, and digital inequality: how technology experiences and resources vary by socioeconomic status, race, and ethnicity. Published September 27, 2017. Accessed April 12, 2020. https://datasociety.net/pubs/prv/DataAndSociety_PrivacySecurityandDigitalInequality.pdf
37.
Fareed  N, Swoboda  CM, Jonnalagadda  P, Walker  DM, Huerta  TR. Differences between races in health information seeking and trust over time: evidence from a cross-sectional, pooled analyses of HINTS data. Am J Health Promot. 2020;35(1):84-92. doi:10.1177/0890117120934609
38.
Auxier  B, Rainie  L, Anderson  M, Perrin  A, Kumar  M, Turner  E. Americans and privacy: concerned, confused and feeling lack of control over their personal information. Pew Research Center: Internet & Technology. Published November 15, 2019. Accessed December 5, 2020. https://www.pewresearch.org/internet/2019/11/15/americans-and-privacy-concerned-confused-and-feeling-lack-of-control-over-their-personal-information/
39.
Grande  D, Luna Marti  X, Feuerstein-Simon  R,  et al.  Health policy and privacy challenges associated with digital technology.   JAMA Netw Open. 2020;3(7):e208285. doi:10.1001/jamanetworkopen.2020.8285PubMedGoogle Scholar
40.
Draper  NA, Turow  J.  The corporate cultivation of digital resignation.   New Media Soc. 2019;21(8):1824-1839. doi:10.1177/1461444819833331Google ScholarCrossref
41.
Veale  M. Privacy is not the problem with the Apple-Google contact-tracing toolkit. The Guardian. Published July 1, 2020. Accessed August 4, 2020. https://www.theguardian.com/commentisfree/2020/jul/01/apple-google-contact-tracing-app-tech-giant-digital-rights
42.
Bradford  L, Aboy  M, Liddell  K.  COVID-19 contact tracing apps: a stress test for privacy, the GDPR, and data protection regimes.   J Law Biosci. 2020;7(1):Isaa034. doi:10.1093/jlb/lsaa034PubMedGoogle ScholarCrossref
43.
Cartmell  KB, Mzik  CR, Sundstrom  BL, Luque  JS, White  A, Young-Pierce  J.  HPV vaccination communication messages, messengers, and messaging strategies.   J Cancer Educ. 2019;34(5):1014-1023. doi:10.1007/s13187-018-1405-xPubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 3,595
    Citations 0
    Original Investigation
    Public Health
    May 19, 2021

    Consumer Views on Using Digital Data for COVID-19 Control in the United States

    Author Affiliations
    • 1Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia
    • 2Division of General Internal Medicine, University of Pennsylvania, Philadelphia
    • 3Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia
    • 4Department of Emergency Medicine, University of Pennsylvania, Philadelphia
    • 5Center for Digital Health, University of Pennsylvania, Philadelphia
    • 6Penn Medicine Center for Health Care Innovation, University of Pennsylvania, Philadelphia
    • 7Department of Family Medicine and Community Health, University of Pennsylvania, Philadelphia
    JAMA Netw Open. 2021;4(5):e2110918. doi:10.1001/jamanetworkopen.2021.10918
    Key Points

    Question  What uses of consumer digital information for COVID-19 control are most acceptable to US adults, and what factors are associated with higher or lower approval of use of this information?

    Findings  In this cross-sectional survey study of 6284 US adults, approval was generally low for use of consumer digital data for activities such as case identification, digital contact tracing, policy setting, and enforcing quarantines. Political ideology and race/ethnicity were associated with approval for scenarios in which digital data were used, whereas local COVID-19 incidence and family experience with COVID were not.

    Meaning  The findings suggest that in current and future pandemics, public health departments should use multiple strategies to gain public trust and accelerate adoption of tools such as digital contact tracing applications.

    Abstract

    Importance  Curbing COVID-19 transmission is currently the greatest global public health challenge. Consumer digital tools used to collect data, such as the Apple-Google digital contact tracing program, offer opportunities to reduce COVID-19 transmission but introduce privacy concerns.

    Objective  To assess uses of consumer digital information for COVID-19 control that US adults find acceptable and the factors associated with higher or lower approval of use of this information.

    Design, Setting, and Participants  This cross-sectional survey study obtained data from a nationally representative sample of 6284 US adults recruited by email from the web-based Ipsos KnowledgePanel in July 2020. Respondents evaluated scenarios reflecting uses of digital data for COVID-19 control (case identification, digital contact tracing, policy setting, and enforcement of quarantines).

    Main Outcomes and Measures  Levels of support for use of personal digital data in 9 scenarios to mitigate the spread of COVID-19 infection, rated on a Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Multivariable linear regression models were fitted for each scenario and included factors hypothesized to be associated with views about digital data use for COVID-19 mitigation measures. Black and Hispanic survey respondents were oversampled; thus, poststratification weights were used so that results are representative of the general US population.

    Results  Of 6284 individuals invited to participate in the study, 3547 responded, for a completion rate of 56%. A total of 1762 participants (52%) were female, 715 (21%) identified as Black, 790 (23%) identified as Hispanic, and 1224 (36%) were 60 years or older; mean (SD) age was 51.7 (16.6) years. Approval of scenarios was low, ranging from 28% to 43% (52%-67% when neutral responses were included). Differences were found based on digital data source (smartphone vs social media: coefficient, 0.29 [95% CI, 0.23-0.35]; P < .001; smart thermometer vs social media: coefficient, 0.09 [95% CI, 0.03-0.16]; P = .004). County COVID-19 rates (coefficient, −0.02; 95% CI, −0.16 to 0.13 for quartile 4 compared with quartile 1) and prior family diagnosis of COVID-19 (coefficient, 0.00; 95% CI, −0.25 to 0.25) were not associated with support. Compared with self-described liberal individuals, conservative (coefficient, −0.81; 95% CI, −0.96 to −0.66; P < .001) and moderate (coefficient, −0.52; 95% CI, −0.67 to −0.38; P < .001) individuals were less likely to support the scenarios. Similarly, large political differences were observed in support of the Apple-Google digital contact tracing program, with less support from conservative (coefficient, −0.99; 95% CI, −1.11 to −0.87; P < .001) and moderate (coefficient, −0.59; 95% CI, −0.69 to −0.48; P < .001) individuals compared with liberal individuals. Respondents from racial/ethnic minority groups were more supportive of the scenarios than were White, non-Hispanic respondents. For example, compared with White respondents, Black respondents were more supportive of the Apple-Google contact tracing program (coefficient, 0.20; 95% CI, 0.07-0.32; P = .002).

    Conclusions and Relevance  In this survey study of US adults, many were averse to their information being used on digital platforms to mitigate transmission of COVID-19. These findings suggest that in current and future pandemics, public health departments should use multiple strategies to gain public trust and accelerate adoption of tools such as digital contact tracing applications.

    Introduction

    The COVID-19 pandemic has prompted governments around the world to use digital information in novel ways as a public health intervention to decrease the transmission of disease.1-3 The Chinese government tracked population movement around the country with closed circuit television and geolocating wristbands.4 South Korea quickly brought the COVID-19 outbreak under control through a rapid testing campaign driven by data from credit card transactions, smartphone location tracking, and closed circuit television.5 In the US, internet-connected devices such as smart thermometers have been used to track the geographic distribution of influenza-like symptoms, and smartphones have been used to measure population mobility in response to various public health restrictions.6-8

    More recently, public health departments have used new functionality in the Apple and Google smartphone operating systems to deploy digital contact tracing programs. Bluetooth measures proximity to other smartphones and alerts program users if they have come into close contact with a person who tested positive for SARS-CoV-2.9 Apple and Google have prioritized privacy; data sharing with public health agencies is opt in, meaning that the consumer must choose to turn it on. There is concern that these privacy protections could lead to low rates of participation, which could render the program ineffective.10-12

    The COVID-19 pandemic has led to the initiation of other public health–related uses of consumer digital information focused on disease surveillance, contact tracing, and enforcement of measures such as quarantines.13,14 Use of digital information for COVID-19 response has raised new ethical considerations and questions about how consumers view the tradeoff between public health and individual privacy. On one hand, digital technology has the potential to markedly accelerate public health measures that can decrease chains of disease transmission. On the other hand, privacy advocates have questioned how personal data will be protected, retained, and deidentified.15,16 In the process of designing programs and policies, debates have often focused on whether participation should be voluntary (ie, opt in vs opt out), whether data should be reported and acted on at the individual or population level, and whether digital data should be used for enforcement of public health measures.17 Some data use, such as the Apple-Google Bluetooth-enabled digital contact tracing program, requires a high level of participation to be effective. Polling in April 202018 by the Kaiser Family Foundation showed that 45% of respondents were willing to download and use a contact tracing application, and individuals in areas with higher COVID-19 rates were more willing to do so than were those in areas with lower rates.

    In this nationally representative survey study, we assessed consumers’ attitudes about 9 use cases for personal digital information to mitigate the impact of the COVID-19 pandemic. We included versions of the Apple-Google digital contact tracing program that is currently being deployed in many US states. We hypothesized that views would vary based on several factors, including political ideology (given partisan debates about COVID-19 policies), the potential for personal and community benefit (local rates, belonging to a population disproportionately affected by COVID-19), a family history of COVID-19, and whether use cases were written to describe individual-level or population-level data use. Understanding consumer views on digital privacy is critical as the world confronts the immediate challenges of the COVID-19 pandemic and considers preparedness for future pandemics.

    Methods
    Participants

    This cross-sectional survey study used responses to a survey administered from July 10 through July 31, 2020. Participants were recruited from the web-enabled Ipsos KnowledgePanel, a probability-based panel designed to be representative of the US population.19 At the time they joined the panel, respondents completed a general informed consent process through an online survey during the KnowledgePanel recruitment process. All data received by the study team were deidentified. Ipsos KnowledgePanel provided the study team with the following measures reflective of the recruitment of individuals into the panel: 11.5% average recruitment rate, 59.5% average household profile completion rate, and 36.2% average household retention rate. This study was reviewed and deemed exempt by the institutional review board at the University of Pennsylvania based on the minimal risk of the research and use of deidentified data. This study followed the American Association for Public Opinion Research (AAPOR) reporting guideline.

    Race/ethnicity was assessed in this study because prior evidence has shown elevated concern regarding digital health privacy among historically oppressed racial/ethnic groups.20,21 When participants joined the panel, they were asked to complete a core survey profile, which included questions about their racial/ethnic background using the US Census Bureau categories and questions. In addition, we oversampled Black and Hispanic participants given the disproportionate effects of COVID-19 in these populations to allow for comparisons by race/ethnicity.22,23 The survey was administered in both Spanish and English.

    Survey Development and Administration

    This study reports responses to scenarios involving the use of consumer digital information to mitigate the spread of COVID-19. The scenarios were developed based on actual or possible data applications described in academic literature, the lay press, and white papers.6,24-26 The scenarios varied in 3 general dimensions: the source of digital data (social media, smart thermometer, or smartphone), the sharing of data with public health officials, and the particular use. The scenarios are described in eAppendix 1 in the Supplement and summarized in the following paragraph.

    Scenarios 1 and 2 described using social media data (online chatter) to monitor for COVID-19–related symptoms. Scenario 1 described using aggregate data to set public health policies. Scenario 2 described using the data to contact symptomatic individuals for COVID-19 testing. Scenarios 3 and 4 were analogous to scenarios 1 and 2, but the data source was body temperature readings from smart thermometers. Scenarios 5, 6, and 7 described versions of the Apple-Google digital contact tracing program (scenario 5: opt in, optional data sharing with public health agencies; scenario 6: opt in, automatic data sharing with public health agencies; scenario 7: automatic participation, automatic data sharing with public health agencies). The Apple-Google digital contact tracing program, actively deployed by many health departments, uses Bluetooth technology to measure exposure (time/distance) to other people. If an individual has a positive test result, the program will notify individuals enrolled in the program who are found to be close contacts of the COVID-19–positive individual. To date, the program is voluntary and does not share data with public health authorities. Scenario 8 described using smartphones to enforce quarantines. Scenario 9 described developing COVID-19 risk profiles on smartphones that could be used to restrict certain activities.

    We randomly assigned participants to receive questions framed at the individual level vs questions framed at the population level to assess whether individual-level saliency influenced responses. Those in the individual-level framing arm of the study responded to scenarios emphasizing use of their own data (eg, “your” data). Those in the population-level framing arm responded to scenarios emphasizing the use of data more generally (eg, “people’s” data). We hypothesized that support for scenarios would be greater for those in the population-level framing arm because the individual-level framing arm would highlight individual privacy concerns.

    For each scenario, participants rated their approval on a Likert scale with the following options: strongly disagree (1), disagree (2), neither agree nor disagree (3), agree (4), or strongly agree (5). The content of other questions included self-reported political ideology and overall health status and whether the respondent or a family member had been diagnosed with COVID-19. Sociodemographic data were previously collected by Ipsos and shared with the study team.

    Statistical Analysis

    We report study-specific survey completion rates as recommended for online probability samples. Panel recruitment and retention rates were reported previously.27 Descriptive statistics, including means and 95% CIs, were calculated for each scenario. Because we oversampled Black and Hispanic survey respondents, poststratification weights were used so that results are representative of the general US population. Of the survey respondents, 158 (4%) had incomplete data. All variables used to develop poststratification weights were complete because they were collected at the time of participants’ entry to the panel. All other variables (COVID-19 scenarios, political ideology, and personal or family history of COVID-19) were more than 98% complete. We used multiple imputation by chained equations and compared the results with results of a complete case analysis and those of an extreme case analysis. The results were nearly identical (eAppendixes 2 and 3 in the Supplement).

    We fit multivariable linear regression models for each scenario. We included factors that we hypothesized would influence views about digital data use for COVID-19 mitigation measures: political ideology and geography, prior personal or family experience with COVID-19, sociodemographic factors associated with COVID-19 risk (race/ethnicity, household income), and county-level COVID-19 rates. We retrieved COVID-19 case counts and rates from usafacts.org for a 2-week period during survey administration (July 8-22, 2020) and linked them to a respondent’s county of residence.

    We then conducted a factor analysis to group similar scenarios for data reduction and better interpretability. Factor loadings revealed 3 factors among the 9 items with eigenvalues above 1 (P < .001): (1) social media data for identifying cases (scenarios 1 and 2); (2) smart thermometer data for identifying cases (scenarios 3 and 4); (3) smartphone data for contact tracing, quarantining, and risk-stratifying individuals (scenarios 5-9). We analyzed the 3 factors together using a multivariable generalized estimating equation model to account for correlation of responses from a participant. We assumed a linear mean model, independent working correlation structure, and empirical SEs. The final model included these 3 factors along with the experimental arm from the framing experiment (individual-level vs population-level framing) and the covariates described above from scenario-specific models. Stata Statistical Software, version 16.1 (StataCorp LLC) was used for all corresponding analyses. A 2-tailed P = .05 was considered to be statistically significant.

    Results

    Of 6284 recruited individuals, 3547 responded to the survey, representing a completion rate of 56%. This included 158 individuals who did not respond to all 9 scenarios but whose data were imputed and therefore included in multivariable analyses. Table 1 describes the characteristics of the survey respondents. A total of 1762 participants (52%) were female, 715 (21%) identified as Black, 790 (23%) identified as Hispanic, and 1224 (36%) were 60 years or older; mean (SD) age was 51.7 (16.6) years. A total of 960 individuals (28%) were non-White, reflecting oversampling of minority populations.

    Support for Public Health Digital Data Use Scenarios

    Overall, 28% to 43% of the respondents agreed or strongly agreed with any of the digital data use scenarios described for COVID-19 mitigation. Figure 1 shows the response distribution for each of the 9 scenarios. Large political differences were observed in support for the Apple-Google digital contact tracing program, with less support from conservative (coefficient, −0.99; 95% CI, −1.11 to −0.87; P < .001) and moderate (coefficient, −0.59; 95% CI, −0.69 to −0.48; P < .001) individuals as compared with liberal individuals. Support was greatest for use of smartphones for digital contact tracing with and without public health agency involvement (43% and 40%, respectively) and for quarantine enforcement (38%). Support was lowest for monitoring social media chatter to set public health policies (28%) or contact individuals for testing (31%), making the Apple-Google digital contact tracing program mandatory (30%), and using smartphones to create COVID-19 risk profiles (31%). When agree and neutral responses were combined, the overall proportion ranged from 52% to 67%.

    In multivariable models for the 9 scenarios (Table 2), consistent associations were found between self-described political ideology and approval of the scenarios. Compared with liberal individuals, moderate and conservative individuals were less likely to support all 9 scenarios. Personal experience with COVID-19 (ie, personal infection or infection of a family member) and higher local COVID-19 incidence rates were generally not associated with support. Respondents in racial/ethnic minority groups expressed greater support for all the scenarios with the exception of the smart thermometer scenarios and the smartphone scenario that did not involve formal public health contact tracing (no significant differences were found between White and Black respondents for these scenarios). However, compared with White respondents, Black respondents were more supportive of the Apple-Google contact tracing program (coefficient, 0.20; 95% CI, 0.07-0.32; P = .002). A consistent association with age across the scenarios was not observed, although older adults (age ≥60 years) expressed greater support than younger adults for the Apple-Google digital contact tracing program (coefficient, 0.28; 95% CI, 0.14-0.43; P < .001).

    Factors Associated With Overall Support for Public Health Digital Data Use

    We also assessed all scenarios together using a single model (Table 3). The model included 3 factors identified through factor analysis, representing the 9 scenarios: cluster 1 (scenarios 1 and 2: social media data), cluster 2 (scenarios 3 and 4: smart thermometer data), and cluster 3 (scenarios 5 and 9: smartphone data). Coefficients from the model represent differences on a scale of 1 to 5, where 1 indicates strongly disagree and 5, strongly agree; positive coefficients represent greater support. Support was greater for smartphone data (coefficient, 0.29; 95% CI, 0.23-0.35; P < .001) and smart thermometer data (coefficient, 0.09; 95% CI, 0.03-0.16; P = .004) compared with social media data. Second, there were significant differences by political ideology; compared with liberal individuals, support for the scenarios overall was lower among conservative (coefficient, −0.81; 95% CI, −0.96 to −0.66; P < .001) and moderate (coefficient, −0.52; 95% CI, −0.67 to −0.38; P < .001) individuals. These political differences were consistent among the 3 clusters of scenarios, although conservative individuals were less likely to support use of smartphone data (coefficient, −0.15; 95% CI, −0.23 to −0.07; P < .001) than were liberal individuals. Members of racial/ethnic minority groups expressed greater support compared with nonminority respondents (Black respondents vs White respondents: coefficient, 0.22; 95% CI, 0.12-0.32; Hispanic respondents vs non-Hispanic respondents: coefficient, 0.38; 95% CI, 0.28-0.48; P < .001 for both). Prior personal or family experience with COVID-19 (coefficient, 0.00; 95% CI, −0.25 to 0.25) and local COVID-19 incidence rates (quartile 4 vs quartile 1: coefficient, −0.02; 95% CI, −0.16 to 0.13) were not associated with support for data use.

    In this model, we also evaluated the effect of the framing experiment embedded in the survey design. There were no statistically significant differences in responses between those randomly assigned to scenarios that were written to frame data use at the individual level vs the population level (“your data” vs “people’s data”). However, some differences were associated with political ideology. Moderates responded more favorably to scenarios that used population-level vs individual-level framing (coefficient, 0.21; 95% CI, 0.02-0.39; P = .03).

    Support for the Apple-Google Digital Contact Tracing Program

    In our survey, 40% of respondents supported the Apple-Google digital contact tracing program. When we described making data sharing with public health departments a standard part of the program, we observed no decrease in support (43%). However, making the program mandatory led to a decrease in support to 30%.

    Figure 2 shows the results for the Apple-Google programs from Table 2 as adjusted probabilities of support. Across the 3 scenarios, liberal individuals were more supportive of the program than conservative individuals. For a completely voluntary program (scenario 5), 57% of liberal individuals supported it compared with 36% of moderates and 29% of conservative individuals. The differences were similar for a voluntary program with public health involvement (scenario 6). Across the 3 scenarios, respondents in racial/ethnic minority groups were more supportive, with differences between Hispanic and non-Hispanic individuals ranging from 7.8 to 16.3 percentage points. Black respondents were more supportive of the program with public health involvement (difference, 3.5 percentage points) and a mandatory program (difference, 5.1 percentage points) compared with White respondents.

    Discussion

    Although digital technologies offer substantial opportunities to mitigate the effects of pandemics on public health, they also invite questions about balancing individual privacy and public health benefits. There were 3 key findings of this survey study. First, consumers reported reluctance to give up digital privacy for public health benefit.28 None of the scenarios we presented received majority support: the highest level of support was 43% for using smartphones for contact tracing. Public health departments across the nation are deploying the Apple-Google Bluetooth-enabled contact tracing program. Our findings suggest that well-designed initiatives are needed to achieve widespread adoption. Given the large percentage of respondents who provided neutral responses (range, 23.2%-26.3%), adoption by employers or educational institutions might be a strategy to increase uptake along with opting out or other behavioral nudges to encourage use; people may be amenable to influence.29,30

    Second, personal experience with COVID-19 or living in a county with higher COVID-19 rates was not associated with support for public health digital tools. Instead, political ideology was the greatest contributor to variation in consumer views; conservative individuals were less supportive of using digital tools to mitigate the transmission of COVID-19 than were moderate and liberal individuals. Prior studies in which consumer views on digital privacy were examined revealed conflicting associations with political ideology, perhaps because of views being issue dependent.31-33 For example, conservative individuals have been more supportive than liberal individuals of requiring Apple to make information available to law enforcement agencies to solve crimes and allowing government surveillance to monitor compliance with government programs.31-33 However, other surveys have revealed increasing privacy concerns among conservative individuals and reduced concerns among liberal individuals.32 Our findings related to public health digital privacy may have been associated with the degree to which COVID-19 became a partisan issue in the 2020 election season.34,35

    Third, respondents from racial/ethnic minority groups expressed greater support for use of digital data to address the pandemic than did White and non-Hispanic respondents. This association was consistent in nearly all 9 scenarios and in a model in which all scenarios were combined. Results of prior research36 suggest that Black and Hispanic adults have higher levels of concern about online privacy and security than do White adults. However, Black and Hispanic adults have also reported greater trust in health information from the internet compared with White respondents37 and report feeling more in control of their online information than White adults.38 The higher levels of support for these public health digital tools among respondents from racial/ethnic minority groups may be explained by the disproportionate health effects COVID-19 has had in these populations.

    Our findings suggest that many US adult consumers may be reluctant to give up digital privacy even when facing substantial health and economic harm associated with the COVID-19 pandemic. Politicization of the pandemic may have contributed to these privacy concerns because conservative individuals were less supportive of these digital public health tools. Despite these political differences, many liberal individuals also expressed reticence. These concerns are not consistent with the nearly ubiquitous sharing of digital data in commercial settings, providing arguably less individual or social benefit.38 However, most commercial data sharing occurs not because consumers are willingly sharing but because it has become difficult to avoid sharing.39,40

    The Apple-Google Bluetooth-enabled contact tracing program was designed with strong privacy protections, generally exceeding those found in most digital devices and programs in the commercial marketplace.41,42 The challenge for the public health community to deploy these tools has been to convince a wary public that privacy will be maintained. Achieving widespread adoption may require intensive advertising, trusted messengers, and behavioral nudges to encourage adoption. For example, results from a previous study43 suggest that successes are associated with use of trusted messengers to overcome issues of general mistrust and partisan differences in public health concerns. Without these efforts, our data suggest that most of the population will not seek out these programs and opt in.

    Limitations

    This study has limitations. The results of this survey are reflective of the time during which it was administered, from July 10 to July 31, 2020. Attitudes toward COVID-19 control may have changed as COVID-19 rates changed around the country. In addition, as with all survey findings, there may be important differences in attitudes or views between responders and nonresponders, although our survey had a relatively high response rate. We asked consumers about scenarios that many believe are hypothetical. We might have observed different responses if we had presented individuals with an actual program and asked them to consent to participate.

    Conclusions

    In this survey study of US adults, many US adults were averse to their information being used on digital platforms to mitigate transmission of COVID-19. These findings suggest that in current and future pandemics, public health departments should use multiple strategies to gain public trust and accelerate adoption of tools such as digital contact tracing applications.

    Back to top
    Article Information

    Accepted for Publication: March 29, 2021.

    Published: May 19, 2021. doi:10.1001/jamanetworkopen.2021.10918

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Grande D et al. JAMA Network Open.

    Corresponding Author: David Grande, MD, MPA, Leonard Davis Institute of Health Economics, University of Pennsylvania, 3641 Locust Walk, CPC 407, Philadelphia, PA 19104 (dgrande@wharton.upenn.edu).

    Author Contributions: Dr Grande had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Grande, Merchant, Asch, Cannuscio.

    Acquisition, analysis, or interpretation of data: Grande, Mitra, Luna Marti, Asch, Dolan, Sharma, Cannuscio.

    Drafting of the manuscript: Grande, Luna Marti, Merchant, Dolan, Sharma.

    Critical revision of the manuscript for important intellectual content: Grande, Mitra, Luna Marti, Merchant, Asch, Cannuscio.

    Statistical analysis: Grande, Mitra.

    Obtained funding: Grande.

    Administrative, technical, or material support: Grande, Luna Marti, Merchant, Dolan, Sharma.

    Conflict of Interest Disclosures: Dr Grande reported receiving grants from the National Human Genome Research Institute (NHGRI), National Institutes of Health (NIH) during the conduct of the study and receiving grants from the Robert Wood Johnson Foundation, the Patient-Centered Outcomes Research Institute, and the National Heart, Lung, and Blood Institute outside the submitted work. Dr Mitra reported receiving grants from the NIH during the conduct of the study. Ms Luna Marti reported receiving grants from the NHGRI during the conduct of the study. Dr Merchant reported receiving grants from the NIH during the conduct of the study. Dr Asch reported receiving grants from the NIH during the conduct of the study and receiving personal fees from VAL Health, Healthcare Financial Management Association, National Alliance of Health Care Purchasing Coalitions, Alliance for Continuing Education in the Health Professions, Deloitte, the American Association for Physician Leadership, and the North American Center for Continuing Medical Education outside the submitted work. Ms Dolan reported receiving grants from the NHGRI, NIH during the conduct of the study. Ms Sharma reported receiving grants from the NHGRI, NIH during the conduct of the study. Dr Cannuscio reported receiving grants from the NIH during the conduct of the study.

    Funding/Support: This research was supported by grant 5R01HG009655-04 (Dr Grande) from the NHGRI, NIH.

    Role of the Funder/Sponsor: The NHGRI/NIH had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    References
    1.
    Tau  B. Government tracking how people move around in coronavirus pandemic. Wall Street Journal. Published March 28, 2020. Accessed March 29, 2020. https://www.wsj.com/articles/government-tracking-how-people-move-around-in-coronavirus-pandemic-11585393202
    2.
    Wang  CJ, Ng  CY, Brook  RH.  Response to COVID-19 in Taiwan: big data analytics, new technology, and proactive testing.   JAMA. 2020;323(14):1341-1342. doi:10.1001/jama.2020.3151PubMedGoogle ScholarCrossref
    3.
    Onishi  N, Méheut  C. France weighs its love of liberty in fight against coronavirus. New York Times. Published April 17, 2020. Accessed April 19, 2020. https://www.nytimes.com/2020/04/17/world/europe/coronavirus-france-digital-tracking.html
    4.
    Mozur  P, Zhong  R, Krolik  A. In coronavirus fight, China gives citizens a color code, with red flags. New York Times. Published March 2, 2020. Accessed April 19, 2020. https://www.nytimes.com/2020/03/01/business/china-coronavirus-surveillance.html
    5.
    Sonn  JW. Coronavirus: South Korea’s success in controlling disease is due to its acceptance of surveillance. The Conversation. Published March 19, 2020. Accessed April 16, 2020. https://theconversation.com/coronavirus-south-koreas-success-in-controlling-disease-is-due-to-its-acceptance-of-surveillance-134068
    6.
    McNeil DG Jr. Can smart thermometers track the spread of the coronavirus? New York Times. Published March 18, 2020. Accessed April 16, 2020. https://www.nytimes.com/2020/03/18/health/coronavirus-fever-thermometers.html
    7.
    Moreland  A, Herlihy  C, Tynan  MA,  et al; CDC Public Health Law Program; CDC COVID-19 Response Team, Mitigation Policy Analysis Unit.  Timing of state and territorial COVID-19 stay-at-home orders and changes in population movement—United States, March 1-May 31, 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(35):1198-1203. doi:10.15585/mmwr.mm6935a2PubMedGoogle ScholarCrossref
    8.
    Couture  V, Dingel  JI, Green  AE, Handbury  J, Williams  KR. Measuring movement and social contact with smartphone data: a real-time application to COVID-19. National Bureau of Economic Research Working Paper 27560. Published July 2020. Accessed March 23, 2020. https://www.nber.org/system/files/working_papers/w27560/w27560.pdf
    9.
    Schechner S, Winkler R. Here’s how Apple and Google plan to track the coronavirus through your phone. Wall Street Journal. Published April 11, 2020. Accessed April 13, 2020. https://www.wsj.com/articles/heres-how-apple-and-google-plan-to-track-the-coronavirus-through-your-phone-11586618075
    10.
    Bengio  Y, Janda  R, Yu  YW,  et al.  The need for privacy with public digital contact tracing during the COVID-19 pandemic.   Lancet Digit Health. 2020;2(7):e342-e344. doi:10.1016/S2589-7500(20)30133-3PubMedGoogle ScholarCrossref
    11.
    Mehta  SJ, Troxel  AB, Marcus  N,  et al.  Participation rates with opt-out enrollment in a remote monitoring intervention for patients with myocardial infarction.   JAMA Cardiol. 2016;1(7):847-848. doi:10.1001/jamacardio.2016.2374PubMedGoogle ScholarCrossref
    12.
    Hinch  R, Probert  W, Nurtay  A,  et al. Effective configurations of a digital contact tracing app: a report to NHSX. Accessed May 2, 2020. https://cdn.theconversation.com/static_files/files/1009/Report_-_Effective_App_Configurations.pdf?1587531217
    13.
    Ferretti  L, Wymant  C, Kendall  M,  et al.  Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing.   Science. 2020;368(6491):eabb6936. doi:10.1126/science.abb6936PubMedGoogle Scholar
    14.
    Ting  DSW, Carin  L, Dzau  V, Wong  TY.  Digital technology and COVID-19.   Nat Med. 2020;26(4):459-461. doi:10.1038/s41591-020-0824-5PubMedGoogle ScholarCrossref
    15.
    Cho  H, Ippolito  D, Yu  YW. Contact tracing mobile apps for COVID-19: privacy considerations and related trade-offs. arXiv. Preprint posted online March 30, 2020. arXiv:2003.11511v2
    16.
    Valentino-DeVries  J, Singer  N, Krolik  A. A scramble for virus apps that do no harm. New York Times. Published April 29, 2020. Accessed April 30, 2020. https://www.nytimes.com/2020/04/29/business/coronavirus-cellphone-apps-contact-tracing.html
    17.
    Ienca  M, Vayena  E.  On the responsible use of digital data to tackle the COVID-19 pandemic.   Nat Med. 2020;26(4):463-464. doi:10.1038/s41591-020-0832-5PubMedGoogle ScholarCrossref
    18.
    Lopez L, Hamel L. Public attitudes towards the coronavirus outbreak in highly affected counties. Kaiser Family Foundation. Published April 29, 2020. Accessed May 11, 2020. https://www.kff.org/policy-watch/public-attitudes-towards-coronavirus-outbreak-highly-affected-counties/
    19.
    Ipsos. KnowledgePanel: a methodological overview. Accessed November 4, 2020. https://www.ipsos.com/sites/default/files/ipsosknowledgepanelmethodology.pdf
    20.
    Dimitropoulos L, Patel V, Scheffler SA, Posnack S. Public attitudes toward health information exchange: perceived benefits and concerns. Am J Manag Care. 2011;17(12):SP111-SP116.
    21.
    Patel  VN, Dhopeshwarkar  RV, Edwards  A,  et al.  Low-income, ethnically diverse consumers’ perspective on health information exchange and personal health records.   Inform Health Soc Care. 2011;36(4):233-252.doi:10.3109/17538157.2011.554930PubMedGoogle ScholarCrossref
    22.
    US Centers for Disease Control and Prevention. Health equity considerations and racial and ethnic minority groups. Published April 30, 2020. Updated February 12, 2021. Accessed February 13, 2021. https://www.cdc.gov/coronavirus/2019-ncov/community/health-equity/race-ethnicity.html
    23.
    Alcendor  DJ.  Racial disparities-associated COVID-19 mortality among minority populations in the US.   J Clin Med. 2020;9(8):2442. doi:10.3390/jcm9082442PubMedGoogle ScholarCrossref
    24.
    Buckee  CO, Balsari  S, Chan  J,  et al.  Aggregated mobility data could help fight COVID-19.   Science. 2020;368(6487):145-146. doi:10.1126/science.abb8021PubMedGoogle Scholar
    25.
    Chen  E, Lerman  K, Ferrara  E.  Tracking social media discourse about the COVID-19 pandemic: development of a public coronavirus Twitter data set.   JMIR Public Health Surveill. 2020;6(2):e19273. doi:10.2196/19273PubMedGoogle Scholar
    26.
    Open Government Partnership. Collecting open government approaches to COVID-19. Accessed April 16, 2020. https://www.opengovpartnership.org/collecting-open-government-approaches-to-covid-19/
    27.
    Callegaro M, DiSogra C.  Computing response metrics for online panels.   Public Opin Q. 2008;72(5):1008–1032. Google ScholarCrossref
    28.
    Auxier  B. How Americans see digital privacy issues amid the COVID-19 outbreak. Pew Research Center. Published May 4, 2020. Accessed November 4, 2020. https://www.pewresearch.org/fact-tank/2020/05/04/how-americans-see-digital-privacy-issues-amid-the-covid-19-outbreak/
    29.
    Beidas  RS, Buttenheim  AM, Feuerstein-Simon  R,  et al.  Optimizing and implementing contact tracing through behavioral economics.   NEJM Catal. Published online June 23, 2020.Google Scholar
    30.
    Patel  MS, Volpp  KG, Asch  DA.  Nudge units to improve the delivery of health care.   N Engl J Med. 2018;378(3):214-216. doi:10.1056/NEJMp1712984PubMedGoogle ScholarCrossref
    31.
    Finkle  J. Solid support for Apple in iPhone encryption fight: poll. Reuters. Published February 24, 2016. Accessed December 5, 2020. https://www.reuters.com/article/us-apple-encryption-poll-idUSKCN0VX159
    32.
    Turow  J, Hennessy  M, Draper  N, Ankabi  O, Virgilio D. Divided we feel: partisan politics drive Americans’ emotions regarding surveillance of low-income populations. Annenberg School for Communication, University of Pennsylvania. Published 2018. Accessed April 3, 2020. https://repository.upenn.edu/asc_papers/543/
    33.
    Doherty  C. Balancing act: national security and civil liberties in post-9/11 era. Pew Research Center. Published June 7, 2013. Accessed April 20, 2020. https://www.pewresearch.org/fact-tank/2013/06/07/balancing-act-national-security-and-civil-liberties-in-post-911-era/
    34.
    Calvillo  DP, Ross  BJ, Garcia  RJB, Smelter  TJ, Rutchick  AM.  Political ideology predicts perceptions of the threat of COVID-19 (and susceptibility to fake news about it).   Soc Psychol Personal Sci. 2020;11(8):1119-1128. doi:10.1177/1948550620940539Google ScholarCrossref
    35.
    Hamel  L, Kearney  A, Kirzinger  A, Lopes  L, Muñana  C, Brodie  M. KFF health tracking poll—May 2020. Kaiser Family Foundation. Published May 27, 2020. Accessed June 2, 2020. https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-may-2020/
    36.
    Madden  M. Privacy, security, and digital inequality: how technology experiences and resources vary by socioeconomic status, race, and ethnicity. Published September 27, 2017. Accessed April 12, 2020. https://datasociety.net/pubs/prv/DataAndSociety_PrivacySecurityandDigitalInequality.pdf
    37.
    Fareed  N, Swoboda  CM, Jonnalagadda  P, Walker  DM, Huerta  TR. Differences between races in health information seeking and trust over time: evidence from a cross-sectional, pooled analyses of HINTS data. Am J Health Promot. 2020;35(1):84-92. doi:10.1177/0890117120934609
    38.
    Auxier  B, Rainie  L, Anderson  M, Perrin  A, Kumar  M, Turner  E. Americans and privacy: concerned, confused and feeling lack of control over their personal information. Pew Research Center: Internet & Technology. Published November 15, 2019. Accessed December 5, 2020. https://www.pewresearch.org/internet/2019/11/15/americans-and-privacy-concerned-confused-and-feeling-lack-of-control-over-their-personal-information/
    39.
    Grande  D, Luna Marti  X, Feuerstein-Simon  R,  et al.  Health policy and privacy challenges associated with digital technology.   JAMA Netw Open. 2020;3(7):e208285. doi:10.1001/jamanetworkopen.2020.8285PubMedGoogle Scholar
    40.
    Draper  NA, Turow  J.  The corporate cultivation of digital resignation.   New Media Soc. 2019;21(8):1824-1839. doi:10.1177/1461444819833331Google ScholarCrossref
    41.
    Veale  M. Privacy is not the problem with the Apple-Google contact-tracing toolkit. The Guardian. Published July 1, 2020. Accessed August 4, 2020. https://www.theguardian.com/commentisfree/2020/jul/01/apple-google-contact-tracing-app-tech-giant-digital-rights
    42.
    Bradford  L, Aboy  M, Liddell  K.  COVID-19 contact tracing apps: a stress test for privacy, the GDPR, and data protection regimes.   J Law Biosci. 2020;7(1):Isaa034. doi:10.1093/jlb/lsaa034PubMedGoogle ScholarCrossref
    43.
    Cartmell  KB, Mzik  CR, Sundstrom  BL, Luque  JS, White  A, Young-Pierce  J.  HPV vaccination communication messages, messengers, and messaging strategies.   J Cancer Educ. 2019;34(5):1014-1023. doi:10.1007/s13187-018-1405-xPubMedGoogle ScholarCrossref
    ×