Frequency of Adverse Events in the Placebo Arms of COVID-19 Vaccine Trials: A Systematic Review and Meta-analysis | Clinical Pharmacy and Pharmacology | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  PRISMA Flow Diagram
PRISMA Flow Diagram
Figure 2.  Forest Plots of Any Systemic Adverse Events After the First and Second Doses of the COVID-19 Vaccine or Placebo
Forest Plots of Any Systemic Adverse Events After the First and Second Doses of the COVID-19 Vaccine or Placebo

Random-effects pooled proportions are shown. Boxes represent the effect size of each study and whiskers, 95% CIs. Box size indicates the study’s weight in the analysis. Diamonds indicate pooled estimates, with whiskers indicating 95% CIs. Studies that did not provide data for the “any systemic adverse event” category were not included in these analyses but only in analyses on other adverse event categories. All placebos used were inert saline solutions.

amRNA-1273 (Moderna, mRNA vaccine, phase 3 trial). Probability of being randomized to placebo group, 50%.

bmRNA-1273 (Moderna, mRNA vaccine, phase 2 trial). Vaccine group was selected for comparison in trials that investigated multiple dosing schemes (100 μg [high dose]). Probability of being randomized to placebo group, 33%.

cNVX-CoV2373 (Novavax, protein-based vaccine, phase 3 trial). Vaccine contained adjuvants. Probability of being randomized to placebo group, 50%.

dNVX-CoV2373 (Novavax, protein-based vaccine, phase 1-2 trial). Vaccine contained adjuvants. Vaccine group was selected for comparison in trials that investigated multiple dosing schemes (5 μg [low dose] + adjuvant). Probability of being randomized to placebo group, 20%.

eBNT162b2 (BioNTech/Pfizer, mRNA vaccine, phase 2-3 trial). Probability of being randomized to placebo group, 50%.

fSCB-2019 (Clover, protein-based vaccine, phase 1 trial). Vaccine contained adjuvants. Vaccine group was selected for comparison in trials that investigated multiple dosing schemes (30 μg [high dose] + adjuvant). Probability of being randomized to placebo group, 20%.

gNVX-CoV2373 (Novavax, protein-based vaccine, phase 2a-b trial); vaccine contained adjuvants. Probability of being randomized to placebo group, 50%.

Table 1.  Characteristics of the 12 Analyzed Randomized Clinical Trials of COVID-19 Vaccines
Characteristics of the 12 Analyzed Randomized Clinical Trials of COVID-19 Vaccines
Table 2.  Coefficients of the Random-Effects Meta-analysis of Adverse Event Proportions in the Placebo Groups
Coefficients of the Random-Effects Meta-analysis of Adverse Event Proportions in the Placebo Groups
Table 3.  Coefficients of the Random-Effects Meta-analysis of Logarithmic Odds Ratios to Compare the Occurrence of Adverse Events in the Placebo and Vaccine Groups
Coefficients of the Random-Effects Meta-analysis of Logarithmic Odds Ratios to Compare the Occurrence of Adverse Events in the Placebo and Vaccine Groups
1.
World Health Organisation. WHO coronavirus (COVID-19) dashboard. 2021. Accessed November 10, 2021. https://covid19.who.int/
2.
Khubchandani  J, Sharma  S, Price  JH, Wiblishauser  MJ, Sharma  M, Webb  FJ.  COVID-19 vaccination hesitancy in the United States: a rapid national assessment.   J Community Health. 2021;46(2):270-277. doi:10.1007/s10900-020-00958-x PubMedGoogle ScholarCrossref
3.
Razai  MS, Chaudhry  UAR, Doerholt  K, Bauld  L, Majeed  A.  COVID-19 vaccination hesitancy.   BMJ. 2021;373(n1138):n1138. doi:10.1136/bmj.n1138 PubMedGoogle Scholar
4.
Robinson  E, Jones  A, Lesser  I, Daly  M.  International estimates of intended uptake and refusal of COVID-19 vaccines: a rapid systematic review and meta-analysis of large nationally representative samples.   Vaccine. 2021;39(15):2024-2034. doi:10.1016/j.vaccine.2021.02.005 PubMedGoogle ScholarCrossref
5.
World Health Organization. Ten threats to global health in 2019. Accessed September 1, 2021. https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019
6.
Taylor  S, Landry  CA, Paluszek  MM, Groenewoud  R, Rachor  GS, Asmundson  GJG.  A proactive approach for managing COVID-19: the importance of understanding the motivational roots of vaccination hesitancy for SARS-CoV2.   Front Psychol. 2020;11:575950. doi:10.3389/fpsyg.2020.575950 PubMedGoogle Scholar
8.
Smith  LE, Amlôt  R, Weinman  J, Yiend  J, Rubin  GJ.  Why do parents not re-vaccinate their child for influenza? a prospective cohort study.   Vaccine. 2020;38(27):4230-4235. doi:10.1016/j.vaccine.2020.04.029 PubMedGoogle ScholarCrossref
9.
Schmid  P, Rauber  D, Betsch  C, Lidolt  G, Denker  ML.  Barriers of influenza vaccination intention and behavior—a systematic review of influenza vaccine hesitancy, 2005-2016.   PLoS One. 2017;12(1):e0170550. doi:10.1371/journal.pone.0170550 PubMedGoogle Scholar
10.
Ramsey  MA, Marczinski  CA.  College students’ perceptions of H1N1 flu risk and attitudes toward vaccination.   Vaccine. 2011;29(44):7599-7601. doi:10.1016/j.vaccine.2011.07.130 PubMedGoogle ScholarCrossref
11.
Gorman  JR, Brewer  NT, Wang  JB, Chambers  CD.  Theory-based predictors of influenza vaccination among pregnant women.   Vaccine. 2012;31(1):213-218. doi:10.1016/j.vaccine.2012.10.064 PubMedGoogle ScholarCrossref
12.
Sun  KS, Lam  TP, Kwok  KW, Lam  KF, Wu  D, Ho  PL.  Seasonal influenza vaccine uptake among Chinese in Hong Kong: barriers, enablers and vaccination rates.   Hum Vaccin Immunother. 2020;16(7):1675-1684. doi:10.1080/21645515.2019.1709351 PubMedGoogle ScholarCrossref
13.
Howick  J, Webster  R, Kirby  N, Hood  K.  Rapid overview of systematic reviews of nocebo effects reported by patients taking placebos in clinical trials.   Trials. 2018;19(1):674. doi:10.1186/s13063-018-3042-4 PubMedGoogle ScholarCrossref
14.
Barsky  AJ, Saintfort  R, Rogers  MP, Borus  JF.  Nonspecific medication side effects and the nocebo phenomenon.   JAMA. 2002;287(5):622-627. doi:10.1001/jama.287.5.622 PubMedGoogle ScholarCrossref
15.
Rief  W, Barsky  AJ, Glombiewski  JA, Nestoriuc  Y, Glaesmer  H, Braehler  E.  Assessing general side effects in clinical trials: reference data from the general population.   Pharmacoepidemiol Drug Saf. 2011;20(4):405-415. doi:10.1002/pds.2067 PubMedGoogle ScholarCrossref
16.
Benedetti  F, Lanotte  M, Lopiano  L, Colloca  L.  When words are painful: unraveling the mechanisms of the nocebo effect.   Neuroscience. 2007;147(2):260-271. doi:10.1016/j.neuroscience.2007.02.020 PubMedGoogle ScholarCrossref
17.
Amanzio  M, Corazzini  LL, Vase  L, Benedetti  F.  A systematic review of adverse events in placebo groups of anti-migraine clinical trials.   Pain. 2009;146(3):261-269. doi:10.1016/j.pain.2009.07.010 PubMedGoogle ScholarCrossref
18.
Rief  W, Nestoriuc  Y, von Lilienfeld-Toal  A,  et al.  Differences in adverse effect reporting in placebo groups in SSRI and tricyclic antidepressant trials: a systematic review and meta-analysis.   Drug Saf. 2009;32(11):1041-1056. doi:10.2165/11316580-000000000-00000 PubMedGoogle ScholarCrossref
19.
Pan  Y, Kinitz  T, Stapic  M, Nestoriuc  Y.  Minimizing drug adverse events by informing about the nocebo effect—an experimental study.   Front Psychiatry. 2019;10:504. doi:10.3389/fpsyt.2019.00504 PubMedGoogle ScholarCrossref
20.
Ballou  S, Iturrino  J, Rangan  V,  et al.  Improving medication tolerance: a pilot study in disorders of gut-brain interaction treated with tricyclic antidepressants.   J Clin Gastroenterol. 2021;(June). doi:10.1097/MCG.0000000000001575 PubMedGoogle Scholar
21.
Faasse  K, Huynh  A, Pearson  S, Geers  AL, Helfer  SG, Colagiuri  B.  The influence of side effect information framing on nocebo effects.   Ann Behav Med. 2019;53(7):621-629. doi:10.1093/abm/kay071 PubMedGoogle ScholarCrossref
22.
Wilhelm  M, Rief  W, Doering  BK.  Decreasing the burden of side effects through positive message framing: an experimental proof-of-concept study.   Int J Behav Med. 2018;25(4):381-389. doi:10.1007/s12529-018-9726-z PubMedGoogle ScholarCrossref
23.
O’Connor  AM, Pennie  RA, Dales  RE.  Framing effects on expectations, decisions, and side effects experienced: the case of influenza immunization.   J Clin Epidemiol. 1996;49(11):1271-1276. doi:10.1016/S0895-4356(96)00177-1 PubMedGoogle ScholarCrossref
24.
Howe  LC, Leibowitz  KA, Perry  MA,  et al.  Changing patient mindsets about non-life-threatening symptoms during oral immunotherapy: a randomized clinical trial.   J Allergy Clin Immunol Pract. 2019;7(5):1550-1559. doi:10.1016/j.jaip.2019.01.022 PubMedGoogle ScholarCrossref
25.
Mitsikostas  DD, Mantonakis  L, Chalarakis  N.  Nocebo in clinical trials for depression: a meta-analysis.   Psychiatry Res. 2014;215(1):82-86. doi:10.1016/j.psychres.2013.10.019 PubMedGoogle ScholarCrossref
26.
Benedetti  F, Amanzio  M, Vighetti  S, Asteggiano  G.  The biochemical and neuroendocrine bases of the hyperalgesic nocebo effect.   J Neurosci. 2006;26(46):12014-12022. doi:10.1523/JNEUROSCI.2947-06.2006 PubMedGoogle ScholarCrossref
27.
Palermo  S, Giovannelli  F, Bartoli  M, Amanzio  M.  Are patients with schizophrenia spectrum disorders more prone to manifest nocebo-like-effects? a meta-analysis of adverse events in placebo groups of double-blind antipsychotic trials.   Front Pharmacol. 2019;10:502. doi:10.3389/fphar.2019.00502 PubMedGoogle ScholarCrossref
28.
Mahr  A, Golmard  C, Pham  E, Iordache  L, Deville  L, Faure  P.  Types, frequencies, and burden of nonspecific adverse events of drugs: analysis of randomized placebo-controlled clinical trials.   Pharmacoepidemiol Drug Saf. 2017;26(7):731-741. doi:10.1002/pds.4169 PubMedGoogle ScholarCrossref
29.
Bender  F, Rief  W, Wilhelm  M. Really just a little prick? a meta-analysis on adverse events in placebo control groups of seasonal influenza vaccination RCTs. Abstract #O1505. Third International Conference of the Society for Interdisciplinary Placebo Studies (SIPS); May 26-28, 2021; Baltimore, MD.
30.
Amanzio  M, Cipriani  GE, Bartoli  M.  How do nocebo effects in placebo groups of randomized controlled trials provide a possible explicative framework for the COVID-19 pandemic?   Expert Rev Clin Pharmacol. 2021;14(4):439-444. doi:10.1080/17512433.2021.1900728 PubMedGoogle ScholarCrossref
31.
Rief  W. Fear of adverse effects and COVID-19 vaccine hesitancy: recommendations of the treatment expectation expert group. JAMA Heal Forum. 2021;2(4):e210804. doi:10.1001/jamahealthforum.2021.0804
32.
Page  MJ, McKenzie  JE, Bossuyt  PM,  et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.   BMJ. 2021;372(n71):n71. doi:10.1136/bmj.n71 PubMedGoogle Scholar
33.
Higgins  JPT, Altman  DG, Gøtzsche  PC,  et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.   BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMedGoogle ScholarCrossref
34.
Microsoft Excel. Microsoft Corporation; 2018. Accessed August 16, 2019. https://office.microsoft.com/excel
35.
JASP. Version 0.14.1. JASP; 2020. Accessed May 21, 2021. https://jasp-stats.org/download/
36.
Higgins  JPT, Thompson  SG.  Quantifying heterogeneity in a meta-analysis.   Stat Med. 2002;21(11):1539-1558. doi:10.1002/sim.1186 PubMedGoogle ScholarCrossref
37.
Duval  S, Tweedie  R.  Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis.   Biometrics. 2000;56(2):455-463. doi:10.1111/j.0006-341X.2000.00455.x PubMedGoogle ScholarCrossref
38.
Baden  LR, El Sahly  HM, Essink  B,  et al; COVE Study Group.  Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.   N Engl J Med. 2020;384(5):403-416. doi:10.1056/NEJMoa2035389 PubMedGoogle ScholarCrossref
39.
Heath  PT, Galiza  EP, Baxter  DN,  et al; 2019nCoV-302 Study Group.  Safety and efficacy of NVX-CoV2373 COVID-19 vaccine.   N Engl J Med. 2021;385(13):1172-1183. doi:10.1056/NEJMoa2107659 PubMedGoogle ScholarCrossref
40.
Walsh  EE, Frenck  RWJ  Jr, Falsey  AR,  et al.  Safety and immunogenicity of two RNA-based COVID-19 vaccine candidates.   N Engl J Med. 2020;383(25):2439-2450. doi:10.1056/NEJMoa2027906 PubMedGoogle ScholarCrossref
41.
Richmond  P, Hatchuel  L, Dong  M,  et al.  Safety and immunogenicity of S-Trimer (SCB-2019), a protein subunit vaccine candidate for COVID-19 in healthy adults: a phase 1, randomised, double-blind, placebo-controlled trial.   Lancet. 2021;397(10275):682-694. doi:10.1016/S0140-6736(21)00241-5 PubMedGoogle ScholarCrossref
42.
Sadoff  J, Le Gars  M, Shukarev  G,  et al.  Interim results of a phase 1-2a trial of Ad26.COV2.S COVID-19 vaccine.   N Engl J Med. 2021;384(19):1824-1835. doi:10.1056/NEJMoa2034201 PubMedGoogle ScholarCrossref
43.
Chu  L, McPhee  R, Huang  W,  et al; mRNA-1273 Study Group.  A preliminary report of a randomized controlled phase 2 trial of the safety and immunogenicity of mRNA-1273 SARS-CoV-2 vaccine.   Vaccine. 2021;39(20):2791-2799. doi:10.1016/j.vaccine.2021.02.007 PubMedGoogle ScholarCrossref
44.
Keech  C, Albert  G, Cho  I,  et al.  Phase 1-2 trial of a SARS-CoV-2 recombinant spike protein nanoparticle vaccine.   N Engl J Med. 2020;383(24):2320-2332. doi:10.1056/NEJMoa2026920 PubMedGoogle ScholarCrossref
45.
Li  J, Hui  A, Zhang  X,  et al.  Safety and immunogenicity of the SARS-CoV-2 BNT162b1 mRNA vaccine in younger and older Chinese adults: a randomized, placebo-controlled, double-blind phase 1 study.   Nat Med. 2021;27(6):1062-1070. doi:10.1038/s41591-021-01330-9 PubMedGoogle ScholarCrossref
46.
Goepfert  PA, Fu  B, Chabanon  A-L,  et al.  Safety and immunogenicity of SARS-CoV-2 recombinant protein vaccine formulations in healthy adults: interim results of a randomised, placebo-controlled, phase 1-2, dose-ranging study.   Lancet Infect Dis. 2021;21(9):1257-1270. doi:10.1016/S1473-3099(21)00147-X PubMedGoogle ScholarCrossref
47.
Madhi  SA, Baillie  V, Cutland  CL,  et al; NGS-SA Group; Wits-VIDA COVID Group.  Efficacy of the ChAdOx1 nCoV-19 COVID-19 vaccine against the B.1.351 variant.   N Engl J Med. 2021;384(20):1885-1898. doi:10.1056/NEJMoa2102214 PubMedGoogle ScholarCrossref
48.
Polack  FP, Thomas  SJ, Kitchin  N,  et al; C4591001 Clinical Trial Group.  Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine.   N Engl J Med. 2020;383(27):2603-2615. doi:10.1056/NEJMoa2034577 PubMedGoogle ScholarCrossref
49.
Shinde  V, Bhikha  S, Hoosain  Z,  et al; 2019nCoV-501 Study Group.  Efficacy of NVX-CoV2373 COVID-19 vaccine against the B.1.351 variant.   N Engl J Med. 2021;384(20):1899-1909. doi:10.1056/NEJMoa2103055 PubMedGoogle ScholarCrossref
50.
Chappell  KJ, Mordant  FL, Li  Z,  et al.  Safety and immunogenicity of an MF59-adjuvanted spike glycoprotein-clamp vaccine for SARS-CoV-2: a randomised, double-blind, placebo-controlled, phase 1 trial.   Lancet Infect Dis. 2021;21(10):1383-1394. doi:10.1016/S1473-3099(21)00200-0 PubMedGoogle ScholarCrossref
51.
Rief  W, Glombiewski  JA.  The hidden effects of blinded, placebo-controlled randomized trials: an experimental investigation.   Pain. 2012;153(12):2473-2477. doi:10.1016/j.pain.2012.09.007 PubMedGoogle ScholarCrossref
52.
Centers for Disease Control and Prevention. Moderna COVID-19 vaccine overview and safety. Updated November 19, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/Moderna.html
53.
Centers for Disease Control and Prevention. Pfizer-BioNTech COVID-19 vaccine (also known as COMIRNATY) overview and safety. Updated November 19, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/Pfizer-BioNTech.html
54.
Centers for Disease Control and Prevention. Johnson & Johnson’s Janssen COVID-19 vaccine overview and safety. Updated October 29, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/janssen.html
55.
Reidenberg  MM, Lowenthal  DT.  Adverse nondrug reactions.   N Engl J Med. 1968;279(13):678-679. doi:10.1056/NEJM196809262791304 PubMedGoogle ScholarCrossref
56.
Wells  RE, Kaptchuk  TJ.  To tell the truth, the whole truth, may do patients harm: the problem of the nocebo effect for informed consent.   Am J Bioeth. 2012;12(3):22-29. doi:10.1080/15265161.2011.652798 PubMedGoogle ScholarCrossref
57.
Rief  W, Glombiewski  JA, Barsky  AJ. Generic assessment of side effects. 2009. Accessed June 21, 2021. http://www.gase-scale.com/Instruction_GASE.pdf
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    2 Comments for this article
    EXPAND ALL
    Replication & Nocebo
    Martina Amanzio, PhD | Department of Psychology, University of Turin
    We read the article by Haas et al. with interest and we are pleased to see that it confirms the results shown in our article published October 2021[1].

    The authors analyzed the frequency of adverse events (AEs) in the placebo arms of COVID-19 vaccine randomized controlled trials (RCTs) with a total of 45 380 participants (22 578 placebo recipients and 22 802 vaccine recipients.

    In our study, we considered only phase III safety data of vaccines approved by the Western pharmaceutical regulatory agencies, such as the Food and Drug Administration (FDA) or the European Medicine Agency (EMA) (Johnson
    & Johnson, Moderna, and Pfizer) with a total of 45 139 participants: two mRNA-based (n38 403) and one adenovirus type (6 736). We excluded the trials in which the control groups were not treated with a verum placebo (only one trial for AZD1222 had administered saline in the placebo group; however, it was not included in the interim primary efficacy analyses).

    Our results showed that the most frequently reported AEs in the placebo arms of both groups of the analyzed vaccines were fatigue, headache, local pain, and myalgia. The frequency of the AEs in the placebo arms considered by Haas et al. were lower compared with those found in our systematic review.

    The difference in results may be due to the high heterogeneity of the data collected by Haas et al. In fact, they considered different data together, such as different trial phases (ie, I, II, and III), and vaccines both approved (AstraZeneca, BioNTech/Pfizer, Johnson & Johnson, Moderna) and not approved (Clover, Novavax, Sanofi Pasteur) by FDA or EMA, and vaccines with different manufacturing methods (mRNA, viral vector, and protein-based) were analyzed together. Finally, Haas et al. did not analyze by age groups, whereas we found that the AEs were mild and reported more in younger participants and after the first dose of placebo groups of the mRNA vaccines.

    We would also call attention to the study by Lee and Song[2], which also focused on nocebo effects in COVID-19 vaccine trials and was published in October 2021. The authors considered four SARS-CoV-2 vaccine trials (Pfizer/BioNTech, Moderna, AstraZeneca/Oxford, Gamaleya (Sputnik V), either approved and not approved by FDA and EMA - with a total of 119 110 participants.

    So now, three studies have come to similar conclusions, which are that a substantial proportion of the side effects experienced by vaccine recipients are likely to be nocebo effects and not a result of the pharmacological action of the vaccine. Given this degree of consistency in the results, the medical and public health communities should consider the nocebo effect seriously in approaching vaccine refusal and hesitancy.

    References

    1. Amanzio et al., 2021; doi: 10.1016/j.lanepe.2021.100253

    2. Lee and Song, 2021; doi: 10.5414/CP204028

    CONFLICT OF INTEREST: None Reported
    READ MORE
    The nocebo effect: an issue that should be considered in immunization campaigns
    Antonio Clavenna, PhD | Department of Public Health, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy
    The systematic review by Julia Haas et al documented a high rate of adverse events (AEs) in subjects receiving placebo in the phase III randomized controlled COVID-19 vaccine trials. Specifically, the review found a rate of 35% of placebo recipients reporting systemic AEs after the first dose, and a incidence ratio between placebo and vaccine arms (nocebo effect) of 76%.[1] Although in active vaccine arms AEs are always higher, a similar nocebo effect in placebo recipients was also reported in the meta-analysis by Amanzio et al, which considered only mRNA vaccines and Ad26.COV2.S.[2]
    The nocebo effect is the opposite of
    the placebo effect, and typically refers to adverse effects of a drug or vaccine that derive from the recipient’s expectations of its effects rather than its biochemistry. Thus, the nocebo effect, measured soon after vaccine administration, can contribute to influencing vaccine hesitancy even if it refers to mild or moderate systemic effects (headache, fatigue, and arm pain were the most common). In fact, the nocebo effect is one of the factors people take into consideration in deciding whether to vaccinate themselves, so it is essential to understand, and communicate, the role of the nocebo effect in order to reduce hesitancy.[3] Gender and age are reported as two of the determinants associated with the nocebo effect. Compared to males, females were more likely to report vaccine-related AEs and to be nocebo prone.[4] Concerning age, similar rates of systemic adverse events were reported in children 5-11 years old and in subjects 16-55 years old receiving placebo in the RCTs evaluating the effectiveness and safety of BNT162b2 vaccine.[5,6] This suggests that no relevant differences may exist in the pediatric versus adult populations.
    The occurrence of the nocebo effect is not new. In trials concerning COVID-19 vaccines, however, the rate is double the rate reported for flu vaccines in healthy adults. [7] The burden of fear and doubt is therefore greater for COVID-19 vaccines compared to other vaccines. This should be considered when evaluating the AEs reported after vaccination and when communicating with people about the importance of adhering to the vaccination campaign.
    The role of placebo is essential for evaluating the safety and the efficacy of a “first-in-class” vaccine, but it is unethical to enroll participants in placebo trial arms when safe and effective vaccines are available.[8] Evaluating the nocebo effect is indispensable for better planning interventions when vaccines are used for mass campaigns.

    Antonio Clavenna and Maurizio Bonati
    Department of Public Health, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy

    References
    1. Haas JW, et al. JAMA Netw Open 2022;5(1):e2143955. doi: 10.1001/jamanetworkopen.2021.43955.
    2. Amanzio M, et al. Lancet Reg Health Eur. 2022;12:100253. doi: 10.1016/j.lanepe.2021.100253.
    3. Attwell K, Hannah A, Leask J. COVID-19: talk of ‘vaccine hesitancy’ lets governments off the hook. Nature 2022; 602, 574-577.
    4. Mitsikostas DD, et al Vaccines (Basel). 2021;9(10):1179. doi: 10.3390/vaccines9101179.
    5. Walter EB, et al. N Engl J Med. 2022;386(1):35-46. doi: 10.1056/NEJMoa2116298.
    6. Polack FP, et al. N Engl J Med. 2020 Dec 31;383(27):2603-2615. doi: 10.1056/NEJMoa2034577.
    7. Demicheli V, et al. Cochrane Database Syst Rev. 2014;(3):CD001269. doi: 10.1002/14651858.
    8. Bonati M. Eur J Clin Pharmacol. 2022;78:147-148. doi: 10.1007/s00228-021-03203-z.
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Original Investigation
    Public Health
    January 18, 2022

    Frequency of Adverse Events in the Placebo Arms of COVID-19 Vaccine Trials: A Systematic Review and Meta-analysis

    Author Affiliations
    • 1Program in Placebo Studies, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
    • 2Division of Gastroenterology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
    • 3Department of Clinical Psychology and Psychotherapy, Philipps University Marburg, Marburg, Germany
    • 4Department of Psychology, Endicott College, Beverly, Massachusetts
    • 5Weill Cornell Medical College, New York, New York
    JAMA Netw Open. 2022;5(1):e2143955. doi:10.1001/jamanetworkopen.2021.43955
    Key Points

    Question  What was the frequency of adverse events (AEs) in the placebo groups of COVID-19 vaccine trials?

    Findings  In this systematic review and meta-analysis of 12 articles including AE reports for 45 380 trial participants, systemic AEs were experienced by 35% of placebo recipients after the first dose and 32% after the second. Significantly more AEs were reported in the vaccine groups, but AEs in placebo arms (“nocebo responses”) accounted for 76% of systemic AEs after the first COVID-19 vaccine dose and 52% after the second dose.

    Meaning  This study found that the rate of nocebo responses in placebo arms of COVID-19 vaccine trials was substantial; this finding should be considered in public vaccination programs.

    Abstract

    Importance  Adverse events (AEs) after placebo treatment are common in randomized clinical drug trials. Systematic evidence regarding these nocebo responses in vaccine trials is important for COVID-19 vaccination worldwide especially because concern about AEs is reported to be a reason for vaccination hesitancy.

    Objective  To compare the frequencies of AEs reported in the placebo groups of COVID-19 vaccine trials with those reported in the vaccine groups.

    Data Sources  For this systematic review and meta-analysis, the Medline (PubMed) and Cochrane Central Register of Controlled Trials (CENTRAL) databases were searched systematically using medical subheading terms and free-text keywords for trials of COVID-19 vaccines published up to July 14, 2021.

    Study Selection  Randomized clinical trials of COVID-19 vaccines that investigated adults aged 16 years or older were selected if they assessed solicited AEs within 7 days of injection, included an inert placebo arm, and provided AE reports for both the vaccine and placebo groups separately. Full texts were reviewed for eligibility by 2 independent reviewers.

    Data Extraction and Synthesis  Data extraction and quality assessment were performed independently by 2 reviewers, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guideline and using the Cochrane risk-of-bias tool. Meta-analyses were based on random-effects models.

    Main Outcomes and Measures  The primary outcomes were the proportions of placebo recipients reporting overall, systemic, and local (injection-site) AEs as well as logarithmic odds ratios (ORs) to evaluate group differences. Outcomes were tested for significance using z tests with 95% CIs.

    Results  Twelve articles with AE reports for 45 380 participants (22 578 placebo recipients and 22 802 vaccine recipients) were analyzed. After the first dose, 35.2% (95% CI, 26.7%-43.7%) of placebo recipients experienced systemic AEs, with headache (19.3%; 95% CI, 13.6%-25.1%) and fatigue (16.7%; 95% CI, 9.8%-23.6%) being most common. After the second dose, 31.8% (95% CI, 28.7%-35.0%) of placebo recipients reported systemic AEs. The ratio between placebo and vaccine arms showed that nocebo responses accounted for 76.0% of systemic AEs after the first COVID-19 vaccine dose and for 51.8% after the second dose. Significantly more vaccine recipients reported AEs, but the group difference for systemic AEs was small after the first dose (OR, −0.47; 95% CI, −0.54 to −0.40; P < .001; standardized mean difference, −0.26; 95% CI, −0.30 to −0.22) and large after the second dose (OR, −1.36; 95% CI, −1.86 to −0.86; P < .001; standardized mean difference, −0.75; 95% CI, −1.03 to −0.47).

    Conclusions and Relevance  In this systematic review and meta-analysis, significantly more AEs were reported in vaccine groups compared with placebo groups, but the rates of reported AEs in the placebo arms were still substantial. Public vaccination programs should consider these high rates of AEs in placebo arms.

    Introduction

    The ongoing COVID-19 pandemic has caused more than 5 million deaths worldwide1 and led to tremendous physical, mental, and economic hardships. Several vaccines have been developed and tested within remarkably short periods. Currently, public vaccination programs have already achieved success in reducing the numbers of new infections in several countries. However, a substantial proportion of the population (internationally estimated at approximately 20%) intends to refuse vaccination.2-4 In 2019, before the COVID-19 pandemic, the World Health Organization claimed vaccination hesitancy as a global health threat5; this threat is particularly salient in the case of COVID-19. Counteracting the underlying motivations for vaccination hesitancy is therefore crucial to overcome this worldwide crisis.

    Although the reasons for vaccination hesitancy are diverse and complex, concerns about potential adverse events (AEs) from the COVID-19 vaccines seem to be a major factor.6 According to a global survey from January 2021, 47% of respondents were worried about AEs from a COVID-19 vaccine.7 With regard to influenza vaccination, there is broad evidence of an association between concerns about AEs and vaccination refusal.8-12 Several systematic reviews of randomized clinical drug trials have demonstrated that the occurrence of AEs can also be substantial in placebo arms.13 Adverse events seemingly elicited by placebos are often called nocebo responses14 and are thought to be caused by misattribution of routine background symptoms,15 anxiety,16 and expectations of AEs.17,18 Emerging research has shown that informing patients about nocebo responses19,20 and providing a positive framing of potential AEs21-24 may be associated with reduced AE-related anxiety and nocebo responses. Although nocebo phenomena have been investigated in many contexts involving medication,18,25-28 evidence of their influence in vaccination remains scarce. However, a recent meta-analysis suggested that a significant proportion of placebo recipients in influenza vaccine trials experienced systemic AEs, such as headache or fatigue, owing to nocebo responses.29

    Researchers of nocebo response have called attention to ubiquitous nocebo responses in COVID-19 vaccination,30,31 but systematic quantification is needed. The current systematic review and meta-analysis aimed to assess the frequencies of AEs reported in the placebo groups of COVID-19 vaccine trials and compare them with the frequencies of AEs reported in the vaccine groups.

    Methods

    This systematic review and meta-analysis followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.32 The study protocol has been registered in PROSPERO (CRD42021256905).

    Eligibility Criteria

    We included randomized clinical trials of experimental COVID-19 vaccines that investigated adults aged 16 years or older and were published in English. Studies were eligible if they assessed solicited AEs within 7 days of injection, included an inert placebo arm (eg, saline), and provided AE reports for both the vaccine and the placebo groups separately. Studies were excluded if the applied placebo contained any active ingredient that could have caused specific AEs (eg, an adjuvant) or if risk of bias was judged as unknown.

    Search Strategy

    A systematic literature search of studies published up to July 14, 2021, was conducted across the Medline database (PubMed) and the Cochrane Central Register of Controlled Trials (CENTRAL). These databases were searched for (1) medical subheading terms and (2) free-text keywords (eAppendix in the Supplement).

    Study Selection and Assessment

    After titles and abstracts were screened for initial eligibility by J.W.H., 2 of us (J.W.H. and F.L.B.) independently reviewed the full texts of potentially eligible articles for inclusion and exclusion criteria. Data extraction and quality assessment were performed independently by J.W.H and F.L.B. Any discrepancies were resolved through discussion and, if necessary, evaluation by a third team member (S.B.).

    Only AEs in terms of solicited reactogenicity symptoms were derived. Long-term observations or follow-up data were not considered as outcomes in the meta-analysis, and neither were serious AEs. Because all included trials investigated 2-dose schemes of vaccination, data on AEs after the first dose and on those after the second dose were extracted separately. If data on AEs were not provided in sufficient detail in the published material, the authors were emailed with a request. For studies that investigated different dosing schemes within 1 trial, only 1 of the vaccine groups was selected for comparison with the placebo group. To ensure practical relevance, we always selected the dose that was chosen for clinical application or described as the preferred dose for further investigation. The quality of the methods and the internal validity of each included study were rated based on predefined criteria covering the 7 categories of the Cochrane risk-of-bias tool.33

    Statistical Analysis

    Data management and statistical analyses were performed using Microsoft Excel (Microsoft Corporation)34 and the R-based software JASP, version 0.14.1.0 (University of Amsterdam).35 Adverse events in different trials were summarized into homogeneous categories, combining AEs that were considered to measure the same symptom (eg, arthralgia and joint pain) or represented symptoms closely related to each other (eg, nausea and vomiting). Overall AE categories were any AE, any local (injection-site) AE, and any systemic AE. Adverse event categories had to be reported in at least 4 trials to be included in the meta-analysis. In addition to the analysis of the first and second doses separately, data on both doses were merged by calculating means if studies did not provide overall AE data. This may have underestimated true AE frequencies and was therefore considered to have increased risk of bias.

    To examine AE frequencies in the placebo groups, the proportions of participants experiencing the respective symptoms were calculated. To compare AE frequencies between the groups, logarithmic odds ratios (log ORs) were calculated. Log ORs were also calculated to compare frequencies of AEs after the first and second doses within the groups. Significance was set at 2-sided P < .05. Meta-analyses were performed based on a restricted maximum-likelihood model (random effects) including 95% CIs. For the categories of any systemic AE and any local AE, ratios between the pooled placebo and vaccine AE proportions were used to calculate the percentage of AEs that were accounted for by nocebo responses. The pooled log ORs were tested for significant divergence from 0 using z tests; for the presence of adverse events, no statistical tests were done because for proportions, the null hypothesis (ie, that there were no cases in the population) could be rejected whenever a single case was reported. Heterogeneity was tested using Q tests and quantified using I2 tests. A variation in outcome (I2) greater than 50% was considered to derive from heterogeneity.36 Funnel plots were visually screened for asymmetry to detect publication bias,37 although publication bias was considered less relevant to a meta-analysis assessing nocebo responses than to one assessing the efficacy of a drug.28 To control for differences in the quality of the methods as identified by the risk-of-bias assessment, all analyses were rerun including the 3 heterogeneous risk-of-bias categories as factors. Owing to the small number of included trials, the results of these mixed-model analyses may be less reliable than those of the primary analyses and are therefore presented in eTable 2 in the Supplement.

    Because data on AE severity grading were not reported in comparable detail by a sufficient number of trials to be analyzed meta-analytically, we descriptively explored the 2 largest trials that provided these data in detail.38,39 In these exploratory analyses, we calculated the proportions of severity grades in participants experiencing any AEs, any local AEs, and any systemic AEs after the first and second doses of placebo or vaccine separately.

    Results
    Search Results and Study Characteristics

    Of 87 screened articles, 27 publications were included for full-text review (Figure 1). Twelve of these studies38-49 were included in our analyses (Table 1), resulting in a total of 45 380 participants; 22 578 placebo recipients and 23 817 active-vaccine recipients provided AE reports, but owing to multiple vaccine groups in some trials,40-46 only 22 802 of the vaccine recipients were included in our analyses. The numbers and percentages of AEs reported in the placebo groups and vaccine groups selected for comparison are provided in eTable 1 in the Supplement.

    The risk-of-bias assessment revealed that the included publications were of high quality in general. In all included studies, the risk of bias was low with regard to randomization, blinding, and outcome assessment. However, the risk of bias was greater in several studies40-43 owing to the inclusion of sentinel participants (enrolled and evaluated before the other participants for safety purposes). One study44 reported a high dropout rate (16%) in the placebo group only. Another study45 provided AE data on the first and second doses within a 14-day period instead of 7 days. One study50 was excluded owing to an unknown risk of bias in the outcome assessment.

    Proportions of AEs

    The random-effects pooled proportion of placebo recipients reporting at least 1 systemic AE after the first dose was 35.2% (95% CI, 26.7%-43.7%); 16.2% (95% CI, 11.3%-21.1%) reported at least 1 local AE (Table 2). In comparison, patients treated with vaccines reported higher AE rates, with 46.3% (95% CI, 38.2%-54.3%) reporting at least 1 systemic AE and 66.7% (95% CI, 53.2%-80.3%) reporting at least 1 local AE. The ratios between the placebo and vaccine AE proportions suggest that after the first vaccine dose, nocebo responses accounted for 76.0% of systemic AEs (Figure 2) and 24.3% of local AEs.

    After the second dose, AE proportions in placebo groups were lower, with 31.8% (95% CI, 28.7%-35.0%) of participants reporting any systemic AEs and 11.8% (95% CI, 6.6%-17.1%) reporting any local AEs (Table 2). These differences in AE rates between the first and second doses within the placebo groups were statistically significant (any systemic AE: log OR, 0.33 [95% CI, 0.18-0.47]; SE, 0.08; z, 4.29; P < .001; any local AE: log OR, 0.22 [95% CI, 0.08-0.36]; SE, 0.07; z, 3.06; P = .002). However, AE proportions in the vaccine groups were greater after the second dose than after the first, with 61.4% (95% CI, 47.4%-75.4%) of participants reporting any systemic AEs and 72.8% (95% CI, 57.4%-88.2%) reporting any local AEs. The differences between the first and second doses within the vaccine groups were statistically significant for systemic AEs (log OR, −0.71 [95% CI, −1.16 to −0.26]; SE, 0.23; z, −3.09, P = .002) but not for local AEs (log OR, −0.29 [95% CI, −0.73 to 0.14]; SE, 0.22; z, −1.33; P = .18).

    Thus, compared with the first dose, a larger difference in AE rates between the placebo groups and vaccine groups was found after the second dose (Figure 2). Nevertheless, ratios between AE proportions in the placebo and vaccine groups indicated that nocebo responses accounted for 51.8% of systemic and 16.2% of local AEs after the second dose. The most commonly reported AEs in the placebo groups were headache (first dose: 19.3% [95% CI, 13.6%-25.1%]; second dose: 16.2% [95% CI, 12.5%-19.8%]) and fatigue (first dose: 16.7% [95% CI, 9.8%-23.6%]; second dose: 14.9% [95% CI, 9.8%-20.1%]). Heterogeneity of the included studies was very high (I2≥50%) for most AE categories.

    Mixed-model analyses indicated that the proportions of placebo recipients with AEs may have been higher when controlling for risk-of-bias variables, with 40.5% (95% CI, 32.9%-48.2%) of participants reporting at least 1 systemic AE after the first dose (eTable 2 in the Supplement). Funnel plots did not show asymmetry of data points, but heterogeneity was still very high for most AE categories.

    Effect Sizes of Group Differences

    Most random-effects pooled log ORs were statistically significant, suggesting that AE rates were significantly higher in the vaccine groups compared with the placebo groups (Table 3). After the first dose, the pooled log OR for any systemic AE was −0.47 (95% CI, −0.54 to −0.40; P < .001), which is equivalent to a standardized mean difference of −0.26 (95% CI, −0.30 to −0.22). For any local AE after the first dose, the pooled log OR was −2.44 (95% CI, −3.21 to −1.66), which is equivalent to a standardized mean difference of −1.34 (95% CI, −1.77 to −0.92). Thus, the difference between placebo and vaccine recipients experiencing any local AE after the first dose equated to a large effect, whereas it equated to only a small effect for any systemic AE. Effect sizes were not significant for nausea and diarrhea, and they were small for fatigue, malaise, joint pain, and headache (Table 3), suggesting only minor differences between the placebo and vaccine groups in the experience of these AEs after the first dose.

    After the second dose, however, the random-effects pooled log ORs and equivalent standardized mean differences indicated larger differences between the vaccine and placebo groups in reporting of AEs (OR, −1.36; 95% CI, −1.86 to −0.86; P < .001). Except for nausea and diarrhea, effects were large or very large for all AE categories (Table 3), with a standardized mean difference of −0.75 (95% CI, 1.03 to −0.47) for any systemic AE and −1.74 (95% CI, −2.16 to −1.32) for any local AE. Heterogeneity of the included studies was very high (I2≥50%) for most AE categories.

    The results of the equivalent log OR meta-analysis controlling for methodologic quality aspects are provided in the eTable 3 in the Supplement. There were no major differences in the results when controlling for risk of bias, and heterogeneity remained high.

    Severity of AEs

    Aside from the presence or absence of AEs, the severity of AEs could serve as an additional indicator to quantify the influence of nocebo responses. In exploratory analyses of the 2 largest trials reporting details on severity,38,39 we found that the proportion of severity grades for participants reporting any systemic AEs after the first dose were similar in the placebo and vaccine groups (eFigures 1 and 2 in the Supplement). However, whereas the pattern of severity grading after the second dose stayed the same for placebo participants, there were proportionally more moderate and severe AEs in the vaccine groups after the second dose.

    Discussion

    This systematic review and meta-analysis evaluated the frequency of solicited AEs in the placebo groups of randomized clinical trials investigating COVID-19 vaccines. The 12 analyzed trials38-49 included different types of vaccines (mRNA, viral vector, or protein-based) and different clinical trial phases. We found that 76.0% of systemic AEs and 24.3% of local AEs after the first vaccination could be attributed to nocebo responses. After the second vaccination, 51.8% of systemic AEs and 16.2% of local AEs were attributable to nocebo responses. Headache and fatigue were the most common AEs in the placebo groups, experienced by 19.3% and 16.7% of participants, respectively, after the first dose.

    Of interest, AE frequencies in the placebo groups were lower after the second dose than after the first dose, although the opposite was true for the vaccine groups. We hypothesize that (1) the second dose of the vaccines may have produced both a more robust immune response and a correspondingly more robust set of AEs and that (2) participants in the vaccine arms, after experiencing more AEs after the first dose than did participants in the control groups, had higher expectations for AEs after the second dose compared with participants in the placebo arms.51

    The finding of decreased AEs in placebo recipients but increased AEs in vaccine recipients after the second dose was supported by our evaluation of standardized mean differences. Although we found significantly higher AE rates in the vaccine groups for nearly every evaluated symptom category, group differences were particularly large after the second dose. After the first dose, however, the group differences for most systemic AEs were small. Headache, fatigue, malaise, and joint pain were common in both groups and seem to have been particularly associated with nocebo. Furthermore, exploratory analyses suggested that nocebo responses may produce AEs of severity grades similar to those of active vaccines after the first dose.

    Healthcare Implications

    Given the large number of people who have received or will receive a COVID-19 vaccine, this study’s findings are important for the general population worldwide. Common nonspecific symptoms such as headache and fatigue, which the study’s findings showed to be particularly associated with nocebo, are listed among the most common AEs after COVID-19 vaccination in many information leaflets.52-54 There is evidence that this sort of information may increase nocebo mechanisms such as AE-related anxiety and expectations.17 Furthermore, the information might cause a misattribution of commonly experienced nonspecific symptoms (eg, headache or fatigue55) as specific AEs due to vaccination, even if these symptoms might have occurred in the absence of receiving any treatment.56 Thus, the current way of informing the public about potential vaccine AEs via leaflets and in the media may prompt or further increase nocebo responses.30 Nonetheless, it is ethically necessary to fully inform participants about the vaccines’ potential AEs. Emergent data suggest that full disclosure and education about nocebo responses may be helpful.19,20 For example, adding simple but accurate information about nocebo responses to the usual informed consent procedure (eg, “participants in the placebo arm of the randomized clinical trials testing this intervention reported similar AEs, probably because of worry and anxiety”) helped reduce medication-related AEs in a clinical population.20 Highlighting the probability of not experiencing AEs might also be beneficial.21 Although more research on these communication strategies is needed, such honest information adds to full disclosure and is unlikely to cause harm. In addition, informing the public about the potential for nocebo responses may help reduce worries about COVID-19 vaccination, which might decrease vaccination hesitancy.9,31

    Limitations

    This study has limitations. The relatively small number of included trials and their high heterogeneity must be considered when interpreting the results. The high heterogeneity may have been caused by (1) different AE assessment methods (ie, the trials used different symptom checklists and did not assess the attributability of symptoms in a standardized way), (2) different types of tested vaccine (mRNA, viral vector, or protein-based), or (3) different probabilities of receiving a vaccine vs a placebo in the different trials (ie, probabilities to receive a placebo ranged from 14% to 50%). Standardization of AE assessment could alleviate the first of these potential causes,57 and larger meta-analyses should investigate the role of the second and third causes using meta-regression. However, because our analyses included randomized clinical trials of different clinical trial phases and different vaccine manufacturing types that are currently used in practice, we believe the findings of high nocebo responses are relevant for COVID-19 vaccination in everyday health care.

    Conclusions

    In this systematic review and meta-analysis, approximately one-third of placebo recipients in COVID-19 vaccine randomized clinical trials reported at least 1 systemic AE after both the first and the second dose, with headache and fatigue being the most common. This nocebo response accounted for 76.0% of systemic AEs after the first dose of COVID-19 vaccine, and for 51.8% after the second dose. Public vaccination programs should consider these high nocebo responses.

    Back to top
    Article Information

    Accepted for Publication: November 22, 2021.

    Published: January 18, 2022. doi:10.1001/jamanetworkopen.2021.43955

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Haas JW et al. JAMA Network Open.

    Correction: This article was corrected on February 11, 2022, to fix an error in Figure 1.

    Corresponding Author: Julia W. Haas, PhD, Program in Placebo Studies, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Ste 1309, Boston, MA 02215 (juliawhaas@gmail.com).

    Author Contributions: Dr Haas and Ms Bender contributed equally to this study. Dr Rief and Prof Kaptchuk contributed equally to this study. Dr Haas and Ms Bender had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Haas, Bender, Ballou, Wilhelm, Miller, Kaptchuk.

    Acquisition, analysis, or interpretation of data: Haas, Bender, Kelley, Rief, Kaptchuk.

    Drafting of the manuscript: Haas, Bender, Kaptchuk.

    Critical revision of the manuscript for important intellectual content: Bender, Ballou, Kelley, Wilhelm, Miller, Rief, Kaptchuk.

    Statistical analysis: Haas, Bender, Kelley, Wilhelm.

    Obtained funding: Haas.

    Administrative, technical, or material support: Rief.

    Supervision: Ballou, Kelley, Wilhelm, Rief, Kaptchuk.

    Conflict of Interest Disclosures: Dr Haas reported receiving a postdoctoral scholarship from the German Academic Exchange Service (Deutscher Akademischer Austauschdienst) during the conduct of the study. No other disclosures were reported.

    References
    1.
    World Health Organisation. WHO coronavirus (COVID-19) dashboard. 2021. Accessed November 10, 2021. https://covid19.who.int/
    2.
    Khubchandani  J, Sharma  S, Price  JH, Wiblishauser  MJ, Sharma  M, Webb  FJ.  COVID-19 vaccination hesitancy in the United States: a rapid national assessment.   J Community Health. 2021;46(2):270-277. doi:10.1007/s10900-020-00958-x PubMedGoogle ScholarCrossref
    3.
    Razai  MS, Chaudhry  UAR, Doerholt  K, Bauld  L, Majeed  A.  COVID-19 vaccination hesitancy.   BMJ. 2021;373(n1138):n1138. doi:10.1136/bmj.n1138 PubMedGoogle Scholar
    4.
    Robinson  E, Jones  A, Lesser  I, Daly  M.  International estimates of intended uptake and refusal of COVID-19 vaccines: a rapid systematic review and meta-analysis of large nationally representative samples.   Vaccine. 2021;39(15):2024-2034. doi:10.1016/j.vaccine.2021.02.005 PubMedGoogle ScholarCrossref
    5.
    World Health Organization. Ten threats to global health in 2019. Accessed September 1, 2021. https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019
    6.
    Taylor  S, Landry  CA, Paluszek  MM, Groenewoud  R, Rachor  GS, Asmundson  GJG.  A proactive approach for managing COVID-19: the importance of understanding the motivational roots of vaccination hesitancy for SARS-CoV2.   Front Psychol. 2020;11:575950. doi:10.3389/fpsyg.2020.575950 PubMedGoogle Scholar
    8.
    Smith  LE, Amlôt  R, Weinman  J, Yiend  J, Rubin  GJ.  Why do parents not re-vaccinate their child for influenza? a prospective cohort study.   Vaccine. 2020;38(27):4230-4235. doi:10.1016/j.vaccine.2020.04.029 PubMedGoogle ScholarCrossref
    9.
    Schmid  P, Rauber  D, Betsch  C, Lidolt  G, Denker  ML.  Barriers of influenza vaccination intention and behavior—a systematic review of influenza vaccine hesitancy, 2005-2016.   PLoS One. 2017;12(1):e0170550. doi:10.1371/journal.pone.0170550 PubMedGoogle Scholar
    10.
    Ramsey  MA, Marczinski  CA.  College students’ perceptions of H1N1 flu risk and attitudes toward vaccination.   Vaccine. 2011;29(44):7599-7601. doi:10.1016/j.vaccine.2011.07.130 PubMedGoogle ScholarCrossref
    11.
    Gorman  JR, Brewer  NT, Wang  JB, Chambers  CD.  Theory-based predictors of influenza vaccination among pregnant women.   Vaccine. 2012;31(1):213-218. doi:10.1016/j.vaccine.2012.10.064 PubMedGoogle ScholarCrossref
    12.
    Sun  KS, Lam  TP, Kwok  KW, Lam  KF, Wu  D, Ho  PL.  Seasonal influenza vaccine uptake among Chinese in Hong Kong: barriers, enablers and vaccination rates.   Hum Vaccin Immunother. 2020;16(7):1675-1684. doi:10.1080/21645515.2019.1709351 PubMedGoogle ScholarCrossref
    13.
    Howick  J, Webster  R, Kirby  N, Hood  K.  Rapid overview of systematic reviews of nocebo effects reported by patients taking placebos in clinical trials.   Trials. 2018;19(1):674. doi:10.1186/s13063-018-3042-4 PubMedGoogle ScholarCrossref
    14.
    Barsky  AJ, Saintfort  R, Rogers  MP, Borus  JF.  Nonspecific medication side effects and the nocebo phenomenon.   JAMA. 2002;287(5):622-627. doi:10.1001/jama.287.5.622 PubMedGoogle ScholarCrossref
    15.
    Rief  W, Barsky  AJ, Glombiewski  JA, Nestoriuc  Y, Glaesmer  H, Braehler  E.  Assessing general side effects in clinical trials: reference data from the general population.   Pharmacoepidemiol Drug Saf. 2011;20(4):405-415. doi:10.1002/pds.2067 PubMedGoogle ScholarCrossref
    16.
    Benedetti  F, Lanotte  M, Lopiano  L, Colloca  L.  When words are painful: unraveling the mechanisms of the nocebo effect.   Neuroscience. 2007;147(2):260-271. doi:10.1016/j.neuroscience.2007.02.020 PubMedGoogle ScholarCrossref
    17.
    Amanzio  M, Corazzini  LL, Vase  L, Benedetti  F.  A systematic review of adverse events in placebo groups of anti-migraine clinical trials.   Pain. 2009;146(3):261-269. doi:10.1016/j.pain.2009.07.010 PubMedGoogle ScholarCrossref
    18.
    Rief  W, Nestoriuc  Y, von Lilienfeld-Toal  A,  et al.  Differences in adverse effect reporting in placebo groups in SSRI and tricyclic antidepressant trials: a systematic review and meta-analysis.   Drug Saf. 2009;32(11):1041-1056. doi:10.2165/11316580-000000000-00000 PubMedGoogle ScholarCrossref
    19.
    Pan  Y, Kinitz  T, Stapic  M, Nestoriuc  Y.  Minimizing drug adverse events by informing about the nocebo effect—an experimental study.   Front Psychiatry. 2019;10:504. doi:10.3389/fpsyt.2019.00504 PubMedGoogle ScholarCrossref
    20.
    Ballou  S, Iturrino  J, Rangan  V,  et al.  Improving medication tolerance: a pilot study in disorders of gut-brain interaction treated with tricyclic antidepressants.   J Clin Gastroenterol. 2021;(June). doi:10.1097/MCG.0000000000001575 PubMedGoogle Scholar
    21.
    Faasse  K, Huynh  A, Pearson  S, Geers  AL, Helfer  SG, Colagiuri  B.  The influence of side effect information framing on nocebo effects.   Ann Behav Med. 2019;53(7):621-629. doi:10.1093/abm/kay071 PubMedGoogle ScholarCrossref
    22.
    Wilhelm  M, Rief  W, Doering  BK.  Decreasing the burden of side effects through positive message framing: an experimental proof-of-concept study.   Int J Behav Med. 2018;25(4):381-389. doi:10.1007/s12529-018-9726-z PubMedGoogle ScholarCrossref
    23.
    O’Connor  AM, Pennie  RA, Dales  RE.  Framing effects on expectations, decisions, and side effects experienced: the case of influenza immunization.   J Clin Epidemiol. 1996;49(11):1271-1276. doi:10.1016/S0895-4356(96)00177-1 PubMedGoogle ScholarCrossref
    24.
    Howe  LC, Leibowitz  KA, Perry  MA,  et al.  Changing patient mindsets about non-life-threatening symptoms during oral immunotherapy: a randomized clinical trial.   J Allergy Clin Immunol Pract. 2019;7(5):1550-1559. doi:10.1016/j.jaip.2019.01.022 PubMedGoogle ScholarCrossref
    25.
    Mitsikostas  DD, Mantonakis  L, Chalarakis  N.  Nocebo in clinical trials for depression: a meta-analysis.   Psychiatry Res. 2014;215(1):82-86. doi:10.1016/j.psychres.2013.10.019 PubMedGoogle ScholarCrossref
    26.
    Benedetti  F, Amanzio  M, Vighetti  S, Asteggiano  G.  The biochemical and neuroendocrine bases of the hyperalgesic nocebo effect.   J Neurosci. 2006;26(46):12014-12022. doi:10.1523/JNEUROSCI.2947-06.2006 PubMedGoogle ScholarCrossref
    27.
    Palermo  S, Giovannelli  F, Bartoli  M, Amanzio  M.  Are patients with schizophrenia spectrum disorders more prone to manifest nocebo-like-effects? a meta-analysis of adverse events in placebo groups of double-blind antipsychotic trials.   Front Pharmacol. 2019;10:502. doi:10.3389/fphar.2019.00502 PubMedGoogle ScholarCrossref
    28.
    Mahr  A, Golmard  C, Pham  E, Iordache  L, Deville  L, Faure  P.  Types, frequencies, and burden of nonspecific adverse events of drugs: analysis of randomized placebo-controlled clinical trials.   Pharmacoepidemiol Drug Saf. 2017;26(7):731-741. doi:10.1002/pds.4169 PubMedGoogle ScholarCrossref
    29.
    Bender  F, Rief  W, Wilhelm  M. Really just a little prick? a meta-analysis on adverse events in placebo control groups of seasonal influenza vaccination RCTs. Abstract #O1505. Third International Conference of the Society for Interdisciplinary Placebo Studies (SIPS); May 26-28, 2021; Baltimore, MD.
    30.
    Amanzio  M, Cipriani  GE, Bartoli  M.  How do nocebo effects in placebo groups of randomized controlled trials provide a possible explicative framework for the COVID-19 pandemic?   Expert Rev Clin Pharmacol. 2021;14(4):439-444. doi:10.1080/17512433.2021.1900728 PubMedGoogle ScholarCrossref
    31.
    Rief  W. Fear of adverse effects and COVID-19 vaccine hesitancy: recommendations of the treatment expectation expert group. JAMA Heal Forum. 2021;2(4):e210804. doi:10.1001/jamahealthforum.2021.0804
    32.
    Page  MJ, McKenzie  JE, Bossuyt  PM,  et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.   BMJ. 2021;372(n71):n71. doi:10.1136/bmj.n71 PubMedGoogle Scholar
    33.
    Higgins  JPT, Altman  DG, Gøtzsche  PC,  et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.   BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMedGoogle ScholarCrossref
    34.
    Microsoft Excel. Microsoft Corporation; 2018. Accessed August 16, 2019. https://office.microsoft.com/excel
    35.
    JASP. Version 0.14.1. JASP; 2020. Accessed May 21, 2021. https://jasp-stats.org/download/
    36.
    Higgins  JPT, Thompson  SG.  Quantifying heterogeneity in a meta-analysis.   Stat Med. 2002;21(11):1539-1558. doi:10.1002/sim.1186 PubMedGoogle ScholarCrossref
    37.
    Duval  S, Tweedie  R.  Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis.   Biometrics. 2000;56(2):455-463. doi:10.1111/j.0006-341X.2000.00455.x PubMedGoogle ScholarCrossref
    38.
    Baden  LR, El Sahly  HM, Essink  B,  et al; COVE Study Group.  Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.   N Engl J Med. 2020;384(5):403-416. doi:10.1056/NEJMoa2035389 PubMedGoogle ScholarCrossref
    39.
    Heath  PT, Galiza  EP, Baxter  DN,  et al; 2019nCoV-302 Study Group.  Safety and efficacy of NVX-CoV2373 COVID-19 vaccine.   N Engl J Med. 2021;385(13):1172-1183. doi:10.1056/NEJMoa2107659 PubMedGoogle ScholarCrossref
    40.
    Walsh  EE, Frenck  RWJ  Jr, Falsey  AR,  et al.  Safety and immunogenicity of two RNA-based COVID-19 vaccine candidates.   N Engl J Med. 2020;383(25):2439-2450. doi:10.1056/NEJMoa2027906 PubMedGoogle ScholarCrossref
    41.
    Richmond  P, Hatchuel  L, Dong  M,  et al.  Safety and immunogenicity of S-Trimer (SCB-2019), a protein subunit vaccine candidate for COVID-19 in healthy adults: a phase 1, randomised, double-blind, placebo-controlled trial.   Lancet. 2021;397(10275):682-694. doi:10.1016/S0140-6736(21)00241-5 PubMedGoogle ScholarCrossref
    42.
    Sadoff  J, Le Gars  M, Shukarev  G,  et al.  Interim results of a phase 1-2a trial of Ad26.COV2.S COVID-19 vaccine.   N Engl J Med. 2021;384(19):1824-1835. doi:10.1056/NEJMoa2034201 PubMedGoogle ScholarCrossref
    43.
    Chu  L, McPhee  R, Huang  W,  et al; mRNA-1273 Study Group.  A preliminary report of a randomized controlled phase 2 trial of the safety and immunogenicity of mRNA-1273 SARS-CoV-2 vaccine.   Vaccine. 2021;39(20):2791-2799. doi:10.1016/j.vaccine.2021.02.007 PubMedGoogle ScholarCrossref
    44.
    Keech  C, Albert  G, Cho  I,  et al.  Phase 1-2 trial of a SARS-CoV-2 recombinant spike protein nanoparticle vaccine.   N Engl J Med. 2020;383(24):2320-2332. doi:10.1056/NEJMoa2026920 PubMedGoogle ScholarCrossref
    45.
    Li  J, Hui  A, Zhang  X,  et al.  Safety and immunogenicity of the SARS-CoV-2 BNT162b1 mRNA vaccine in younger and older Chinese adults: a randomized, placebo-controlled, double-blind phase 1 study.   Nat Med. 2021;27(6):1062-1070. doi:10.1038/s41591-021-01330-9 PubMedGoogle ScholarCrossref
    46.
    Goepfert  PA, Fu  B, Chabanon  A-L,  et al.  Safety and immunogenicity of SARS-CoV-2 recombinant protein vaccine formulations in healthy adults: interim results of a randomised, placebo-controlled, phase 1-2, dose-ranging study.   Lancet Infect Dis. 2021;21(9):1257-1270. doi:10.1016/S1473-3099(21)00147-X PubMedGoogle ScholarCrossref
    47.
    Madhi  SA, Baillie  V, Cutland  CL,  et al; NGS-SA Group; Wits-VIDA COVID Group.  Efficacy of the ChAdOx1 nCoV-19 COVID-19 vaccine against the B.1.351 variant.   N Engl J Med. 2021;384(20):1885-1898. doi:10.1056/NEJMoa2102214 PubMedGoogle ScholarCrossref
    48.
    Polack  FP, Thomas  SJ, Kitchin  N,  et al; C4591001 Clinical Trial Group.  Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine.   N Engl J Med. 2020;383(27):2603-2615. doi:10.1056/NEJMoa2034577 PubMedGoogle ScholarCrossref
    49.
    Shinde  V, Bhikha  S, Hoosain  Z,  et al; 2019nCoV-501 Study Group.  Efficacy of NVX-CoV2373 COVID-19 vaccine against the B.1.351 variant.   N Engl J Med. 2021;384(20):1899-1909. doi:10.1056/NEJMoa2103055 PubMedGoogle ScholarCrossref
    50.
    Chappell  KJ, Mordant  FL, Li  Z,  et al.  Safety and immunogenicity of an MF59-adjuvanted spike glycoprotein-clamp vaccine for SARS-CoV-2: a randomised, double-blind, placebo-controlled, phase 1 trial.   Lancet Infect Dis. 2021;21(10):1383-1394. doi:10.1016/S1473-3099(21)00200-0 PubMedGoogle ScholarCrossref
    51.
    Rief  W, Glombiewski  JA.  The hidden effects of blinded, placebo-controlled randomized trials: an experimental investigation.   Pain. 2012;153(12):2473-2477. doi:10.1016/j.pain.2012.09.007 PubMedGoogle ScholarCrossref
    52.
    Centers for Disease Control and Prevention. Moderna COVID-19 vaccine overview and safety. Updated November 19, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/Moderna.html
    53.
    Centers for Disease Control and Prevention. Pfizer-BioNTech COVID-19 vaccine (also known as COMIRNATY) overview and safety. Updated November 19, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/Pfizer-BioNTech.html
    54.
    Centers for Disease Control and Prevention. Johnson & Johnson’s Janssen COVID-19 vaccine overview and safety. Updated October 29, 2021. Accessed August 16, 2021. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/janssen.html
    55.
    Reidenberg  MM, Lowenthal  DT.  Adverse nondrug reactions.   N Engl J Med. 1968;279(13):678-679. doi:10.1056/NEJM196809262791304 PubMedGoogle ScholarCrossref
    56.
    Wells  RE, Kaptchuk  TJ.  To tell the truth, the whole truth, may do patients harm: the problem of the nocebo effect for informed consent.   Am J Bioeth. 2012;12(3):22-29. doi:10.1080/15265161.2011.652798 PubMedGoogle ScholarCrossref
    57.
    Rief  W, Glombiewski  JA, Barsky  AJ. Generic assessment of side effects. 2009. Accessed June 21, 2021. http://www.gase-scale.com/Instruction_GASE.pdf
    ×