[Skip to Navigation]
Sign In
September 3, 2021

Safety Surveillance of COVID-19 mRNA Vaccines Through the Vaccine Safety Datalink

Author Affiliations
  • 1Division of Rheumatology, Allergy and Immunology, Massachusetts General Hospital, Boston
  • 2Harvard Medical School, Boston, Massachusetts
  • 3Mongan Institute, Massachusetts General Hospital, Boston
  • 4Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
  • 5Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
JAMA. 2021;326(14):1375-1377. doi:10.1001/jama.2021.14808

Vaccines represent one of the greatest public health achievements of modern medicine. They must pass rigorous and predetermined efficacy and safety metrics prior to licensure. Additionally, postmarketing safety surveillance is essential to detect rare or severe vaccine-associated adverse events, particularly because of the large numbers of individuals exposed.

A primary method for postmarketing vaccine safety surveillance is voluntary reporting through the Vaccine Adverse Event Reporting System (VAERS), cosponsored by the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA). However, voluntary reporting has limitations, such as case underreporting. The sensitivity of VAERS for capturing anaphylaxis ranged from 13% to 76% and Guillain-Barré syndrome from 12% to 64% for different vaccines.1 Additional concerns with VAERS data include challenges with determination of causality between the vaccine and the reported event and the lack of a comparison group to assess excess vs baseline risk in a given population. A 2011 report from the Institute of Medicine found inadequate evidence to accept or reject a causal relationship for 85% of vaccine–adverse event pairings studied.2

To address these issues, the CDC Immunization Safety Office created the Vaccine Safety Datalink (VSD) project to conduct postmarketing evaluations of vaccine safety within a defined population. A primary analytic approach for postmarketing surveillance by VSD is rapid cycle analysis (RCA), whereby the observed number of adverse events is compared with the expected number of events, with the expected number of events determined from prior data, a concurrent comparison control group, or self-control methods. Weekly VSD comparisons assess many safety outcomes of interest; as such, the threshold for statistical significance is adjusted to account for multiple outcomes and multiple data assessments. Studies by VSD using RCA were conducted previously for rotavirus, acellular diphtheria-tetanus-pertussis, and meningococcal conjugate vaccines.3,4

In this issue of JAMA, Klein et al5 apply RCA to assess the safety of the mRNA COVID-19 vaccines (BNT162b2 by Pfizer-BioNTech and mRNA-1273 by Moderna) from December 2020 through June 2021. The exposure of interest was receipt of an mRNA COVID-19 vaccine, from 1 to 21 days after dose 1 or dose 2, of vaccine-eligible members of 8 US health plans that participated in the VSD. The expected number of events was determined by a concurrent comparison control group, primarily members who were vaccinated 22 to 42 days after their recent dose, but a secondary comparison control group also included unvaccinated members. Comparators were in the same age, sex, race and ethnicity, and site stratum as the vaccinated cases. Twenty-three serious outcomes were considered, selected based on data from the phase 3 clinical trial results for the mRNA vaccines (eg, Bell palsy, appendicitis), COVID-19 disease concerns (eg, acute myocardial infarction, acute respiratory distress syndrome), and other historical vaccine safety concerns (eg, Guillain-Barré syndrome, anaphylaxis). Each outcome was identified through diagnostic code algorithms that were supported based on prior studies, expert opinion, or both. Cases of Guillain-Barré, acute disseminated encephalomyelitis, transverse myelitis, cerebral venous sinus thrombosis, and myocarditis/pericarditis required confirmation by health record review for inclusion. There was a predetermined significance threshold of a 1-sided P value <0.0048 to keep type I error below .05 considering weekly analyses over the planned 2-year surveillance period.

For 4 other outcomes without appropriate comparators (anaphylaxis, acute respiratory distress syndrome, multisystem inflammatory syndrome, and narcolepsy), descriptive analyses were performed considering cases within 84 days postvaccination. Potential anaphylaxis cases were considered on vaccination days 0 and 1 only and required medical record review to exclude other allergens and apply the Brighton Collaboration criteria with level 1, 2, or 3 considered confirmed anaphylaxis.

Across almost 12 million mRNA COVID-19 vaccine doses (57% Pfizer-BioNTech, 43% Moderna) administered to 6.2 million individuals aged 12 years or older (54% female; mean age, 49 years; 15% Asian, 5% Black non-Hispanic, 22% Hispanic/Latino, and 43% White non-Hispanic), no outcomes met the prespecified signaling criteria for statistical significance. Rate ratios (RRs) were largest for thrombotic thrombocytopenic purpura (2.60), cerebral venous sinus thrombosis (1.55), and transverse myelitis (1.45), but these measures of association had wide 95% CIs and nonsignificant P values. The RR for venous thromboembolism in the risk interval vs comparison interval was 1.16 (95% CI, 1.00 to 1.34); however, the 2-sided P value of .05 was not significant in this study given the predetermined threshold. The highest estimates of excess cases per million doses were 7.5 (95% CI, −0.1 to 14.0) for venous thromboembolism, 1.2 (95% CI, −6.9 to 8.3) for acute myocardial infarction, and 1.2 (95% CI, −2.1 to 3.3) for myocarditis/pericarditis. Venous thromboembolism was linked with viral vector COVID-19 vaccines, but a link to mRNA vaccines is limited to case reports.6,7

While mRNA vaccination was not associated with an increased risk of myocarditis/pericarditis overall, mRNA vaccination was associated with excess risk of myocarditis/pericarditis among those aged 12 to 39 years with an estimated 6.3 (95% CI, 4.9 to 6.8) additional cases per million doses in days 0 through 7 after vaccination. Although prior studies have described myocarditis developing rapidly in younger patients, mostly after vaccine dose 2, and pericarditis affecting older patients with delayed onset after either dose 1 or dose 2,8 the myocarditis/pericarditis cases in this study (n = 7 myocarditis, n = 6 pericarditis, n = 21 myopericarditis) occurred largely within 5 days of mRNA vaccination (median, 2 days [range, 0-20 days]) and the risk was highest after dose 2, with an excess of 11.2 (95% CI, 8.9 to 12.1) cases per million doses in days 0 through 7 for individuals aged 12 to 39 years. A large Israeli study that assessed the safety of the Pfizer-BioNTech vaccine among patients aged 16 years and older identified an excess risk of myocarditis (RR, 3.24 [95% CI, 1.55 to 12.44], 1 to 5 events/100 000 persons) but not pericarditis in vaccinated vs unvaccinated individuals.9 In the study by Klein et al, the 34 confirmed cases of myocarditis/pericarditis had elevated troponin levels, and many had electrocardiographic changes, cardiac MRI changes, or both. However, just 2 individuals (6%) required intensive care unit care, and consistent with previously described cases,8 all patients survived to hospital discharge. The FDA labels for the mRNA COVID-19 vaccines were revised to indicate a risk of myocarditis/pericarditis in June 2021.

Anaphylaxis was identified in 55 individuals, corresponding to an incidence of 4.8 (95% CI, 3.2 to 6.9) cases per million doses for Pfizer-BioNTech and 5.1 (95% CI, 3.3 to 7.6) cases per million doses for Moderna.5 Anaphylaxis incidence was higher in this study than with VAERS, consistent with prior knowledge regarding improved anaphylaxis case capture with VDS compared with VAERS.1 However, anaphylaxis incidence in the study by Klein et al was still lower than reported in a meta-analysis of the incidence of anaphylaxis across 26 studies with adjudicated anaphylaxis cases (7.91 [95% CI, 4.02 to 15.59] cases per million)10 and lower than a prospective cohort study of health care workers that determined anaphylaxis after mRNA vaccines occurred in 247 per million doses.11 The timing of onset of anaphylaxis was largely within CDC-advised monitoring periods: 15 minutes (65%) and 30 minutes (87%). Most (78%) patients had a history of previous unrelated allergies (eg, vaccines, bee sting); some (36%) had a history of anaphylaxis. One patient had a history of allergy to polyethylene glycol, an excipient potentially implicated in mRNA vaccine anaphylaxis.12 Consistent with prior knowledge, patients who experienced anaphylaxis were mostly female; dose 1 anaphylaxis was more common (82%) than dose 2 anaphylaxis (18%).11,13,14 While 19 cases (35%) of anaphylaxis in the study by Klein et al were classified as high certainty by Brighton Collaboration criteria, recent data suggest that not all allergic symptoms after mRNA vaccination necessitate future mRNA COVID-19 vaccine avoidance.15

Strengths of this study include its study design with 2 different comparison groups and concurrent comparators with similar demographic factors. Another strength of this study is the population of more than 12 million vaccine-eligible members from 8 US health plans with substantial racial and ethnic diversity. The population includes an estimated 3.6% of the US population and 16% of the population aged 65 years and older. Limitations include limited power for rare outcomes as illustrated by wide CIs, even for some outcomes with possibly clinically significant RR estimates. Another limitation is the short at-risk period (ie, 21 days), although vaccine adverse event risk is often highest within this short time frame.2

People in the US have received more than 342 million doses of COVID-19 vaccines, with the vast majority being mRNA vaccines from Pfizer-BioNTech or Moderna. In the study by Klein et al, the mRNA COVID-19 vaccines were safe for the population overall (ie, there was no difference for any of the serious outcomes assessed), but an excess risk of myocarditis/pericarditis was identified for vaccinees aged 12 to 39 years. Anaphylaxis after mRNA vaccination was rare. Additional monitoring by VSD will continue to assess for clinically relevant adverse events associated with mRNA vaccination, including following booster doses. Support for collaborations like VSD, which include detailed data on large and diverse populations, is essential for robust vaccine safety assessments to inform the public and help overcome vaccine hesitancy, particularly in pandemic situations when large-scale vaccination is critical and very large numbers of individuals are exposed to new vaccines.

Back to top
Article Information

Corresponding Author: Kimberly G. Blumenthal, MD, MSc, The Mongan Institute, Massachusetts General Hospital, 100 Cambridge St, 16th Floor, Boston, MA 02114 (kblumenthal@mgh.harvard.edu).

Published Online: September 3, 2021. doi:10.1001/jama.2021.14808

Conflict of Interest Disclosures: Dr Blumenthal reports royalties from UpToDate and grants from NIH, AHRQ, and the Massachusetts General Hospital, outside the submitted work. Dr Phadke reports spousal employment by Chiesi Farmaceutici, outside the submitted work. Dr Bates reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, Clew, and MDClone, personal fees and equity from AESOP, and grants from IBM Watson Health, outside the submitted work.

Miller  ER, McNeil  MM, Moro  PL, Duffy  J, Su  JR.  The reporting sensitivity of the Vaccine Adverse Event Reporting System (VAERS) for anaphylaxis and for Guillain-Barré syndrome.   Vaccine. 2020;38(47):7458-7463. doi:10.1016/j.vaccine.2020.09.072PubMedGoogle ScholarCrossref
Stratton  K, Ford  A, Rusch  E, Clayton  EW, eds.  Adverse Effects of Vaccines: Evidence and Causality. Institute of Medicine; 2012.
Davis  RL, Kolczak  M, Lewis  E,  et al.  Active surveillance of vaccine safety: a system to detect early signs of adverse events.   Epidemiology. 2005;16(3):336-341. doi:10.1097/01.ede.0000155506.05636.a4PubMedGoogle ScholarCrossref
Lieu  TA, Kulldorff  M, Davis  RL,  et al; Vaccine Safety Datalink Rapid Cycle Analysis Team.  Real-time vaccine safety surveillance for the early detection of adverse events.   Med Care. 2007;45(10)(suppl 2):S89-S95. doi:10.1097/MLR.0b013e3180616c0aPubMedGoogle ScholarCrossref
Klein  NP, Lewis  N, Goddard  K,  et al.  Surveillance for adverse events after COVID-19 mRNA vaccination.   JAMA. Published online September 3, 2021. doi:10.1001/jama.2021.15072Google Scholar
Carli  G, Nichele  I, Ruggeri  M, Barra  S, Tosetto  A.  Deep vein thrombosis (DVT) occurring shortly after the second dose of mRNA SARS-CoV-2 vaccine.   Intern Emerg Med. 2021;16(3):803-804. doi:10.1007/s11739-021-02685-0PubMedGoogle ScholarCrossref
Dias  L, Soares-Dos-Reis  R, Meira  J,  et al.  Cerebral venous thrombosis after BNT162b2 mRNA SARS-CoV-2 vaccine.   J Stroke Cerebrovasc Dis. 2021;30(8):105906. doi:10.1016/j.jstrokecerebrovasdis.2021.105906PubMedGoogle Scholar
Diaz  GA, Parsons  GT, Gering  SK, Meier  AR, Hutchinson  IV, Robicsek  A.  Myocarditis and pericarditis after vaccination for COVID-19.   JAMA. 2021. doi:10.1001/jama.2021.13443PubMedGoogle Scholar
Barda  N, Dagan  N, Ben-Shlomo  Y,  et al.  Safety of the BNT162b2 mRNA Covid-19 vaccine in a nationwide setting.   N Engl J Med. Published online August 25, 2021. doi:10.1056/NEJMoa2110475PubMedGoogle Scholar
Greenhawt  M, Abrams  EM, Shaker  M,  et al.  The risk of allergic reaction to SARS-CoV-2 vaccines and recommended evaluation and management: a systematic review, meta-analysis, GRADE assessment, and international consensus approach.   J Allergy Clin Immunol Pract. 2021;S2213-2198(21)00671-1.PubMedGoogle Scholar
Blumenthal  KG, Robinson  LB, Camargo  CA  Jr,  et al.  Acute allergic reactions to mRNA COVID-19 vaccines.   JAMA. 2021;325(15):1562-1565. doi:10.1001/jama.2021.3976PubMedGoogle ScholarCrossref
Risma  KA, Edwards  KM, Hummell  DS,  et al  Potential mechanisms of anaphylaxis to COVID-19 mRNA vaccines.   J Allergy Clin Immunol. 2021;147(6):2075-2082.Google ScholarCrossref
Shimabukuro  T, Nair  N.  Allergic reactions including anaphylaxis after receipt of the first dose of Pfizer-BioNTech COVID-19 vaccine.   JAMA. 2021;325(8):780-781. doi:10.1001/jama.2021.0600PubMedGoogle ScholarCrossref
Shimabukuro  TT, Cole  M, Su  JR.  Reports of anaphylaxis after receipt of mRNA COVID-19 vaccines in the US: December 14, 2020-January 18, 2021.   JAMA. 2021;325(11):1101-1102. doi:10.1001/jama.2021.1967PubMedGoogle ScholarCrossref
Krantz  MS, Kwah  JH, Stone  CA  Jr,  et al.  Safety evaluation of the second dose of messenger RNA COVID-19 vaccines in patients with immediate reactions to the first dose.   JAMA Intern Med. Published online July 26, 2021. doi:10.1001/jamainternmed.2021.3779PubMedGoogle Scholar