Comparative Safety of BNT162b2 and mRNA-1273 Vaccines in a Nationwide Cohort of US Veterans | Vaccination | JAMA Internal Medicine | JAMA Network
[Skip to Navigation]
Sign In
Figure.  Selection of Persons for the Emulation of a Target Trial Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines (January 4 to October 5, 2021)
Selection of Persons for the Emulation of a Target Trial Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines (January 4 to October 5, 2021)

BMI indicates body mass index; VA, US Department of Veterans Affairs.

Table 1.  Baseline Characteristics of Matched Persons in the Target Trial Emulation Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administrationa
Baseline Characteristics of Matched Persons in the Target Trial Emulation Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administrationa
Table 2.  Estimated Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administration (January 4 to October 5, 2021)a
Estimated Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administration (January 4 to October 5, 2021)a
1.
Polack  FP, Thomas  SJ, Kitchin  N,  et al; C4591001 Clinical Trial Group.  Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine.   N Engl J Med. 2020;383(27):2603-2615. doi:10.1056/NEJMoa2034577 PubMedGoogle ScholarCrossref
2.
Baden  LR, El Sahly  HM, Essink  B,  et al; COVE Study Group.  Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.   N Engl J Med. 2021;384(5):403-416. doi:10.1056/NEJMoa2035389 PubMedGoogle ScholarCrossref
3.
Barda  N, Dagan  N, Ben-Shlomo  Y,  et al.  Safety of the BNT162b2 mRNA COVID-19 vaccine in a nationwide setting.   N Engl J Med. 2021;385(12):1078-1090. doi:10.1056/NEJMoa2110475 PubMedGoogle ScholarCrossref
4.
Klein  NP, Lewis  N, Goddard  K,  et al.  Surveillance for adverse events after COVID-19 mRNA vaccination.   JAMA. 2021;326(14):1390-1399. doi:10.1001/jama.2021.15072 PubMedGoogle ScholarCrossref
5.
Dickerman  BA, Gerlovin  H, Madenci  AL,  et al.  Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in US veterans.   N Engl J Med. 2022;386(2):105-115. doi:10.1056/NEJMoa2115463PubMedGoogle ScholarCrossref
6.
Montgomery  J, Ryan  M, Engler  R,  et al.  Myocarditis following immunization with mRNA COVID-19 vaccines in members of the US military.   JAMA Cardiol. 2021;6(10):1202-1206. doi:10.1001/jamacardio.2021.2833 PubMedGoogle ScholarCrossref
7.
Marshall  M, Ferguson  ID, Lewis  P,  et al.  Symptomatic acute myocarditis in 7 adolescents after Pfizer-BioNTech COVID-19 vaccination.   Pediatrics. 2021;148(3):e2021052478. doi:10.1542/peds.2021-052478 PubMedGoogle ScholarCrossref
8.
Muthukumar  A, Narasimhan  M, Li  QZ,  et al.  In-depth evaluation of a case of presumed myocarditis after the second dose of COVID-19 mRNA vaccine.   Circulation. 2021;144(6):487-498. doi:10.1161/CIRCULATIONAHA.121.056038 PubMedGoogle ScholarCrossref
9.
Diaz  GA, Parsons  GT, Gering  SK, Meier  AR, Hutchinson  IV, Robicsek  A.  Myocarditis and pericarditis after vaccination for COVID-19.   JAMA. 2021;326(12):1210-1212. doi:10.1001/jama.2021.13443 PubMedGoogle ScholarCrossref
10.
Carli  G, Nichele  I, Ruggeri  M, Barra  S, Tosetto  A.  Deep vein thrombosis (DVT) occurring shortly after the second dose of mRNA SARS-CoV-2 vaccine.   Intern Emerg Med. 2021;16(3):803-804. doi:10.1007/s11739-021-02685-0 PubMedGoogle ScholarCrossref
11.
Dias  L, Soares-Dos-Reis  R, Meira  J,  et al.  Cerebral venous thrombosis after BNT162b2 mRNA SARS-CoV-2 vaccine.   J Stroke Cerebrovasc Dis. 2021;30(8):105906. doi:10.1016/j.jstrokecerebrovasdis.2021.105906 PubMedGoogle ScholarCrossref
12.
Chapin-Bardales  J, Gee  J, Myers  T.  Reactogenicity following receipt of mRNA-based COVID-19 vaccines.   JAMA. 2021;325(21):2201-2202. doi:10.1001/jama.2021.5374 PubMedGoogle ScholarCrossref
13.
Chapman  A, Peterson  K, Turano  A, Box  T, Wallace  K, Jones  M. A natural language processing system for national COVID-19 surveillance in the US Department of Veterans Affairs. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. July 2020. https://www.aclweb.org/anthology/2020.nlpcovid19-acl.10
14.
Harder  VS, Stuart  EA, Anthony  JC.  Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research.   Psychol Methods. 2010;15(3):234-249. doi:10.1037/a0019623 PubMedGoogle ScholarCrossref
15.
Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations.   J Am Stat Assoc. 1958;53:457-481. doi:10.1080/01621459.1958.10501452 Google ScholarCrossref
16.
Lebedev  L, Sapojnikov  M, Wechsler  A,  et al.  Minimal change disease following the Pfizer-BioNTech COVID-19 vaccine.   Am J Kidney Dis. 2021;78(1):142-145. doi:10.1053/j.ajkd.2021.03.010 PubMedGoogle ScholarCrossref
17.
D’Agati  VD, Kudose  S, Bomback  AS, Adamidis  A, Tartini  A.  Minimal change disease and acute kidney injury following the Pfizer-BioNTech COVID-19 vaccine.   Kidney Int. 2021;100(2):461-463. doi:10.1016/j.kint.2021.04.035 PubMedGoogle ScholarCrossref
18.
Holzworth  A, Couchot  P, Cruz-Knight  W, Brucculeri  M.  Minimal change disease following the Moderna mRNA-1273 SARS-CoV-2 vaccine.   Kidney Int. 2021;100(2):463-464. doi:10.1016/j.kint.2021.05.007 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 11,898
    Citations 0
    Original Investigation
    June 13, 2022

    Comparative Safety of BNT162b2 and mRNA-1273 Vaccines in a Nationwide Cohort of US Veterans

    Author Affiliations
    • 1CAUSALab, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
    • 2Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
    • 3Department of Surgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
    • 4Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts
    • 5Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts
    • 6Division of Aging, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
    • 7Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
    JAMA Intern Med. Published online June 13, 2022. doi:10.1001/jamainternmed.2022.2109
    Key Points

    Question  How do risks of adverse events compare after vaccination with BNT162b2 (Pfizer-BioNTech) vs mRNA-1273 (Moderna Inc) vaccines?

    Findings  In this cohort study of 433 672 US veterans during 38 weeks of follow-up, recipients of the BNT162b2 vaccine, compared with recipients of the mRNA-1273 vaccine, had an excess per 10 000 persons of 10.9 ischemic stroke events, 14.8 myocardial infarction events, 11.3 other thromboembolic events, and 17.1 kidney injury events. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.

    Meaning  This study’s findings suggest that there were few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days and 38 weeks of the first dose.

    Abstract

    Importance  The risk of adverse events has been found to be low for participants receiving the BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna Inc) vaccines in randomized trials. However, a head-to-head comparison of their safety for a broader range of potential adverse events over longer follow-up and in larger and more diverse populations is lacking, to our knowledge.

    Objective  To compare the head-to-head safety in terms of risk of adverse events of the BNT162b2 and mRNA-1273 vaccines in the national health care databases of the US Department of Veterans Affairs, the largest integrated health care system in the US.

    Design, Setting, and Participants  In this cohort study, the electronic health records of US veterans who received a first dose of the BNT162b2 or mRNA-1273 vaccine between January 4 and September 20, 2021, were used. Recipients of each vaccine were matched in a 1:1 ratio according to their risk factors.

    Exposures  Vaccination with either the BNT162b2 vaccine, with a second dose scheduled 21 days later, or the mRNA-1273 vaccine, with a second dose scheduled 28 days later.

    Main Outcomes and Measures  A large panel of potential adverse events was evaluated; the panel included neurologic events, hematologic events, hemorrhagic stroke, ischemic stroke, myocardial infarction, other thromboembolic events, myocarditis or pericarditis, arrhythmia, kidney injury, appendicitis, autoimmune events, herpes zoster or simplex, arthritis or arthropathy, and pneumonia. Risks over 38 weeks were estimated using the Kaplan-Meier estimator.

    Results  Among 433 672 persons included in the matched vaccine groups, the median age was 69 years (IQR, 60-74 years), 93% of individuals were male, and 20% were Black. Estimated 38-week risks of adverse events were generally low after administration of either the BNT162b2 or the mRNA-1273 vaccine. Compared with the mRNA-1273 group, the BNT162b2 group had an excess per 10 000 persons of 10.9 events (95% CI, 1.9-17.4 events) of ischemic stroke, 14.8 events (95% CI, 7.9-21.8 events) of myocardial infarction, 11.3 events (95% CI, 3.4-17.7 events) of other thromboembolic events, and 17.1 events (95% CI, 8.8-30.2 events) of kidney injury. Estimates were largely similar among subgroups defined by age (<40, 40-69, and ≥70 years) and race (Black, White), but there were higher magnitudes of risk differences of ischemic stroke among older persons and White persons, kidney injury among older persons, and other thromboembolic events among Black persons. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.

    Conclusions and Relevance  The findings of this cohort study suggest that there were few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days of the first dose. The 38-week risks of adverse events were low in both vaccine groups, although risks were lower for recipients of the mRNA-1273 vaccine than for recipients of the BNT162b2 vaccine. Although the primary analysis was designed to detect safety events unrelated to SARS-CoV-2 infection, the possibility that these differences may partially be explained by a lower effectiveness of the BNT162b2 vaccine in preventing the sequelae of SARS-CoV-2 infection compared with the mRNA-1273 vaccine could not be ruled out. These findings may help inform decision-making in future vaccination campaigns.

    Introduction

    Early randomized trials of the 2 messenger RNA (mRNA)–based vaccines—BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna Inc)—revealed a low incidence of adverse events, largely limited to transient reactogenicity common among other viral vaccines.1,2 However, the randomized trials were relatively small and short, and their participants did not reflect the entire populations ultimately receiving the vaccine. Postapproval surveillance via observational studies is therefore critical for monitoring adverse events in vaccine campaigns.

    Observational studies3,4 have provided evidence of the overall safety of these vaccines: for adverse events for which the risk appears to increase after vaccination,3,4 the risk is greater after infection with SARS-CoV-2 among unvaccinated persons.3 However, the 2 mRNA-based vaccines need to be compared head-to-head to inform public health decisions. Ideally, these comparative safety studies would be large enough to provide precise estimates for a broad panel of potential adverse events, have sufficiently long follow-up to evaluate adverse events for which recording may be delayed or that may unfold over time, and include vulnerable (eg, older) and racially and ethnically diverse groups.

    A previous head-to-head comparison of the effectiveness of the BNT162b2 and mRNA-1273 vaccines5 showed a low risk of documented SARS-CoV-2 infection in both vaccine groups but a lower risk for the mRNA-1273 vaccine. Because of this difference in effectiveness, evaluating vaccine safety requires distinguishing adverse events that arise from the vaccines themselves from adverse events due to the infections they are intended to prevent.

    In this study, we used data from the national health care databases of the US Department of Veterans Affairs (VA), the largest integrated health care system in the US, to compare the safety profiles of the BNT162b2 and mRNA-1273 vaccines during 38 weeks of follow-up overall and in subgroups defined by age and by race.

    Methods
    Specification of the Target Trials

    We designed this cohort study to emulate target trials (ie, hypothetical pragmatic trials that would answer the causal questions of interest) of BNT162b2 as compared with mRNA-1273 and risk of potential adverse events in the VA health care system. eTable 1 in the Supplement summarizes the key protocol components. This work was approved by the VA Boston Healthcare System Research & Development Committee and Human Subjects Subcommittee, and an exemption of informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization was granted because it was determined that the research presented no more than minimal risk to privacy and could not be practically conducted without the waiver.

    Eligibility criteria included veteran status; age of at least 18 years between January 4 and September 20, 2021; no previously documented COVID-19 vaccination; no previously documented SARS-CoV-2 infection; no history of the adverse event of interest; known residential address outside of a long-term care facility; known smoking status; and known body mass index within the previous year. Individuals had to have used the VA health care system during the previous year (defined as receiving care at a station eligible to administer both vaccines under study and having at least 1 primary care visit) but not within the previous 3 days (which might suggest the start of symptomatic disease that precluded vaccination).

    The interventions of interest were vaccination with either the BNT162b2 vaccine with a second dose scheduled 21 days later or the mRNA-1273 vaccine with a second dose scheduled 28 days later. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to 1 of these 2 vaccines within strata defined by calendar date (5-day bins), age (5-year bins), sex (male, female), race (Black, White, other, unknown), urbanicity of residence (urban, not urban), and geographic location (coded as 19 categories of Veterans Integrated Services Network). Race and ethnicity were reported by each person.

    The prespecified adverse events of interest were classified as neurologic events, hematologic events, hemorrhagic stroke, ischemic stroke, myocardial infarction, other thromboembolic events, myocarditis or pericarditis, arrhythmia, kidney injury, appendicitis, autoimmune events, herpes zoster or simplex, arthritis or arthropathy, and pneumonia. We selected these categories based on their inclusion in prior vaccine safety studies,3,4 imbalances in results of phase 3 randomized trials,1,2 and potential signals from ongoing surveillance and case reports.4,6-11 Mild and transient local and systemic reactions (eg, pain at the injection site, fatigue, headache), which have been commonly reported after receipt of mRNA-based vaccines12 but are unlikely to require medical attention, were not studied.

    For each eligible individual, follow-up started on the day the first dose of vaccine was received (baseline) and ended on the day of the outcome of interest, 38 weeks after baseline or October 5, 2021, whichever occurred first. Follow-up of a matched pair also ended when either member developed a documented SARS-CoV-2 infection for evaluation of potential complications arising from the vaccines themselves and not from the infections they were intended to prevent.

    Emulation of the Target Trials

    We emulated the aforementioned target trials using the VA health care databases as described previously.5 Vaccination was identified using records in the immunization domain and procedures recorded in the outpatient or inpatient domain of the database. The adverse events were defined using diagnosis codes recorded in inpatient and outpatient domains as well as fee domains (claims for out-of-network care) to capture diagnoses documented both inside and outside the VA health care system. For diagnoses that may be sequelae of SARS-CoV-2 infection (ie, myocarditis or pericarditis, kidney injury, thromboembolic events), we verified that they could not be attributed to infection based on manual review of clinical notes available from the electronic health records for a sample of individuals. SARS-CoV-2 infections were identified with the VA COVID-19 National Surveillance Tool,13 which integrates data on polymerase chain reaction laboratory tests with natural language processing of clinical notes to capture diagnoses inside and outside the VA health care system. Detailed definitions of all study variables are provided in eTable 2 in the Supplement.

    In the emulation of the target trials, vaccine type was not randomly assigned within the strata defined previously. In some instances, individuals may have received the first dose of whichever vaccine was readily available to them; in others, individuals may have sought out a particular vaccine type. We attempted to mimic the stratified randomization of the target trial by matching eligible persons who were vaccinated with BNT162b2 to eligible persons vaccinated with mRNA-1273, using the same matching algorithm described in a previous study.5 The matching factors (calendar date, age, sex, race, urbanicity of residence, and geographic location) are associated with the probability of receiving a particular vaccine and risk of the outcomes.

    Statistical Analysis

    Covariate balance after matching was assessed by evaluating mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered acceptable.14 Cumulative incidence (risk) curves for the vaccination groups were estimated using the Kaplan-Meier estimator.15 We then reported the 38-week risks of each outcome and compared them between vaccination groups via differences and ratios.

    SARS-CoV-2 infections may cause some adverse events (eg, myocarditis) that might have been prevented by vaccination. Then, even if both vaccines had identical safety profiles in the absence of SARS-CoV-2 infection, the more effective vaccine would appear to be safer during a COVID-19 outbreak. To estimate the risk of adverse events attributable to a harmful effect of the vaccine rather than to lower effectiveness, we truncated the follow-up of a matched pair when either member had a documented SARS-CoV-2 infection, as described in the Specification of the Target Trials subsection. Because some undocumented asymptomatic and mild infections might cause adverse events, we also studied the outcome “pneumonia,” which was not expected to be a complication of vaccination, as a marker of undocumented SARS-CoV-2 infection. In a secondary analysis, individuals who had a documented SARS-CoV-2 infection during the follow-up period were not censored to estimate risks that capture complications arising both from the vaccines themselves and from the incident infections that they were intended to prevent (see also the eMethods in the Supplement).

    We also conducted secondary analyses evaluating 14-day and 42-day risk to distinguish adverse events that might be documented in the weeks after the first dose and after the second dose of the vaccine, respectively. We conducted subgroup analyses according to age (<40, 40-69, ≥70 years) and race (Black, White).

    A nonparametric bootstrapping procedure with 500 iterations was used to calculate percentile-based 95% CIs for all estimates. The bootstrap populations were resampled from the entire eligible population, and the bootstrap procedure included all analyses beginning with matching to account for all contributors to the variance estimates.

    Analyses were performed with R software, version 3.6.0 (R Foundation for Statistical Computing) and SAS software, version 8.2 (SAS Institute Inc).

    Results
    Study Population and Follow-up

    Of the 369 846 recipients of the BNT162b2 vaccine and the 400 261 recipients of the mRNA-1273 vaccine who were eligible for the study (Figure), 216 836 BNT162b2 recipients were matched to 216 836 mRNA-1273 recipients before outcome-specific exclusion criteria were applied. Table 1 provides the baseline characteristics of this matched population, which were similar to those of the eligible population (eTable 3 in the Supplement). The median age was 69 years (IQR, 60-74 years), 93% of individuals were male, 20% were Black, and 8% were Hispanic. The 2 vaccine groups had similar distributions of all measured variables, including demographics, coexisting conditions, and markers of health care utilization. The median follow-up was 223 days (IQR, 203-246 days) for BNT162b2 and 223 days (IQR, 203-245 days) for mRNA-1273. The baseline characteristics of matched populations after application of outcome-specific exclusion criteria are provided in eTable 4 in the Supplement.

    Adherence to vaccine protocols was strict. Among persons who received a dose of the BNT162b2 vaccine and had at least 21 days of follow-up, 98% received a second dose of the vaccine (of whom 92% received it before day 24 and 96% received it before day 28). Among persons who received a dose of the mRNA-1273 vaccine and had at least 28 days of follow-up, 98% received a second dose of the vaccine (of whom 92% received it before day 31 and 97% received it before day 35).

    Comparative Safety

    The risks of adverse events ranged from approximately 3 to 347 events per 10 000 persons across 38 weeks, depending on the event (Table 2 and eFigure 1 in the Supplement). During those 38 weeks, recipients of the BNT162b2 vaccine, compared with recipients of the mRNA-1273 vaccine, had an excess per 10 000 persons of 10.9 events (95% CI, 1.9-17.4 events) of ischemic stroke, 14.8 events (95% CI, 7.9-21.8 events) of myocardial infarction, 11.3 events (95% CI, 3.4-17.7 events) of other thromboembolic events, and 17.1 events (95% CI, 8.8-30.2 events) of kidney injury (Table 2). The corresponding risk ratios (BNT162b2 vs mRNA-1273) were 1.17 (95% CI, 1.03-1.28) for ischemic stroke, 1.32 (95% CI, 1.16-1.49) for myocardial infarction, 1.20 (95% CI, 1.05-1.32) for other thromboembolic events, and 1.16 (95% CI, 1.08-1.29) for kidney injury. In analyses that did not attempt to exclude COVID-19-driven adverse events, estimated risk differences were only slightly greater except for the estimated risk difference for pneumonia, which increased from 5.9 (95% CI, −2.9 to 12.7) events per 10 000 persons to 20.9 (95% CI, 11.4-29.5) events per 10 000 persons (eTable 5, eFigure 2 in the Supplement).

    During the first 14 days after receipt of the first vaccine dose, the estimated risks of the evaluated safety outcomes were low and similar in both vaccine groups (eTable 6 in the Supplement). The estimated risks were also low during the first 42 days after receipt of the first vaccine dose, but recipients of the BNT162b2 vaccine compared with recipients of the mRNA-1273 vaccine had an excess per 10 000 persons of other thromboembolic events (1.8; 95% CI, 0.2-4.2 events), arrhythmia (7.5; 95% CI, 1.9-11.5 events), and kidney injury (3.8; 95% CI, 2.1-8.0 events) (eTable 7 in the Supplement).

    Estimates were largely similar among subgroups defined according to age and race, with several exceptions. For example, the estimated 38-week risk difference (BNT162b2 minus mRNA-1273) was higher for ischemic stroke among persons 70 years or older (15.5; 95% CI, 3.2-23.2) than among persons aged 40 to 69 years (5.7; 95% CI, −1.2 to 19.3) and younger than 40 years (3.4; 95% CI, −3.8 to 10.4); higher for kidney injury among persons 70 years or older (24.4; 95% CI, 11.7-42.6) than among persons aged 40 to 69 years (2.4; 95% CI, −8.8 to 34.2) and younger than 40 years (5.7; 95% CI, −0.7 to 21.5); higher for ischemic stroke among White persons (10.5; 95% CI, 2.4-20.4) than among Black persons (−1.2; 95% CI, −20.5 to 13.2); and higher for other thromboembolic events among Black persons (38.7; 95% CI, 15.8-49.0) than among White persons (10.0; 95% CI, −0.7 to 13.9) (eTables 8-12 in the Supplement).

    Discussion

    In this cohort study, we evaluated the comparative safety of the BNT162b2 and mRNA-1273 vaccines in the largest integrated health care system in the US over a relatively long follow-up, which allowed us to quantify the risk of adverse events for which reporting may be delayed or that may emerge over time. The 38-week risks of adverse events were low in both vaccine groups, but recipients of BNT162b2 had an excess risk of ischemic stroke, myocardial infarction, other thromboembolic events, and kidney injury compared with recipients of mRNA-1273. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.

    When the safety of vaccines with varying effectiveness is compared,5 an important consideration is whether to target risk that includes adverse events due to SARS-CoV-2 infection in addition to adverse events of the vaccines themselves. If no attempt is made to distinguish adverse events arising from infections vs those due to vaccines, then the less effective vaccine may appear to have more adverse events during a COVID-19 outbreak, even if the 2 vaccines had an identical safety profile in the absence of SARS-CoV-2 infection. In our primary analysis, we censored matched pairs when either member had a documented SARS-CoV-2 infection in an attempt to target the risk of adverse events due to the vaccines themselves. In our secondary analysis, we did not perform this censoring to capture the risk of adverse events due to the vaccines or COVID-19. As expected, the estimated risk difference for pneumonia (a known sequela of SARS-CoV-2 infection) was lower in the primary analysis (5.9 events per 10 000 persons) than it was in the secondary analysis (20.9 events per 10 000 persons), suggesting that our censoring in the primary analysis appropriately excluded most infection-driven events. Incomplete or lagged information on SARS-CoV-2 infection may have led to the inadvertent inclusion of some infection-driven events in our primary analysis. However, incomplete or lagged information on SARS-CoV-2 infection was expected to be nondifferential between the 2 vaccine groups. In addition, the similarity in the risk curves for pneumonia between the vaccine groups suggests that incomplete or lagged information on infection would not entirely explain the divergence of risk curves observed for other outcomes.

    Head-to-head comparisons of the safety of the mRNA-based COVID-19 vaccines, as in the present analysis, have been lacking. The safety of these vaccines has been assessed separately in studies that compared (1) the short-term risk of adverse events among vaccine recipients vs unvaccinated persons (eg, a study of approximately 900 000 individuals in each group in Israel3); (2) the risk during an interval shortly after receipt of the vaccine with the risk during a later interval among the same individuals (eg, analyses of surveillance data from Vaccine Safety Datalink4); or (3) the incidence of adverse events after vaccination with the incidence during a period before vaccination (eg, a report from 40 US hospitals9). Findings from these studies have suggested that the risk of adverse events after vaccination with an mRNA-based vaccine was low. Compared with no vaccination, the risk for some events (eg, myocarditis) was higher after vaccination but not as high as the risk after SARS-CoV-2 infection.3 Case reports (ie, without any comparator) have described myocarditis diagnoses within 4 days after receipt of an mRNA-based vaccine, particularly after the second dose.6-8 Likewise, findings for thromboembolism and kidney injury after receipt of mRNA-based vaccines have been largely limited to case reports.10,11,16-18

    Both mRNA-based vaccines encode the prefusion stabilized full-length spike protein of SARS-CoV-2, but each uses a slightly different system for intracellular delivery of the mRNA. They also differ with respect to the total dose of mRNA (30 μg for BNT162b2 vs 100 μg for mRNA-1273) and dosing schedule (3 weeks between first and second doses for BNT162b2 vs 4 weeks for mRNA-1273). However, specific mechanisms underlying any observed difference in safety profiles are not understood.

    Strengths and Limitations

    This study has several strengths. First, the VA health care databases contain detailed information on demographic characteristics and medical history, which allowed us to carefully characterize and match recipients of each vaccine type according to potential confounders. Second, the large size of the study population allowed us to evaluate the comparative safety of mRNA-based vaccines for a broad panel of potentially serious and rare adverse events. Third, the relatively long follow-up (38 weeks) allowed us to examine the risk of adverse events for which reporting may be delayed or that may emerge over time. Fourth, the demographic composition of the US veteran population allowed us to provide evidence for a diverse cohort in terms of racial and ethnic groups (20% Black, 8% Hispanic) and to conduct subgroup analyses of various age groups (<40, 40-69, ≥70 years) and among Black persons.

    This study also has several limitations. First, as in any analysis of nonrandomized data, a key challenge was ensuring that the vaccine groups under study were comparable with respect to risk factors for the outcomes. However, after careful matching, the 2 vaccine groups were similar with respect to demographic characteristics, coexisting conditions, and markers of health care–seeking behavior. Furthermore, much less confounding is expected when unintended effects (ie, adverse events) are evaluated than when intended effects (ie, COVID-19 outcomes) are evaluated and when recipients of different vaccines are compared than when vaccinated and unvaccinated persons are compared.

    Second, incomplete outcome ascertainment is possible. To reduce this problem, the analysis captured adverse events recorded in both inpatient and outpatient encounters and was restricted to patients who regularly used VA health care services, had a known residential address, and had routinely collected information (body mass index, smoking status) documented in the database. We expected residual incomplete ascertainment to be nondifferential between the 2 vaccine groups, which may have slightly biased the absolute risks but had few repercussions for the relative effect measures. As with any analysis of electronic health records, a lag between the occurrence of an adverse event and its reporting cannot be ruled out. This potential lag was expected to be nondifferential but may have contributed to a delayed divergence of the cumulative incidence curves between the 2 groups.

    Third, the estimated absolute risks and risk differences were dependent on the baseline risk, which may vary across populations. However, we also presented risk ratios, which are less dependent on baseline risk. Fourth, our study population was restricted to individuals with no history of the adverse event of interest in outcome-specific analyses (to evaluate incident events), comprised regular users of the VA health care system (to maximize capture of these events), and was mostly made up of men (93%) and older individuals (90% were >50 years of age), which may limit the generalizability of the findings.

    Decisions about vaccination campaigns are complex and must incorporate considerations that extend beyond the scope of the present study. Therefore, our findings need to be combined with those from previous studies to inform future decisions about choice of vaccine in coordinated public health responses. Research has shown that the benefit of any available vaccine (vs no vaccine) on an individual’s risk related to SARS-CoV-2 infection clearly outweighs the risk of rare vaccination-related adverse effects.

    Conclusions

    This cohort study found few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days of the first dose. The 38-week risks of adverse events were low in both vaccine groups, although risks were lower for recipients of the mRNA-1273 vaccine than the recipients of the BNT162b2 vaccine. Although our primary analysis was designed to detect safety events unrelated to SARS-CoV-2 infection, we are unable to exclude the possibility that these differences may be partially explained by a lower effectiveness of the BNT162b2 vaccine in preventing the sequelae of SARS-CoV-2 infection compared with the mRNA-1273 vaccine.

    Back to top
    Article Information

    Accepted for Publication: April 21, 2022.

    Published Online: June 13, 2022. doi:10.1001/jamainternmed.2022.2109

    Corresponding Author: Arin L. Madenci, MD, PhD, Department of Epidemiology, Harvard T.H. Chan School of Public Health, 677 Huntington Ave, Boston, MA 02115 (arin_madenci@g.harvard.edu).

    Author Contributions: Drs Dickerman and Madenci contributed equally to this work, had full access to all of the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Dickerman, Madenci, Gagnon, Cho, Casas, Hernán.

    Acquisition, analysis, or interpretation of data: Dickerman, Madenci, Gerlovin, Kurgansky, Wise, Figueroa Muñiz, Ferolito, Gagnon, Gaziano, Cho, Hernán.

    Drafting of the manuscript: Dickerman, Madenci, Cho, Casas, Hernán.

    Critical revision of the manuscript for important intellectual content: Dickerman, Madenci, Gerlovin, Kurgansky, Wise, Figueroa Muñiz, Ferolito, Gagnon, Gaziano, Cho, Hernán.

    Statistical analysis: Dickerman, Madenci, Gagnon, Hernán.

    Obtained funding: Gaziano, Cho, Casas, Hernán.

    Administrative, technical, or material support: Gaziano, Cho.

    Supervision: Gaziano, Cho, Hernán.

    Conflict of Interest Disclosures: Mr Ferolito reported completing a 6-month paid internship with Moderna Therapeutics to help develop bioinformatic next-generation sequencing pipelines and owning 5 shares of Moderna Therapeutics stock purchased in February 2020 outside the submitted work. Dr Hernán reported receiving grants from the US Department of Veterans Affairs (VA) during the conduct of the study and personal fees from Cytel and ProPublica outside the submitted work. No other disclosures were reported.

    Funding/Support: This research was supported in part by the VA Office of Research and Development Cooperative Studies Program (CSP) Epidemiology Center at the VA Boston Healthcare System through CSP 2032, by resources and the use of facilities at the VA Boston Healthcare System and VA Informatics and Computing Infrastructure (VINCI) (VA HSR RES 13-457), and by the use of data from the VA COVID-19 Shared Data Resource. Dr Dickerman is supported by grant K99 CA248335 from the National Institutes of Health. Dr Gerlovin and Mr Ferolito are supported by grant MVP000 from the VA Million Veteran Program. Mr Figueroa Muñiz is supported by grant T32 GM140972 from the National Institute of General Medical Sciences Interdisciplinary Training Program for Biostatisticians.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The contents of this article do not represent the views of the VA or the US government. The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Department of Health and Human Services and its agencies, including Biomedical Advanced Research and Development Authority and the Food and Drug Administration, as well as any other agency of the US government. Assumptions made within and interpretations from the analysis do not necessarily reflect the position of any US government entity.

    Additional Contributions: We thank Daniel C. Posner, PhD, and Yuk-Lam (Anne) Ho, MPH (both Massachusetts Veterans Epidemiology Research and Information Center [MAVERIC]), for insights on COVID-19 data extraction and phenotype definitions and Constance A. Hoag, BA (MAVERIC), for management of the administrative and regulatory aspects of the project. We also thank the VA COVID-19 Shared Data Resource team for their contributions and support and the VA health care providers, employees, and volunteers for their dedication to caring for our veterans throughout this pandemic. These individuals were not paid for their contributions.

    References
    1.
    Polack  FP, Thomas  SJ, Kitchin  N,  et al; C4591001 Clinical Trial Group.  Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine.   N Engl J Med. 2020;383(27):2603-2615. doi:10.1056/NEJMoa2034577 PubMedGoogle ScholarCrossref
    2.
    Baden  LR, El Sahly  HM, Essink  B,  et al; COVE Study Group.  Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.   N Engl J Med. 2021;384(5):403-416. doi:10.1056/NEJMoa2035389 PubMedGoogle ScholarCrossref
    3.
    Barda  N, Dagan  N, Ben-Shlomo  Y,  et al.  Safety of the BNT162b2 mRNA COVID-19 vaccine in a nationwide setting.   N Engl J Med. 2021;385(12):1078-1090. doi:10.1056/NEJMoa2110475 PubMedGoogle ScholarCrossref
    4.
    Klein  NP, Lewis  N, Goddard  K,  et al.  Surveillance for adverse events after COVID-19 mRNA vaccination.   JAMA. 2021;326(14):1390-1399. doi:10.1001/jama.2021.15072 PubMedGoogle ScholarCrossref
    5.
    Dickerman  BA, Gerlovin  H, Madenci  AL,  et al.  Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in US veterans.   N Engl J Med. 2022;386(2):105-115. doi:10.1056/NEJMoa2115463PubMedGoogle ScholarCrossref
    6.
    Montgomery  J, Ryan  M, Engler  R,  et al.  Myocarditis following immunization with mRNA COVID-19 vaccines in members of the US military.   JAMA Cardiol. 2021;6(10):1202-1206. doi:10.1001/jamacardio.2021.2833 PubMedGoogle ScholarCrossref
    7.
    Marshall  M, Ferguson  ID, Lewis  P,  et al.  Symptomatic acute myocarditis in 7 adolescents after Pfizer-BioNTech COVID-19 vaccination.   Pediatrics. 2021;148(3):e2021052478. doi:10.1542/peds.2021-052478 PubMedGoogle ScholarCrossref
    8.
    Muthukumar  A, Narasimhan  M, Li  QZ,  et al.  In-depth evaluation of a case of presumed myocarditis after the second dose of COVID-19 mRNA vaccine.   Circulation. 2021;144(6):487-498. doi:10.1161/CIRCULATIONAHA.121.056038 PubMedGoogle ScholarCrossref
    9.
    Diaz  GA, Parsons  GT, Gering  SK, Meier  AR, Hutchinson  IV, Robicsek  A.  Myocarditis and pericarditis after vaccination for COVID-19.   JAMA. 2021;326(12):1210-1212. doi:10.1001/jama.2021.13443 PubMedGoogle ScholarCrossref
    10.
    Carli  G, Nichele  I, Ruggeri  M, Barra  S, Tosetto  A.  Deep vein thrombosis (DVT) occurring shortly after the second dose of mRNA SARS-CoV-2 vaccine.   Intern Emerg Med. 2021;16(3):803-804. doi:10.1007/s11739-021-02685-0 PubMedGoogle ScholarCrossref
    11.
    Dias  L, Soares-Dos-Reis  R, Meira  J,  et al.  Cerebral venous thrombosis after BNT162b2 mRNA SARS-CoV-2 vaccine.   J Stroke Cerebrovasc Dis. 2021;30(8):105906. doi:10.1016/j.jstrokecerebrovasdis.2021.105906 PubMedGoogle ScholarCrossref
    12.
    Chapin-Bardales  J, Gee  J, Myers  T.  Reactogenicity following receipt of mRNA-based COVID-19 vaccines.   JAMA. 2021;325(21):2201-2202. doi:10.1001/jama.2021.5374 PubMedGoogle ScholarCrossref
    13.
    Chapman  A, Peterson  K, Turano  A, Box  T, Wallace  K, Jones  M. A natural language processing system for national COVID-19 surveillance in the US Department of Veterans Affairs. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. July 2020. https://www.aclweb.org/anthology/2020.nlpcovid19-acl.10
    14.
    Harder  VS, Stuart  EA, Anthony  JC.  Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research.   Psychol Methods. 2010;15(3):234-249. doi:10.1037/a0019623 PubMedGoogle ScholarCrossref
    15.
    Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations.   J Am Stat Assoc. 1958;53:457-481. doi:10.1080/01621459.1958.10501452 Google ScholarCrossref
    16.
    Lebedev  L, Sapojnikov  M, Wechsler  A,  et al.  Minimal change disease following the Pfizer-BioNTech COVID-19 vaccine.   Am J Kidney Dis. 2021;78(1):142-145. doi:10.1053/j.ajkd.2021.03.010 PubMedGoogle ScholarCrossref
    17.
    D’Agati  VD, Kudose  S, Bomback  AS, Adamidis  A, Tartini  A.  Minimal change disease and acute kidney injury following the Pfizer-BioNTech COVID-19 vaccine.   Kidney Int. 2021;100(2):461-463. doi:10.1016/j.kint.2021.04.035 PubMedGoogle ScholarCrossref
    18.
    Holzworth  A, Couchot  P, Cruz-Knight  W, Brucculeri  M.  Minimal change disease following the Moderna mRNA-1273 SARS-CoV-2 vaccine.   Kidney Int. 2021;100(2):463-464. doi:10.1016/j.kint.2021.05.007 PubMedGoogle ScholarCrossref
    ×