COVID-19 Vaccination and Estimated Public Health Impact in California

Key Points Question How many COVID-19 cases, hospitalizations, and deaths were averted because of COVID-19 vaccination in California? Findings In this modeling study using data from the California Department of Public Health, COVID-19 vaccination was estimated to have prevented more than 1.5 million COVID-19 cases, 72 000 hospitalizations, and 19 000 deaths during the first 10 months of vaccination, through October 16, 2021. Meaning These findings suggest that COVID-19 vaccination had a large public health benefit in California, which can be generalized across the United States.


eAppendix. Supplemental Methods
In this appendix, we provide further methodologic detail on the model structure and statistical analysis.

Study outcomes
Calculation of relative reduction of COVID-19 cases We estimated the relative reduction of COVID-19 cases in the entire vaccine-eligible population (≥12 years) and each age group (12-17 years, 18-49 years, 50-64 years, and ≥65 years) after the start of Phase 1A of vaccination (November 29, 2020), adjusting for vaccine coverage in the relevant population. The formula for percentage reduction in COVID-19 cases over a fixed period of time is as follows: In sensitivity analysis, we estimated alternative formulations of the relative reduction: (1) accounting for age-specific eligibility over time; and (2) not adjusting by vaccine coverage (see Sensitivity analyses).

Estimation of averted COVID-19 cases: Additional methodology Primary model
We defined the lower bound for the number of weekly averted COVID-19 cases as zero based on bioplausibility.

Alternative model
Model of natural immunity We assumed that natural infection provided perfect immunity without waning though we relaxed this assumption in a sensitivity analysis. We assumed complete reporting of COVID-19 cases. We estimated total infections in the unvaccinated and each vaccine-eligible age group (<12 years, 12-17 years, 18-49 years, 50-64 years, ≥65 years) using literature estimates of the subclinical proportion by age (<19 years, 19-59 years, ≥60 years) 1 . We used the reported means and 95% confidence intervals of each age-specific subclinical fraction of infection to fit optimal beta distributions. The mean and the fitted shape parameters of each distribution are shown in Table A1.

Model of vaccine-induced immunity
We modeled vaccine effectiveness (against clinical disease) and waning immunity on a personlevel based on vaccine (BNT162b2, mRNA-1273, Ad26.COV2.S) and the number of doses received. We assumed six possible vaccination scenarios: 1) BNT162b2 single dose; 2) BNT162b2 two doses; 3) mRNA-1273 single dose; 4) mRNA-1273 two doses; 5) Ad26.COV2.S single dose; and 6) unvaccinated. We did not include boosters given limited use over the study period.
We used published literature to estimate the vaccine effectiveness in each scenario over time, assuming instantaneous onset of protection and waning immunity at various time points 2-6 . We fit beta distributions using the published mean and 95% confidence intervals of each estimate of vaccine effectiveness (see Table A2). We made the simplifying assumption that all individuals who received two doses of the BNT162b2 vaccine received their second dose three weeks after their first dose and all individuals that received two doses of the mRNA-1273 vaccine received their second dose four weeks after receiving their first dose based on published literature 7 . We did not account for potential differences in vaccine effectiveness by age, which is broadly supported by literature 6,8,9 . We accounted for possible changes in vaccine effectiveness against the highly infectious Delta variant of SARS-CoV-2 as a sensitivity analysis (see Sensitivity analyses) but did not account for variant specific effectiveness in the main analysis. Average vaccine effectiveness and waning over time is shown in Figure A1, and the distributions of vaccine effectiveness are shown in Table A2.  We used publicly available COVID-19 vaccination data 10 to estimate the weekly number of newly vaccinated individuals in each of the six scenarios and age groups (12-17 years, 18-49 years, 50-64 years, ≥65 years). These age groups were based on vaccine prioritization age groupings. Date of receipt of first and second doses of the BNT162b2 and mRNA-1273 vaccines was not available in our data. We therefore calculated the mean fraction of individuals that received BNT162b2 or mRNA-1273 vaccines in each age group to estimate weekly BNT162b2 and mRNA-1273 vaccinations. We additionally used published literature to estimate the proportion of individuals who received only a single dose of the BNT162b2 or mRNA-1273 vaccines 7 .
We combined our estimates of the number of individuals newly vaccinated each week by vaccine type and number of doses received and corresponding vaccine effectiveness over time to estimate the fraction of the population with immunity due to vaccination. Since we assumed that natural infection provided perfect immunity, we first calculated the number of newly vaccinated individuals not previously infected with SARS-CoV-2 each week before estimating the number of protected individuals over time due to vaccination. We assumed previously infected and uninfected individuals were equally likely to receive any COVID-19 vaccine, following the observed weekly distribution of vaccines by vaccine type. The formula for calculating the number of new vaccinations in previously uninfected individuals in an age group a at week t is as follows:

Monte Carlo simulation
We used Monte Carlo simulation to capture uncertainty for the analysis in the alternative model, with a focus on accounting for uncertainty in vaccine effectiveness and estimates of subclinical infection. We ran 1000 simulations using randomly sampled values of parameters from fitted parameter distributions (Table A1 and A2). We reported the mean and 95% uncertainty intervals (95% UI) of study outcomes.
To generate random samples of our parameters for each simulation, we independently sampled from the distributions of sub-clinical fractions in three age groups: <19 years, 19-59 years, and ≥60 years. We sampled independently from the standard uniform distribution for three vaccines (BNT162b2, mRNA-1273, and Ad26.COV2.S) and used inversion sampling to generate samples of vaccine effectiveness to account for changes in effectiveness over time.
Estimation of averted COVID-19 hospitalizations and deaths: Additional methodology We estimated monthly risks of hospitalization and death in each age group of the population (<12 years, 12-17 years, 18-49 years, 50-64 years, ≥65 years) by finding the proportion of cases that resulted in hospitalization or death each month using CDPH data.
Due to lag in reporting of severe COVID-19 outcomes, we used the age-specific monthly risk of hospitalization and death in August 2021 to predict averted hospitalizations and deaths in September and October 2021 (eFigure 1). We defined the lower bound for the number of weekly averted COVID-19 hospitalizations and deaths as zero based on bioplausibility.
We also used values from literatures for risk of hospitalization and death from COVID-19 in sensitivity analysis.
Prediction and uncertainty intervals Prediction intervals from the primary modeling approach reflect both uncertainty in the estimated model parameters and variation expected for the outcome, while the uncertainty intervals from the alternative modeling approach reflect the uncertainty in the parameters inputted into the alternative model. These represent different measures of statistical variability in estimation.

Sensitivity analyses
Age-specific vaccine eligibility In both modeling approaches, we performed a sensitivity analysis to account for age-specific differences in COVID-19 vaccine eligibility over time among the four vaccine-eligible age groups (12-17 years, 18-49 years, 50-64 years, ≥65 years).
COVID-19 vaccines became available for the general population ≥16 years and 12-15 years in mid-April 2021 and mid-May 2021 respectively 11 . We assumed vaccination in the population 12-17 years began on April 11, 2021.
Vaccination in adults 18-49 years and 50-64 years began before vaccines were widely available in those populations due to occupational risk. Healthcare and other frontline workers became eligible for COVID-19 vaccines in Phase 1A of vaccination and essential workers became eligible for vaccines in Phase 1B of vaccination 11,12 . For this analysis, we assumed widespread vaccination in the populations 18-49 years and 50-64 years began with Phase 1B of vaccination and used February 14, 2021 to mark vaccine eligibility [13][14][15][16] .
We used January 10, 2021 as the start of vaccination among adults ≥65 years, since they were eligible for COVID-19 vaccines beginning mid-January 2021 17 .
We performed the main analysis in each age group after each age group became eligible for vaccines, reporting both unadjusted and adjusted relative reduction of outcomes. Dates of vaccine-eligibility by age group used in this analysis are shown in Table A3.  Table A4. We did not perform this sensitivity analysis in the primary model since person-level vaccination was not explicitly included. We conducted a separate sensitivity analysis that relaxed the assumption of perfect immunity from infection. We assumed that SARS-CoV-2 infection was 86% effective against reinfection within 1 year of primary infection with waning after 1 year based on recent published literature 22,23 . We included these estimates of effectiveness of natural infection against reinfection as additional parameters to sample from in the Monte Carlo simulation. The fitted beta distributions of these parameters are shown in Table A5. We additionally assumed that all COVID-19 vaccines were 90% effective against reinfection after previous natural infection, which is supported by literature 22 .

Hospitalizations and deaths
We assessed hospitalization and death outcomes of the main analyses when using literature estimates of the risk of hospitalization and death in cases that were not fully vaccinated 25 rather than estimates from CDPH data. A comparison of the hospitalization and death risk used in the main analyses and literature estimates are shown in Table A6. The alternative model we developed is applicable to other COVID-19 outcomes. As an additional sensitivity analysis, we adapted the alternative model to predict hospitalizations and deaths that would have occurred in the absence of vaccination. We estimated the incidence of hospitalization and deaths instead of incidence of cases, incorporating literature estimates of vaccine effectiveness against hospitalizations and death to estimate the susceptibility profile of the population. The distributions of vaccine effectiveness against hospitalization and death are in Table A7.