Association of the COVID-19 Pandemic With Routine Childhood Vaccination Rates and Proportion Up to Date With Vaccinations Across 8 US Health Systems in the Vaccine Safety Datalink

Key Points

Question  Is the COVID-19 pandemic associated with persistent disruptions in routine childhood vaccination in the US, and have there been variations in routine vaccination by age, race, and/or ethnicity?

Findings  In this cohort study using a prepandemic-postpandemic control design with data from 8 US health systems, there were weekly vaccine administration rates that were substantially lower across pediatric age groups during the COVID-19 pandemic. The proportion of participants up to date with vaccinations was lower for the groups aged 7 months, 18 months, and 13 years, while coverage varied by race and ethnicity.

Meaning  Targeted interventions to increase routine childhood vaccination in undervaccinated communities are needed.

Importance  The COVID-19 pandemic has affected routine vaccine delivery in the US and globally. The magnitude of these disruptions and their association with childhood vaccination coverage are unclear.

Objectives  To compare trends in pediatric vaccination before and during the pandemic and to evaluate the proportion of children up to date (UTD) with vaccinations by age, race, and ethnicity.

Design, Setting, and Participants  This surveillance study used a prepandemic-postpandemic control design with data from 8 health systems in California, Oregon, Washington, Colorado, Minnesota, and Wisconsin in the Vaccine Safety Datalink. Children from age groups younger than 24 months and 4 to 6, 11 to 13, and 16 to 18 years were included if they had at least 1 week of health system enrollment from January 5, 2020, through October 3, 2020, over periods before the US COVID-19 pandemic (January 5, 2020, through March 14, 2020), during age-limited preventive care (March 15, 2020, through May 16, 2020), and during expanded primary care (May 17, 2020, through October 3, 2020). These individuals were compared with those enrolled during analogous weeks in 2019.

Exposures  This study evaluated UTD status among children reaching specific ages in February, May, and September 2020, compared with those reaching these ages in 2019.

Main Outcomes and Measures  Weekly vaccination rates for routine age-specific vaccines and the proportion of children UTD for all age-specific recommended vaccines.

Results  Of 1 399 708 children in 2019 and 1 402 227 in 2020, 1 371 718 were female (49.0%) and 1 429 979 were male (51.0%); 334 216 Asian individuals (11.9%), 900 226 were Hispanic individuals (32.1%), and 201 619 non-Hispanic Black individuals (7.2%). Compared with the prepandemic period and 2019, the age-limited preventive care period was associated with lower weekly vaccination rates, with ratios of rate ratios of 0.82 (95% CI, 0.80-0.85) among those younger than 24 months, 0.18 (95% CI, 0.16-0.20) among those aged 4 to 6 years, 0.16 (95% CI, 0.14-0.17) among those aged 11 to 13 years, and 0.10 (95% CI, 0.08-0.13) among those aged 16 to 18 years. Vaccination rates during expanded primary care remained lower for most ages (ratios of rate ratios: <24 months, 0.96 [95% CI, 0.93-0.98]; 11-13 years, 0.81 [95% CI, 0.76-0.86]; 16-18 years, 0.57 [95% CI, 0.51-0.63]). In September 2020, 74% (95% CI, 73%-76%) of infants aged 7 months and 57% (95% CI, 56%-58%) of infants aged 18 months were UTD vs 81% (95% CI, 80%-82%) and 61% (95% CI, 60%-62%), respectively, in September 2019. The proportion UTD was lowest in non-Hispanic Black children across most age groups, both during and prior to the COVID-19 pandemic (eg, in May 2019, 70% [95% CI, 64%-75%] of non-Hispanic Black infants aged 7 months were UTD vs 82% [95% CI, 81%-83%] in all infants aged 7 months combined).

Conclusions and Relevance  As of September 2020, childhood vaccination rates and the proportion who were UTD remained lower than 2019 levels. Interventions are needed to promote catch-up vaccination, particularly in populations at risk for underimmunization.

On-time administration of vaccines is essential for the prevention of certain communicable diseases. The childhood immunization schedule recommended by the US Advisory Committee on Immunization Practices protects children against 14 infectious diseases before their second birthday.¹ Routine vaccines for children and adolescents induce long-term immunity for many infectious diseases and prevent cervical, oropharyngeal, and other cancers. Additionally, routine childhood vaccines reduce the transmission of infections within communities.^2-4 Vaccines are typically administered to children during well-child visits that occur at specific intervals throughout infancy, childhood, and adolescence.⁵ Documentation of vaccines is generally required for daycare or school entry,⁶ and adherence with vaccination guidelines is a quality measure for health systems.⁷ As such, prior to the COVID-19 pandemic, the US maintained high coverage for most recommended pediatric vaccines.^8-11

Disruptions to health care associated with the COVID-19 pandemic have been well described.^12-15 Soon after the national emergency declaration in March 2020, both the US Centers for Disease Control and Prevention (CDC) and the American Academy of Pediatrics designated pediatric well-child visits and specifically routine childhood vaccination as essential health services. In addition, early in the pandemic, CDC and American Academy of Pediatrics guidelines specified that in-person visits for children younger than 2 years should be prioritized.¹⁶ Nevertheless, reductions in ambulatory visits and vaccine administrations for all pediatric age groups occurred.^17-20

As states relaxed stay-at-home orders and health systems expanded outpatient care, ambulatory visits in adults returned to pre–COVID-19 levels.²¹ Pediatric well-child visits have not rebounded to the same extent, potentially affecting routine childhood vaccination.²² Furthermore, the COVID-19 pandemic has disproportionately affected communities of color, highlighting disparities in access to care and health outcomes.^23-25 It is unknown whether the uneven burden of the COVID-19 pandemic also affected pediatric vaccine delivery for vulnerable populations. The National Immunization Survey (NIS) is the gold standard for assessing vaccination coverage in the US. However, the NIS is time intensive and subject to falling response rates. Data from electronic health data in large health systems have increasingly been used to assess coverage, because these data are available quickly, allowing for more rapid assessments than is possible through the NIS. In this study, we aimed to evaluate trends in vaccination rates and proportion of children up to date (UTD) with vaccinations among infants, children, and adolescents in 2020 and explore variation by age and by race and ethnicity using electronic health data from the Vaccine Safety Datalink (VSD).

We conducted an observational study to evaluate (1) weekly routine childhood vaccination rates by age group over 3 periods in 2020 and (2) the proportion UTD by age as of February, May, and September 2020. Changes in vaccination rates and the proportion UTD were then compared with analogous periods in 2019. We report weekly vaccination rates separately from the proportion UTD because vaccination rates reflect the immediate outcome of the COVID-19 pandemic, while the proportion UTD is a longer-term outcome associated with both prepandemic vaccination status and health seeking during the pandemic. The proportion UTD is a more clinically relevant outcome that has important implications for public health interventions.

This study was approved by the institutional review boards of all participating sites, with a waiver of informed consent because the research could not practicably be carried out without the waiver and was deemed minimal risk. We did not require assent from participants younger than 18 years as part of the study procedures. The study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. This activity was reviewed by the CDC and was conducted in a manner consistent with applicable federal law and CDC policy.

The source population was patients with enrollment in 1 of 8 VSD health systems. At Kaiser Permanente in its Washington, Northwest, Northern California, Southern California, and Colorado locations; HealthPartners; and Marshfield Clinic, enrollment was defined as having health insurance coverage. At Denver Health, enrollment was defined as having had an ambulatory visit in the previous 18 months.²⁶ The VSD is a collaborative project between the CDC’s Immunization Safety Office and 9 health systems representing approximately 3% of the US population.²⁷ For the evaluation of pandemic-associated changes in weekly vaccine administrations rates from January 5, 2020, through October 3, 2020, we included children younger than 24 months and 4 to 6 years, 11 to 13 years, and 16 to 18 years in week-specific analyses if they had at least 1 day of enrollment during that week. These enrollment criteria were used to identify the denominator to estimate vaccination rates. We grouped observation periods into 3 periods based on (1) date of the US emergency declaration, (2) pandemic-associated health care changes with limited access to in-person ambulatory visits, and (3) subsequent reopening of primary care. While the exact dates for expanding in-person well-child visits varied within and across health systems, we applied cutoff dates based on input from investigators at each site. The 3 periods were prepandemic (January 5, 2020, through March 14, 2020, or epidemiologic weeks 2-11), age-limited preventive care (March 15, 2020, through May 16, 2020, or epidemiologic weeks 12-20), and expanded primary care (May 17, 2020, through October 3, 2020, or epidemiologic weeks 21-40). We included children with at least 1 day of enrollment during comparison weeks between January 6, 2019, through October 5, 2019, or epidemiologic weeks 2 through 40 in 2019, as the comparison population.

The evaluation of vaccination coverage included patients who reached the ages 7 months, 18 months, 6 years, 13 years, and 18 years during the months of February, May, or September 2020. Children reaching these specified ages in February, May, or September 2019 were the comparison population. We required that patients had a minimum of 6 months of enrollment preceding the index month and met criteria for receiving care at a VSD site. This minimum enrollment was chosen to account for any vaccinations given outside the VSD site because claims data could be incorporated. For infants aged 7 months, we required at least 1 outpatient encounter following the birth hospitalization; for children aged 18 months, we required at least 1 outpatient encounter in the 12 months preceding the index month; and for all other age groups, we required 1 outpatient encounter at a VSD site in the preceding 18 months. Requiring an outpatient encounter at a VSD site allowed for the capture of vaccinations given in other health systems through bidirectional communication with state and regional immunization information systems.

Vaccines were identified from standardized VSD files that were updated on a weekly basis, which include administrations recorded in the electronic health record, through medical or pharmacy claims, or through bidirectional communication with state immunization information systems. Consistent with US Advisory Committee on Immunization Practices recommendations, for children younger than 24 months, we included the following vaccines: hepatitis B; rotavirus; diphtheria, tetanus, and acellular pertussis (DTaP); Haemophilus influenzae type B conjugate; 13-valent pneumococcal conjugate; inactivated polio; measles, mumps, and rubella; and varicella-zoster virus. For children ages 4 through 6 years, we included measles, mumps, and rubella; varicella-zoster virus; DTaP; and inactivated polio. For those aged 11 to 13 years, we included vaccines for human papillomavirus; tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis; and quadrivalent meningococcal conjugate. For those aged 16 to 18 years, we included human papillomavirus and quadrivalent meningococcal conjugate vaccines.²⁸ Weekly vaccination rates for routine age-specific vaccines were calculated as the number of administered vaccines by selected age group, divided by the number of children in the age group enrolled during each epidemiologic week. Combination vaccines, such as DTap–inactivated polio–H influenzae type B conjugate, were counted as 3 individual vaccines. Up-to-date status for child vaccinations at specified ages was defined per US Advisory Committee on Immunization Practices guidelines, excluding hepatitis A and influenza vaccines (eTable 1 in the Supplement).²⁹

Age, sex, race, and ethnicity came from electronic health record data. Race and ethnicity were grouped by Hispanic or Latino, non-Hispanic Asian, non-Hispanic Black or African American, non-Hispanic white, and other, which included all other racial categories, including individuals reporting multiple races and those with missing race and ethnicity data. Race and ethnicity were included to allow for assessment of racial and ethnic disparities.

Using a ratio of rate ratios (ROR) approach, we compared age-specific vaccination rates during limited preventive care and expanded primary care periods to rates in the prepandemic period for 2020 and used the same periods in 2019 as the reference. We used a Poisson regression model to estimate RORs and 95% CIs by age group, with the number of children enrolled as the offset and covariates for race, ethnicity, sex, and VSD site.

We estimated the proportion UTD for each age and study month (February, May, and September 2020) and used 2019 as a reference year, to account for seasonality. Confidence intervals for UTD were estimated using a normal approximation interval for binomial proportions. Similarly, we estimated proportion UTD by race and ethnicity for 2020. Analysis was performed using SAS/STAT and SAS/GRAPH version 9.4 (SAS Institute Inc).

We included 1 399 708 children with enrollment in 2019 and 1 402 227 with enrollment in 2020 in analyses of vaccine administration rates. The population was 49.0% female (n = 1 371 718) and 51.0% male (n = 1 429 979), 11.9% Asian (n = 334 216), 7.2% Black (n = 201 619), 32.1% Hispanic (n = 900 226), and 32.4% White (n = 909 189). The evaluation of UTD status included 39 113 children reaching specified ages (7 months, 18 months, 6 years, 13 years, and 18 years) in February, May, or September 2019 and 40 373 children reaching these ages in February, May, or September 2020 (Table 1). The small proportion of the overall population included in the UTD analysis is because of both minimum enrollment criteria as well as the number of children reaching a targeted age during the specific study months.

Vaccination rates varied widely by age group, period, and year (Figure 1; Table 2). For the 2020 prepandemic period, mean weekly vaccination rates for all age groups were similar to 2019 vaccination rates. During age-limited preventive care (epidemiologic weeks 12-20 of 2020), mean weekly vaccination rates per 1000 population for all age groups were lower than 2019 rates for the same epidemiologic weeks (eg, for those aged <24 months: 194.6 [95% CI, 191.3-197.9] per 1000 in 2019 vs 160.1 [95% CI, 157.1-163.0] per 1000 in 2020; Table 2). During expanded primary care (epidemiologic weeks 21-40 in 2020), mean weekly vaccination rates were higher in all age groups compared with age-limited preventive care (eg, for those aged <24 months: from 160.1 [95% CI, 157.1-163.0] per 1000 to 181.8 [95% CI, 179.5-184.1] per 1000; Table 2).

Comparing RORs in the age-limited preventive care (epidemiologic weeks 12-20) period relative to the prepandemic period (epidemiologic weeks 2-11) in 2020 and the same epidemiologic weeks in 2019, RORs for all age groups demonstrated substantially lower vaccination rates (Table 2). Lower RORs persisted during expanded primary care (epidemiologic weeks 21-40) for all but the age group 4 to 6 years (RORs: <24 months, 0.96 [95% CI, 0.93-0.98]; 4-6 years, 1.02 [95% CI, 0.96-1.07]; 11-13 years, 0.81 [95% CI, 0.76-0.86]; 16-18 years, 0.57 [95% CI, 0.51-0.63]).

During February, May, and September 2019 and February 2020, the proportion of children reaching targeted ages who were UTD with recommended vaccines remained relatively stable by age group (7 months, 81%-82%; 18 months, 60%-61%; 6 years, 76%-79%; 13 years, 57%-59%; 18 years, 58%-61%). Decreases in the proportions UTD were noted for infants reaching 7 and 18 months of age in May 2020 and September 2020 (infants reaching 7 months: May 2020, 72% [95% CI, 71%-73%]; September 2020, 74% [95% CI, 73%-76%]; 18 months: May 2020, 56% [95% CI, 55%-57%]; September 2020, 57% [95% CI, 56%-58%]; Figure 2; eTable 2 in the Supplement). A decrease in the proportion UTD was also noted in September 2020 for children turning 13 years (13 years: 56% [95% CI, 55%-57%]). By September 2020, the proportions UTD among children turning 6 and 18 years were similar to 2019 and February 2020 (eTable 2 in the Supplement).

Racial differences in the proportion UTD varied by age group (eTable 2 in the Supplement; Figure 3). As of September 2020, the proportion UTD at 7 months was highest in Asian infants (88% [95% CI, 86%-90%]) and lowest in Black infants (61% [95% CI, 55%-67%]). These differences in UTD by race and ethnicity at age 7 months were similar in magnitude to those present before the pandemic. The proportion UTD at 18 months as of September 2020 was highest in Asian infants (76% [95% CI, 74%-78%]) and lowest in Black infants (41% [95% CI, 37%-45%]). Differences in the proportion UTD at 18 months between Black and other groups may have increased after onset of the pandemic. Differences in UTD by race and ethnicity in September 2020 among 6-year-old individuals, 13-year-old individuals, and 18-year-old individuals were similar in magnitude to those present before the pandemic. Among 6-year-old participants, the proportion UTD in September 2020 was lowest among Black individuals (70% [95% CI, 67%-74%]) and highest among Hispanic individuals (79% [95% CI, 78%-80%]). For adolescents reaching 13 years in September 2020, the proportion UTD was lowest in Black adolescents (51% [95% CI, 47%-54%]) and White adolescents (51% [95% CI, 50%-53%]). Finally, in September 2020, the proportion UTD was lowest in Black adolescents (59% [95% CI, 55%-62%]) and White adolescents (55% [95% CI, 53%-57%]) who were reaching 18 years of age.

In this large study with data from 8 US health systems, we observed substantial and persistent disruptions to routine infant, childhood, and adolescent vaccine delivery following the start of the COVID-19 pandemic. As in prior reports, we noted a precipitous decline in vaccinations at the onset of the pandemic through May 2020. Lower vaccination rates persisted in most age groups through September 2020. During the period of expanded primary care, vaccine administration rates among infants younger than 24 months and children aged 4 to 6 years approached or were equal to 2019 rates. Nevertheless, only 74% of infants turning 7 months in September 2020 were UTD, compared with 81% of infants turning 7 months in September 2019. Similarly, only 57% of infants reaching 18 months in September 2020 were UTD, compared with 61% in September 2019. For most ages and across all periods, the proportion UTD was lowest in Black children. While disparities were evident prior to the COVID-19 pandemic, during the pandemic, these differences became more pronounced for the 18-month-old age group.

This study expands on prior work by our group and others demonstrating marked decreases in vaccine administrations following the COVID-19 national emergency declaration.^17-19,22 Our prior VSD article¹⁷ described the dramatic decrease in administration of measles-containing vaccines, especially in children older than 24 months, from March 16, 2020, through April 19, 2020. Similar decreases in routine childhood vaccine ordering and administration were reported using data through April 2020 from the Vaccines for Children program¹⁷ and the Colorado Immunization Information System.¹⁹ Bramer and colleagues¹⁸ demonstrated decreases in vaccine coverage at specified ages (1, 3, 5, 7, 16, 19, and 24 months) in Michigan as of May 2020, compared with the prior 4 years. More recently, using data from 1 VSD health system, Ackerson and colleagues²² reported persistent declines in pediatric vaccine administration and coverage through August 2020. Our study adds to this body of literature, providing data on vaccine administrations, coverage, and persistent disparities for infants, children, and adolescents at specified ages through September 2020.

One strength of this study was that we quantified changes in vaccination rates by period and then compared these rate ratios to those for the analogous weeks in 2019, thus accounting for seasonal variation in vaccine administration. Applying this approach, we demonstrated that the most dramatic decreases in vaccination rates occurred among those aged 4 to 6 years, 11 to 13 years, and 16 to 18 years during the period of age-limited preventive care. These findings are consistent with clinical guidelines at the time, in that health systems limited in-person visits and focused vaccine delivery on children younger than 24 months. Nevertheless, rebounds in vaccine administrations with the expanded opening of primary care were not sufficient to make up for the many missed vaccines. Disruptions to the timing of vaccine appointments in early infancy can lead to substantial delays in completion of vaccine series because of the required minimum intervals between vaccine doses and the need for additional health care visits to receive missed vaccines. These data suggest that barriers to vaccination persisted after the opening of preventive care for all age groups. These may have included fears regarding acquiring COVID-19 during in-person medical visits, staffing shortages at clinical sites because of COVID-19 infections or exposures, and financial constraints.

An additional strength of this study was that we were able to describe trends in proportion UTD by race and ethnicity.^17-19 For the age groups turning 7 and 18 months, coverage among Black infants was lower than other racial and ethnic groups during the 2019 and 2020 periods evaluated. This finding highlights existing disparities in pediatric vaccine coverage that have also been documented in the NIS.¹¹ The reasons for these disparities are likely multifactorial, with structural, logistical, cultural, and other variables contributing. One recent study³⁰ at Denver Health, a VSD site, found that 25% of Black caregivers were vaccine hesitant and hesitancy was associated with a 4-month increase in the mean numbers of days undervaccinated when children were assessed at age 2 years. Health system and community-level interventions are needed to support on-time vaccination for children, especially in underresourced communities. Enforcement of mandates requiring vaccination prior to school entry is also important, because these have been shown to not only increase vaccine uptake across populations^31,32 but also reduce disparities in vaccination.³³

Limitations to this research should be noted. First, the VSD population is insured, and patients included were receiving care in large integrated health systems. Vaccination coverage may be lower in other populations.^11,18 Prior to the pandemic, vaccination coverage among children aged 19 to 35 months in the VSD tended to be higher than for the general US population, based on NIS results.^34,35 In addition, vaccine administration rates and proportion UTD by race and ethnicity we report may reflect differences in site-based interventions, such as reminder-recalls, that occurred during the period of expanded primary care. Furthermore, our assessment of proportion UTD aimed to reflect recommended timing of routine childhood and adolescent vaccinations; yet other definitions or age cutoffs could have been applied. For example, to reduce differences associated with seasonality, we did not include influenza vaccine when assessing UTD. Also, to simplify data analysis, we counted 2 doses of H influenzae type B conjugate and rotavirus vaccines by 7 months and 2 doses of human papillomavirus vaccine by 13 or 18 years as being UTD and did not account for manufacturer recommendations (for H influenzae type B conjugate and rotavirus vaccines) or age at first dose (for the human papillomavirus vaccine). As such, reported UTD percentages may be higher than what the coverage would be using stricter definitions. Our assessment of disparities in vaccination coverage is presented for descriptive purposes; statistical comparisons between groups were not conducted. While beyond the scope of this study, evaluating the reasons for these disparities is needed so that tailored interventions can be implemented with the goal of reducing disparities in vaccination coverage. Finally, our grouping of periods was based on estimated transitions from limited to expanded primary care; the actual timing of transitions varied by VSD site and even within sites. Applying different cutoff values would have changed the ROR point estimates.

The protection of our population from infectious diseases depends on high uptake of routine vaccinations. In this multisite observational study, we found that, as of September 2020, routine infant, child, and adolescent weekly vaccination rates and coverage remained lower than 2019 levels. There were also persistent disparities in vaccination coverage by race and ethnicity. Interventions are needed to promote catch-up vaccination, particularly focusing on populations in which disparities in vaccination coverage were evident prior to the pandemic.

Accepted for Publication: August 12, 2021.

Published Online: October 7, 2021. doi:10.1001/jamapediatrics.2021.4251

Corresponding Author: Malini B. DeSilva, MD, MPH, HealthPartners Institute, 8170 33rd Ave South, MS21112R, Bloomington, MN 55440 (malini.b.desilva@healthpartners.com).

Author Contributions: Drs DeSilva and Kharbanda had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Mr Haapala and Dr Vazquez-Benitez from HealthPartners Institute conducted and are responsible for the data analysis.

Concept and design: DeSilva, Vazquez-Benitez, Daley, Jackson, Lindley, Gee, Weintraub, Kharbanda.

Acquisition, analysis, or interpretation of data: DeSilva, Haapala, Vazquez-Benitez, Daley, Nordin, Klein, Henninger, Williams, Hambidge, Jackson, Donahue, Qian, Weintraub, Kharbanda.

Drafting of the manuscript: DeSilva, Vazquez-Benitez, Gee, Kharbanda.

Critical revision of the manuscript for important intellectual content: DeSilva, Haapala, Vazquez-Benitez, Daley, Nordin, Klein, Henninger, Williams, Hambidge, Jackson, Donahue, Qian, Lindley, Weintraub.

Statistical analysis: Haapala, Vazquez-Benitez, Daley, Qian, Weintraub.

Obtained funding: Klein, Kharbanda.

Administrative, technical, or material support: Daley, Henninger, Donahue, Gee, Weintraub.

Supervision: Klein, Hambidge, Qian, Weintraub, Kharbanda.

Conflict of Interest Disclosures: Dr Klein reported grants from the US Centers for Disease Control and Prevention (CDC) during the conduct of the study and grants from Merck, Sanofi Pasteur, GlaxoSmithKline, Pfizer, and Protein Science (now Sanofi Pasteur) outside the submitted work. Dr Donahue reported grants from the CDC during the conduct of the study and grants from Janssen Global Services, LLC, for exploratory research related to a non–COVID-19 vaccine outside the submitted work. Dr Jackson has received a research grant from Sanofi Pasteur to study the epidemiology of respiratory syncytial virus. Dr Jackson reported grants from the CDC during the conduct of the study; grants from Sanofi Pasteur outside the submitted work. Dr Qian has received research support from Moderna, GlaxoSmithKline, and Dynavax for unrelated studies. Drs DeSilva, Haapala, and Kharbanda reported grants from the CDC (contract 200-2012-53526) during the conduct of the study. Drs Daley and Henninger reported grants from the CDC during the conduct of the study. Dr Williams reported Vaccine Safety Datalink COVID-19 Infrastructure Funding grants from the CDC during the conduct of the study. No other disclosures were reported.

Funding/Support: This work was supported by grant funding from the US Centers for Disease Control and Prevention (contract 200-2012-53526).

Role of Funder/Sponsor: The US Centers for Disease Control and Prevention participated in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and the decision to submit the manuscript for publication.

Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Mention of a product or company name is for identification purposes only and does not constitute endorsement by the Centers for Disease Control and Prevention.

Additional Contributions: We would like to acknowledge Leslie Kuckler for her role as project manager at HealthPartners Institute for which she provides administrative and technical support, Yingbo Lou, MS, for his role as data manager and contributing data analysis at Denver Health, and Bradley Crane, MS, who is responsible for the standardization of VSD files at Northwest Kaiser Permanente. They were compensated for their contributions through grants from the US Centers for Disease Control and Prevention.