Hoekstra EJ, LeBaron CW, Megaloeconomou Y, Guerrero H, Byers C, Johnson-Partlow T, Lyons B, Mihalek E, Devier J, Mize J. Impact of a Large-Scale Immunization Initiative in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). JAMA. 1998;280(13):1143–1147. doi:10.1001/jama.280.13.1143
From the National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Ga (Drs Hoekstra and LeBaron, Mss Lyons and Devier, and Mr Mize); Catholic Charities, Chicago, Ill (Mr Megaloeconomou); the Chicago Immunization Program, Chicago Department of Public Health, Chicago, Ill (Mss Guerrero and Byers and Mr Mihalek); and the Special Supplemental Nutrition Program for Women, Infants, and Children, Chicago, Ill (Ms Johnson-Partlow).
Context.— Inner-city immunization rates have lagged behind those in other areas
of the country.
Objective.— To evaluate the impact of an initiative linking immunization with distribution
of food vouchers in the inner city.
Design.— Retrospective analysis of immunization data gathered in 1996 and 1997.
Setting.— Nineteen Special Supplemental Nutrition Program for Women, Infants,
and Children (WIC) sites serving 30% of the Chicago, Ill, birth cohort.
Participants.— A total of 16581 children 24 months old or younger.
Interventions.— Voucher incentives (varying frequency of food voucher issuance based
on immunization status) and assessment of immunization status and referral
to immunization provider.
Main Outcome Measures.— Age-appropriate immunization rates and WIC enrollment rates.
Results.— During the 15-month period of evaluation, immunization rates increased
from 56% to 89% at sites performing voucher incentives. The proportion of
children needing voucher incentives declined from 51% to 12%. Sites performing
assessment and referral, but not providing voucher incentives, showed no evidence
of improvement in immunization coverage. No difference was observed in enrollment
rates between sites performing voucher incentives and those that did not.
Conclusion.— Applied in a large-scale, programmatic fashion, voucher incentives in
WIC can rapidly increase and sustain high childhood immunization rates in
an inner-city population.
AFTER THE MEASLES resurgence of 1989-1991, public and private investment
in immunization increased dramatically, and immunization rates have risen
across the nation.1- 3
However, approximately 80% of unvaccinated measles cases during the resurgence
occurred among urban preschoolers,4,5
and immunization rates in inner cities have often lagged behind the rest of
the country.3,6 This raised concerns
that high-risk urban children might lack the minimum level of population immunity
necessary to prevent outbreaks.7
A 6-month randomized trial in 1991 showed that varying the frequency
of food voucher issuance (voucher incentives) in the Special Supplemental
Nutrition Program for Women, Infants, and Children (WIC) could increase measles
vaccination rates among preschoolers enrolled in 6 of New York City's more
than 100 WIC sites.8,9 These findings
have been confirmed by preliminary results from 2 randomized trials elsewhere.10,11 Since 44% of the US birth cohort
is enrolled in WIC (US Department of Agriculture, unpublished data, 1995),
with age and demographic characteristics that match immunization needs, a
nationwide initiative to promote activities that raise immunization coverage
in WIC began in 1995, with the endorsement of the Advisory Committee on Immunization
Practices (ACIP).12 States and localities now
must spend a minimum of 10% of all federal immunization funds on WIC linkages
(1996 Senate Appropriations Committee Report language). A recent survey suggested
that 75% of all states have already established WIC immunization linkages.13
This national WIC initiative represents the most sweeping effort to
link immunization to a public program since the school law campaigns of the
1970s. But can interventions, which were successful in relatively small, time-limited,
and intensively monitored studies, actually raise coverage when they are applied
programmatically to whole populations, particularly in inner cities where
infrastructure problems are intense? Questions have also been raised as to
whether some immunization interventions in WIC might have an adverse effect
Chicago, Ill, was a major source of cases during the measles resurgence,4,14- 16 with
data indicating that city-wide vaccination coverage was 16% to 25% over 8
years, despite the epidemic.17 A 1994 survey
of children residing in public housing suggested that immunization series
completion rates were 23% by 2 years of age.18
In 1995, the Chicago Department of Public Health (CDPH) began implementation
of immunization interventions in WIC, a supplemental food voucher program
in which approximately 70% of the Chicago birth cohort is enrolled. We present
the first evaluation of the impact of a large-scale immunization initiative
The Chicago WIC is administered through 47 sites, at which about 37000
infants are enrolled (Table 1).
Approximately 45% of these clients are served by 22 sites under the direct
administration of the CDPH, while the rest are served by 25 sites administered
by contract agencies. In December 1995, CDPH required its sites to begin the
following: (1) to ask all families with children 24 months of age or younger
to bring vaccination documentation to WIC visits; (2) to assess the vaccination
status of these children; and (3) to refer children in need of vaccinations
to a provider. In May 1996, CDPH began the progressive implementation of voucher
incentives for immunization (see below). By June 1997 (the end of the evaluation
period), 14 sites were providing voucher incentives. To implement this ongoing
initiative, 11 clerks were hired, under the supervision of a coordinator,
at an annual cost of $271000.
Interventions.Assessment and Referral. At each WIC certification and recertification visit (which occur every
6 months), the clerk reviewed the vaccination status of each child 24 months
of age or younger. Where documentation of vaccination was provided (eg, vaccination
card), the vaccination dates for each antigen were entered into a software
program on the WIC computer. The program determined if the child was age-appropriately
vaccinated by ACIP standards. If the child was not age-appropriately vaccinated,
the family was referred to a provider for vaccination.
Voucher Incentive. In Chicago, a 3-month supply of food vouchers is usually issued by WIC
to enrolled families. When a child is designated as being at high risk for
illness (eg, anemic), only a 1-month supply of vouchers is issued at a time,
to ensure frequent contact with the family regarding the high-risk condition.
In the immunization initiative, children who were not age-appropriately vaccinated
were treated as high-risk clients. A 1-month supply of vouchers was issued
until the child was age-appropriately vaccinated, at which time the issuance
of a 3-month supply was resumed. Only voucher frequency was varied; no voucher
was ever withheld from a child because of immunization status.
Evaluation Approach. We retrospectively evaluated the impact of the initiative using existing
data generated for the period April 1996 (by which date all sites had been
performing assessment and referral for 4 months or longer, but none had started
voucher incentives) through June 1997 (when the majority of sites had been
performing voucher incentives for more than 12 months). From these data, we
identified groups of sites (Table 1)
whose different immunization activities provided comparisons to help distinguish
the effects of voucher incentives from the impact of other immunization-related
Group A (+ Incentives; + Monitoring). Four CDPH sites serving about 4000 infants (8% of the Chicago birth
cohort) had been prospectively selected by initiative staff as sentinel sites
for intensive monitoring of the initiative from its inception (see process
measures below). Selection was based on inner-city location and on the program
staff's perception of the population's risk for low vaccination coverage.
Group B (+ Incentives; − Monitoring). Ten CDPH sites serving about 9000 infants (18% of the Chicago birth
cohort) began voucher incentives at the same time as group A (May 1996) but
were not intensively monitored. Monthly immunization rates were not collected
until September 1996.
Group C (− Incentives; + Monitoring). Three CDPH sites serving about 2000 infants (4% of the Chicago birth
cohort) did not begin providing voucher incentives during the evaluation period
but collected immunization rate data, starting in February 1996, in preparation
for the subsequent implementation of voucher incentives. Implementation did
not take place at these sites during the study period because of administrative
delays in hiring and assigning staff.
Group D (− Incentives; − Monitoring). Two CDPH sites serving about 1000 infants (2% of the Chicago birth cohort)
did not implement voucher incentives and were not monitored during the evaluation
period, although they reported having performed assessment and referral.
Data Collected.Population Characteristics.
The 1996 Chicago birth cohort, the children of Chicago with reported
preschool measles during the 1989-1990 epidemic, and the infants enrolled
in WIC in Chicago in April 1996 were compared for population size and race/ethnicity.
Immunization Services Available at WIC Sites. One of 3 levels of immunization services was available at each WIC site:
an on-site nurse, a colocated clinic, or no immunization services whatsoever
(Table 1). The availability of
these services at each site predated the initiative and did not change during
the evaluation period. We examined the availability of these services as a
potential cofactor in the impact of the initiative.
Process Measures. From the group A (+ incentives; + monitoring) sites, the following data
were collected: documentation rates—the monthly proportion of children
for whom vaccination status could be documented; voucher incentive eligibility
rates—the monthly proportion of children seen who were eligible for
a voucher incentive because they needed a vaccination; and voucher incentive
delivery rates—the monthly proportion of children eligible for a voucher
incentive who actually received the incentive.
Age-Appropriate Vaccination Rates. This rate is the proportion of children 24 months of age or younger
who were documented to be age-appropriately vaccinated based on the series
currently recommended by the ACIP (including hepatitis B and Haemophilus influenzae type b). Since the group D (− incentives; −
monitoring) sites did not report immunization rates during the evaluation
period, records of all children 24 months of age or younger seen at these
sites for WIC certification or recertification during June 1997 (the last
month of evaluation) were checked against computerized immunization files,
and age-appropriate immunization rates for these sites were calculated. These
results were compared to rates obtained by identical methods from the group
A (+ incentives; + monitoring) sites.
WIC Enrollment Rates. Monthly enrollment data for each site were obtained from Illinois WIC
to evaluate the impact of the initiative on WIC enrollment.
Statistics. Statistical tests were not applied since this is a description of a
large-scale programmatic initiative rather than a report of the results of
a formal scientific trial.
The WIC clients served by CDPH were comparable to WIC clients in Chicago
as a whole (Table 1). Each of
the 4 groups of CDPH sites served a predominantly minority population, although
the racial/ethnic composition differed among groups.
Immunization Services Available Through WIC Sites. Preexisting immunization services were not comparably distributed among
groups of sites, but the distribution did not appear to favor immunization
at voucher incentive sites (Table 1).
Among the 14 sites performing voucher incentives, 5 (36%) lacked any services,
1 (7%) had an on-site nurse, and 8 (57%) were colocated with a clinic. In
contrast, all 5 sites not performing voucher incentives had available services:
2 (40%) with on-site nurses and 3 (60%) with colocated clinics.
Process Measures.Documentation Rates.
The proportion of children whose immunization status could be documented
at the certification visit started at a high level (range, 87%-100%), reflecting
the effect of previous assessment and referral activities. Thereafter, it
improved to the point where documentation was almost universal (range, 98%-100%)
(Figure 1, A).
Voucher Incentive Eligibility Rates. The proportion of children seen who were eligible to receive a voucher
incentive fell from a high of 51% in June 1996 to a low of 5% in April 1997.
It rose to 12% in the 2 succeeding months because computer problems prevented
voucher incentive delivery in May 1997 (Figure
Voucher Incentive Delivery Rates. The proportion of incentive-eligible children who actually received
the incentive began at a low level but after 4 months was consistently more
than 80%, except for May 1997, during which month WIC computer problems caused
a transient drop (Figure 1, C).
Age-Appropriate Vaccination Rates. Among the 4 group A (+ incentives; + monitoring) sites, immunization
rates rose from 56% to 89% during the 15 months of evaluation (Figure 2). Among the 10 group B (+ incentives; − monitoring)
sites, data are absent for the early phases of incentive implementation, but
the final immunization rate was identical to that of the group A sites (89%).
In contrast, the 3 group C (− incentives; + monitoring) sites showed
no improvement, despite the monitored implementation of assessment and referral;
their final vaccination rate was almost identical to the starting rate of
the group A sites (57% vs 58%). The 2 group D (− incentives; −
monitoring) sites, which had been performing unmonitored assessment and referral,
had a coverage rate at the end of the evaluation period of 42% (42/99). Using
the same computer-record survey method, coverage at the 4 group A sites for
the same month was 80% (312/391).
Immunization Services and Initiative Impact. Differences in the availability of immunization services did not account
for the initiative's impact. Among the sites performing voucher incentives,
the final vaccination coverage at the 8 sites that were colocated with clinics
was 87% (1876/2145), slightly lower than the 91% (331/362) coverage at the
5 that had no immunization delivery services available. Among the sites not
performing voucher incentives, final coverage at the 3 sites that were colocated
with clinics was 57% (352/622) compared to 42% (42/99) at the 2 that had an
on-site nurse for vaccinations.
Race/Ethnicity and Initiative Impact. Differences in the race/ethnicity composition among sites did not account
for the initiative's impact. Among the sites performing voucher incentives,
the final coverage in the 8 sites with majority black populations was 93%
(808/873), slightly higher than the 86% (464/538) coverage in the 2 sites
serving majority Hispanic populations. Among the sites not performing voucher
incentives, the final coverage in the 3 sites serving majority black populations
was 48% (260/540), slightly lower than the 55% (21/38) in the 1 site serving
a majority Hispanic population.
Changes in WIC Enrollment. Enrollment at the 14 sites providing voucher incentives was 13874 in
April 1996 and 13066 in June 1997, while the 5 sites not providing incentives
had an enrollment of 2742 in April 1996 and 2765 in June 1997 (Figure 3). Over the evaluation period, average net monthly enrollment
(reflecting both attrition and addition of new clients and expressed as a
fraction of starting enrollment) was the same at the incentive and nonincentive
sites. Several large group B (+ incentives; − monitoring) sites instituted
computerization for WIC eligibility processing, which created increased waiting
periods for clients. Since WIC clients are permitted to transfer their care
to a site of their choice, the increased waits apparently caused a loss of
enrollment at these sites, from which some of the nearby group A (+ incentives;
+ monitoring) sites were beneficiaries.
This study was designed to answer a question that is often neglected
in health services research: can an intervention that has been demonstrated
to be efficacious in the controlled environment of relatively small, time-limited
intervention trials actually prove effective in the "real world"? The population
targeted by the Chicago WIC immunization initiative appears appropriate for
answering such a question, since this population comprises almost one third
of the birth cohort of the third largest city in the nation, all with family
incomes of 185% or less of the defined poverty level, and 80% black or Hispanic.
The results suggest that the programmatic implementation of voucher incentives
was associated with a marked rise in immunization rates in this population
and did not appear to be associated with a decline in WIC enrollment.
Prior to the evaluation period, CDPH sites had been assessing the immunization
status of enrolled children and referring those in need of vaccination to
a provider. This produced a high level of immunization documentation, a result
that is consistent with previous studies,8,9
but there was no apparent impact of assessment and referral on immunization
rates. In our study, 5 sites provided only assessment and referral during
the evaluation period. Of these, the 3 group C (− incentives; + monitoring)
sites showed no improvement in coverage during a 5-month period of monitoring
(62% to 57%), and the 2 group D (− incentives; − monitoring) sites
had 42% coverage at the end of the evaluation period.
In contrast, voucher incentives appeared to have a marked effect on
immunization rates, raising coverage to 89%. This final high rate is remarkable,
given the inclusion of all ACIP-recommended antigens. The impact did not appear
to be attributable to any other identifiable factor: project monitoring (the
intensively monitored group A and relatively unmonitored group B voucher incentive
sites reached the same 89% final coverage); the availability of immunization
services at the WIC site (final coverage at voucher incentive sites with no
immunization services was 91% compared to 87% for those colocated with a clinic
that administered vaccinations); or the racial/ethnic composition of the population
served (final coverage for voucher incentive sites serving majority black
populations was 93% compared to 86% for those serving majority Hispanic populations).
At the 5 sites not performing voucher incentives, final coverage was low (42%-57%),
regardless of whether the site was monitored, was colocated with a clinic
or had a nurse administering vaccinations on-site, or was serving a majority
black or Hispanic population.
Changes occurred in enrollment rates but were of small magnitude, and
average enrollment was the same for incentive and nonincentive sites. Trends
among groups were apparently attributable to WIC changes (eg, computerization
delays causing clients to seek transfers) rather than immunization activities.
This is consistent with the results of a recent focus group study, in which
mothers enrolled in WIC did not mention voucher incentives as a reason to
drop out of WIC,19 as well as with intervention
trials that did not show an impact of voucher incentives on enrollment.8- 11
This evaluation was limited by the use of existing data and by the potential
confounders that may be present when subject populations have not been highly
characterized. The immunization rates in this study (based on age-appropriate
receipt of all recommended antigens) are lower than would be produced by a
provider-verified survey of the same population based on 4:3:1 series completion
by 24 months of age.
The annual labor cost of providing voucher incentives in Chicago was
relatively modest ($271 000), but of the 65 federal immunization grantees
(50 states plus 15 cities), only 7 are known by the Centers for Disease Control
and Prevention to be providing immunization voucher incentives as a part of
WIC. The rest mainly offer assessment and referral, although no study to date
has established any impact for this intervention.8,9,11
The Chicago experience illustrates the potential value of using the
results of intervention trials to formulate strategies for public health problems.
But beyond science, Chicago made several management decisions that may have
been responsible for the initiative's success: dedicated staff were hired;
these staff were provided with dedicated supervision; the intervention was
phased in site by site rather than begun abruptly at all sites; a small number
of sentinel sites were intensively monitored to detect implementation problems;
and vaccination rates were obtained monthly from all intervention sites to
assess outcome. In addition, the Chicago WIC staff were highly cooperative
with the initiative. A different impact might have resulted had the initiative's
funds been used instead to reimburse a reluctant WIC for generalized but unmonitored
immunization activities by existing hardworking WIC staff, with outcomes that
were never measured. The applicability of Chicago's scientifically based and
carefully managed approach needs to be explored for a wide range of inner-city
Since the manuscript was accepted for publication, all 19 CDPH sites
have begun providing voucher incentives. By March 1998, coverage for the 5
sites that did not provide incentives during the study period had risen to
89% (694/777). Coverage in the other 14 sites, which continued providing voucher
incentives, was 91% (3746/4111). WIC enrollment at all 19 sites was stable:
16001 in March 1998, compared to 16616 in April 1996, the month before the