Kerpelman LC, Connell DB, Gunn WJ. Effect of a Monetary Sanction on Immunization Rates of Recipients of Aid to Families With Dependent Children. JAMA. 2000;284(1):53-59. doi:10.1001/jama.284.1.53
Context Immunization rates among low-income families have lagged behind those
for the general community, with several possible barriers cited in the literature.
Objective To evaluate the effect of an initiative aimed at improving immunization
rates among low-income preschool children by imposing a sanction on families
who failed to provide proof of up-to-date immunization status.
Design and Setting Randomized, controlled before-after trial conducted from January 1,
1993, through December 31, 1996, in Muscogee County, Georgia.
Participants A total of 2500 families with children aged 6 years or younger who received
Aid to Families with Dependent Children assistance.
Intervention Families in the intervention group (n=1500) were informed that receipt
of the welfare benefit for any preschool-aged children was contingent on provision
of proof of up-to-date immunization status at the beginning of welfare eligibility
and, subsequently, semiannually or annually. Case families in the control
group (n=1000) were encouraged to immunize their preschool children but were
not informed of any aid sanctions nor did such sanctions apply to them.
Main Outcome Measure Age-appropriate rates of 5 immunizations (measles-mumps-rubella; poliovirus;
diphtheria and tetanus toxoids and pertussis; Haemophilus
influenzae type b; and hepatitis B), based on examination (with family's
written consent) of medical provider records, compared among intervention-group
vs control-group families.
Results There were no significant differences at baseline between intervention
and control families in immunization rates of preschool children. Families
in the intervention group were significantly more likely than families in
the control group to have up-to-date immunization status in all 4 years of
the study for all 5 immunizations (with 3 exceptions). At age 2 years, 72.4%
of children in the intervention group vs 60.6% of those in the control group
achieved vaccine series completion, which included 4 diphtheria and tetanus
toxoids and pertussis, 3 poliovirus, and 1 measles-mumps-rubella (P<.001). Sanctions were implemented only 11 times. There was relatively
little increased burden on the part of families to comply with requirements.
Conclusion In our study, a monetary sanction in a population receiving welfare
benefits stimulated a significant increase in childhood immunization rates,
suggesting that when welfare recipients are given an incentive to keep their
children's immunizations up-to-date, most are able to do so.
Although immunization rates have increased in general over the last
decade,1,2 the rates for poor
and minority children have not kept pace.3,4
Robinson and colleagues5 have identified several
risk factors that highly correlate with failure to immunize children on schedule,
including having low parental education level, large family size, low socioeconomic
status, or ethnic or minority group membership; receiving services through
public health clinics; being a single parent; starting the immunization series
late; and having inadequate insurance coverage for immunizations. Others have
indicated that the difficulties poor families have in keeping their children's
immunizations up-to-date result from deficiencies in the health care system,6,7 and a lack of transportation, social
support, and understanding of immunization schedules.8,9
These factors strongly suggest that families receiving welfare assistance
are at risk of not having their children immunized in a timely fashion.
In the 1990s, the federal government encouraged states to seek waivers
of federal welfare rules to test innovative programs and policies, including
those that aimed at improving the health of public aid recipients. These waivers,
which carried with them an obligation to evaluate the effects of the innovative
programs and policies, allowed states to engage in experiments to encourage
behaviors among welfare recipients that would lead, among other things, to
better family health. Twenty-one states applied for health-related welfare
waivers.10 After the passage of the Personal
Responsibility and Work Opportunity Act of 1996, which gave states greater
flexibility in administering their welfare programs, many states ceased evaluation
activities. To our knowledge, of the states that received waivers for encouraging
immunization, only Georgia and Maryland have completed program evaluations.
We present herein the results of a 4-year evaluation of the effect of
the Preschool Immunization Project (PIP), a large-scale immunization initiative.
The PIP was designed and implemented by the Georgia Department of Human Resources,
Division of Family and Children Services, the agency responsible for administering
that state's Aid to Families with Dependent Children (AFDC) program. Although
the 1996 welfare reform legislation replaced AFDC with Temporary Assistance
for Needy Families midway through our evaluation, we refer to Georgia's welfare
program as AFDC for the purposes of consistency.
On January 1, 1993, all families (except for 1000 families in the control
group) who either applied or reapplied for AFDC benefits in Georgia were told
that they had to provide proof of up-to-date immunizations for their preschool-aged
children (≤6 years). They were reminded of their obligation both when they
applied and when they were recertified for welfare eligibility, which was
required semiannually until 1996, when it became an annual requirement. If
the family did not present such proof without good cause, such as having religious
objections or known allergic reactions, a sanction could be applied after
oral or written warnings were issued. The sanction was losing AFDC benefits
normally provided for the nonimmunized child. Medicaid benefits and those
for Early Periodic Screening, Diagnosis, and Treatment were not affected.
To evaluate the effect of the sanction, a randomized controlled trial
was instituted in Muscogee County, Georgia, using a systematic random sampling
method, by which 1500 families were randomly selected as the intervention
group and 1000 families were randomly selected as the control group. Although
families in the control group were encouraged to immunize their preschool
children, they were neither told about the sanction nor penalized for failure
to immunize them. Families in the intervention group, as all the other families
receiving AFDC in Georgia, were told about the sanction for not immunizing
their preschool children.
Muscogee County, Georgia, consists mainly of the city of Columbus (1995
population 178,681) and the Fort Benning Military Reservation. Almost all
of the sample cases were from Columbus. The PIP was approved by the US Department
of Health and Human Services, which mandated a randomized assignment evaluation
of the project by an independent organization. In doing so, the US Department
of Health and Human Services counsel indicated that the project and its evaluation
were exempt from human subjects review.
The Georgia Department of Administrative Services used a systematic
random sampling method to assign families into the intervention and control
groups in November 1992, before the project began on January 1, 1993. Data
from the state's Public Assistance Reporting and Information System files
indicated that 3600 families receiving AFDC in Muscogee County were subject
to the PIP. From those families, every third family was assigned to the control
group until 1000 cases were selected. Every third family from the remaining
2600 was assigned to the intervention group until 1500 cases were selected
To assess the incidence of the application of sanctions, we reviewed
the Muscogee County AFDC case record files of 2265 of the 2500 cases in the
study. The remaining 235 case records were not available in the Muscogee County
Age-Appropriate Immunization Rates. The main outcome measure entailed examination of medical records of
preschool-aged children to assess whether they had received their diphtheria
and tetanus toxoids and pertussis (DTP), poliovirus, measles-mumps-rubella
(MMR), Haemophilus influenzae type b (Hib), and hepatitis
B (HBV) vaccines when required. Children were considered to be up-to-date if they had received age-appropriate immunizations based
on the recommendations of the Advisory Committee on Immunization Practices
and on Georgia PIP requirements in effect at that time. A 1-month grace period
was allowed before categorizing a child as not up-to-date. We considered each
immunization separately, as well as series completion at age 2 years (4 DPT,
3 poliovirus, and 1 MMR immunizations).
Conditions for review of children's immunization records were: (1) the
child's parent or guardian gave written informed consent for the review of
medical records; (2) a local immunization provider or providers (whether private
physician or county health department) could be identified; and (3) the medical
provider(s) gave permission to examine the records. For each child old enough
to be eligible for at least 1 immunization at each time point examined, immunization
records were recorded from all providers that could be identified as having
immunized the child at any point since birth.
Five trained abstractors, from Columbus State University's Health Science
Department, recorded all vaccinations. The abstractors were blinded to the
status of the children. The record abstracts were edited, entered, verified,
and checked for internal consistency and obvious errors.
The collection of children's immunization data (through review of their
medical records) began in 1995 and continued into early 1998. All medical
record reviews completed before December 1996 were updated during 1997 and
1998. The resulting immunization data for this evaluation were current as
of December 31, 1996. Analyses of the medical records file were completed
for the children in the sample, even those whose families no longer received
Client Burden. Ten-minute-long telephone interviews were conducted between October
1995 and December 1996. Forty families in the intervention group were asked
about the extra burden imposed on them by the PIP. These interviews attempted
to determine the annual amount of time spent, time lost from work, and out-of-pocket
transportation costs required to address the immunization documentation requirements.
Rates of participation (in AFDC and in the study) were compared using χ2 analysis. The effect of the intervention on immunization rates was
assessed using logit (or log of the odds) analysis, with 4 variables entered
into the equations: intervention status (intervention/control), age (in months),
sex (male/female), and ethnicity (minority/nonminority). Children's up-to-date
immunization status was evaluated at 5 points: baseline (January 1, 1993),
after 1 year (December 31, 1993), 2 years (December 31, 1994), 3 years (December
31, 1995), and 4 years (December 31, 1996) of the demonstration's operation.
All available immunization records were included in each analysis reported
herein whether or not the child remained subject to PIP requirements (ie,
active AFDC status and age ≤6 years). Series-completion rates by age 2
years were assessed. For this analysis, records of children who turned 2 years
old during the study period (January 1, 1993-December 31, 1996) were examined
for these immunization series.
Sample Characteristics. As of January 1, 1993, the intervention group included 2488 children
(1.66 per family); the control group included 1662 children (1.66 per family).
At that time, the average age of the preschool-aged children was 3.22 years
for the intervention group and 3.34 years for the control group (Table 1). In each group 85% were black
and 14% were white. The intervention group had the same proportion of males
and females, but the control group had slightly fewer males proportionally
(48.5%). During the study, the 278 infants born in the control group families
and 483 in the intervention group families were included in this study. Toward
the end of the study, the average number of preschool-aged children in intervention
group families was 1.98 and 1.94 in the control group families. Age breakdowns
indicate that similar proportions of children were in each age group for the
Participation in AFDC. Over the 4-year course of this evaluation, families in both intervention
and control groups left the Muscogee County AFDC program, either by income
ineligibility or by moving out of the county. In December 1993, 86.3% of the
control group and 82.8% of the intervention group remained active. That difference
in the percentages remaining active is statistically significant (χ2=5.8, P<.02). By December 1994, 76.4% of
the control group vs 71.6% of the intervention group remained active (χ2=7.3, P<.01). By December 1995, however,
that difference narrowed with 68.2% in the control group vs 66.9% in the intervention
group remaining active recipients (χ2=0.6, P<.50). As of December 31, 1996, 65.5% of the control group vs 61.7%
in the intervention group remained active (χ2=3.8, P<.05).
Study Participation Rates. The ability to collect information from the entire research sample for
the duration of an evaluation affects the reliability of observed results.
If a segment of the original sample were not involved in the analysis and
if that segment were different in important ways relevant to the measures
of interest, the final results could be biased. In this study, written consent
to review medical records was obtained for 3001 (61.1%) of the 4911 sampled
children: 1864 children (62.7%) in the intervention group and 1137 children
(58.7%) in the control group. This difference is statistically significant
Preexisting differences between families in the intervention group that
did and did not provide consent to review records were assessed by analyzing
demographic characteristics at baseline. Families providing consent had more
children and were more likely to be ethnic minorities.
Children in the intervention group for whom permission to review medical
records was received were younger than those in the control group for whom
permission was received, but all other demographic characteristics were similar
for the 2 groups. The difference in children's ages observed in this analysis
contrasts with the lack of difference in average age of children in the intervention
and control groups at the time of original family selection (Table 1).
Families in the intervention group received 17 warnings that benefits
would be decreased if parents continued not to provide proof of immunization.
Families in the control group mistakenly received 3 warnings. Of the 20 warnings,
11 resulted in sanctions being applied, all to families in the intervention
group, affecting a total of 18 children. Eight of the 11 sanctioned families
reapplied and were approved for benefits. The period required for reactivation
ranged from 1 month (as was the case for 5 families) to 6 months.
Up-to-Date Immunization Rates. There were no differences between the immunization rates of intervention
and control groups at baseline. After baseline, immunization rates were higher
for intervention group children for all immunizations in each year. Differences
after the project's first year are statistically significant for all immunizations
(P<.05), with the lone exception of Hib in the
second year (Table 2). Observed
intervention effects generally increased each year for the first 2 years,
as the PIP regulations took root in the intervention population and were maintained
in the succeeding 2 years (Figure 2).
Children's age was significantly related to up-to-date status for each
immunization at each time point, with 2 exceptions (poliovirus in the third
year and MMR in the first year). For DTP, poliovirus, and MMR vaccine, older
children were more likely to be fully immunized. However, for Hib and HBV
vaccines younger children were more likely to have completed the series requirements.
Sex was unrelated to immunization status at all time points except for
the baseline Hib vaccine, for which males were more likely to be up-to-date.
Ethnicity was unrelated to immunization for all vaccinations except HBV. For
HBV, nonminority children were more likely to be up-to-date after baseline
regardless of intervention status, age, or sex.
Analyses conducted of only those children aged 6 years or younger provided
equivalent results; in fact, stronger intervention effects were observed for
that sample alone.
Series Completion. Of the 510 intervention group children who became 2 years old from January
1, 1993, through December 31, 1996, 369 (72.4%) achieved series-completion.
Of the 340 control group children, 206 (60.6%) achieved series-completion.
This difference is statistically significant (χ2=13.4, P<.001).
The interviews with 40 families showed that the level of additional
annual burden of being required to provide immunization documentation was
0.66 hours in time spent, 0.13 hours in time lost from work, and $0.41 in
out-of-pocket transportation costs.
Age-appropriate vaccination coverage rates, although virtually identical
at baseline, increased in the intervention group after 1 year and remained
level for the remainder of the study. With just 3 exceptions, the intervention
group had statistically significant and clinically meaningful higher coverage
rates (by about 6-7 percentage points) for all 5 of the vaccines from the
project's first through fourth years.
Immunization rates for MMR and poliovirus are high (≥80% after the
first year), and the rate for DTP approaches that of these 2 immunizations
(≥70%). For the Hib and HBV vaccinations, rates are far lower (<30%).
The Hib and HBV vaccines have been a relatively recent requirement, so it
might be expected that compliance on these 2 immunizations might take several
years to reach high levels. A study of HBV vaccination rates in a similar
population for which intensive in-person efforts were made to improve immunization
coverage found similar rates of HBV coverage but with a more rapid rate of
increase in up-to-date levels.11
Given that the already high levels of immunization rates for MMR, poliovirus,
and DTP present a potential ceiling effect, the increase of as much as 7 percentage
points in immunization up-to-date rates achieved with these 3 vaccinations
is all the more remarkable. Finally, not only are the PIP's effects positive,
but the burden on families to adhere to the PIP requirements was low.
Although age of a child is significantly related to all immunization
rates, the direction of the relationship is different depending on the immunization.
A positive relationship between age and immunization rate was found for MMR,
poliovirus, and DTP, whereas a negative relationship was found for Hib and
HBV immunization rates. At the outset of the PIP, up-to-date rates for Hib
and HBV immunizations were extremely low, no doubt because inclusion of these
2 in the immunization series was fairly new. It appears that as the original
sample of children aged, little attempt was made to bring them up-to-date
for these vaccines, but children born into the sample as the PIP progressed
were being immunized more routinely with Hib and HBV vaccines.
The finding that minority children were significantly underimmunized
with HBV vaccines appears contrary to 1997 data from the National Immunization
Survey.4 In our study, that difference progressively
decreased with time, so it is possible that had data been obtained in our
samples for 1997, that difference would have disappeared.
Comparison of the intervention and control groups' demographic data
indicate that the systematic random assignment procedure was performed correctly
and had the desired result. That the intervention and control groups can be
considered statistically similar at baseline makes the intervention effects
found all the more compelling. On the other hand, given that the intervention-control
group difference among families actively participating in AFDC at most points
in the time studied is statistically significant, it is conceivable that the
intervention may have influenced some families to leave AFDC.
The net effect of the potential bias that may have resulted from families
not consenting to let their children's medical records be examined was determined
by analyzing family demographics. Although the ages of the children in the
intervention and control groups for whom medical records were examined were
different, unlike the ages of the groups in the entire sample, when adjusted
for in the logit analysis, that difference did not appear to have an effect
on the intervention results. Despite the potential bias inherent in obtaining
family consent, a record check of immunizations yields a more valid and reliable
indication of immunization status than does health care clinician or parental
recall,12 measures that are often used in other
studies of childhood immunization rates.
In any project requiring additional activities from the administering
agency and the program recipients, program implementation cannot be expected
to be perfect. Results from a process evaluation of the program showed some
difficulties in implementation. Case workers said (or at least remembered)
that they received relatively little training, and the sanction was not applied
according to agency procedures in some instances. Among the difficulties in
administering the study were (1) some control group families were treated
like intervention group members; (2) other unrelated immunization promotional
efforts were operating in the community during the same period diluting the
effect of the intervention being evaluated; and (3) during the 4-year evaluation
period, national welfare policy changes occurred that affected all state welfare
Factors such as treatment crossover and dilution of the treatment intervention
operated to mitigate the purity of this evaluation. Yet the intervention accomplished
its objectives despite these adversities, for the efforts of the PIP are decidedly
positive for ensuring childhood immunizations.
In the Georgia PIP, sanctions were seldom implemented, yet the families
to whom the threat of sanctions were made had their children immunized at
higher levels than did the families not subject to such sanctions. The sanction
intervention undoubtedly served several purposes. Semiannually, at first,
then annually, it focused attention on a desired health behavior, imparted
information about what was expected of families, and threatened monetary loss
for not carrying out the desired health behavior.
We found only 1 other completed welfare waiver evaluation of the role
of sanctions on the immunization behavior of persons receiving public assistance,
namely, Maryland's Primary Prevention Initiative (PPI). Even though many welfare
waiver projects like the PIP were initiated in the early 1990s, with planned
evaluations, changes in the 1996 welfare reform legislation caused many states
to drop both their demonstration projects and their evaluations of them.
The Maryland PPI aimed at changing a more comprehensive range of behaviors
than did Georgia's PIP, including requirements related to preventive health
care (over and above immunizations), prenatal care, and attendance at school
on the part of school-aged children in the family. Concerning immunization,
the Maryland PPI evaluation found "no consistent evidence that PPI contributed
to an increase in immunization coverage,"13
nor did it find an effect on any of the other desired health behaviors, nor
on school attendance. The evaluators of the Maryland program attributed the
total lack of effect of that program to several factors, including weaknesses
in implementing the sanction, compensatory increases in other aid to families
that were sanctioned, and inadvertent exposure of control families to 1 or
more intervention components.14
In Georgia's PIP, there were, similarly, weaknesses in implementing
the sanction, little overall intrafamilial economic impact of the sanctions,
and inadvertent exposure of control families to the intervention. However,
the PIP's improvement in immunization rates is the opposite of those found
in Maryland. It is plausible to conclude that the Georgia project achieved
its desired ends because it focused on 1 aspect of health behavior (immunizations),
which was easy to understand and required clear actions from participants.
Clients subjected to the requirements of Georgia's PIP more than likely could
comprehend readily what was expected of them to avoid being penalized and
found it relatively easy and convenient to have their children immunized.
Maryland's PPI, on the other hand, aimed its efforts at a broader range of
health (and nonhealth) behaviors. Participants in PPI may have found it harder
to understand what was required of them, to effect that broader range of health
behaviors, and thus to adhere to that project's requirements.
Linking health activities to welfare benefit payments and concomitantly
placing responsibility for their children's immunizations on the recipients
of welfare was somewhat controversial when this project was launched (and
remains so). Linking welfare payments to health-related actions on the part
of welfare recipients is arguably unfair, in that the problems it intends
to address are due more to problems in the health care delivery system than
to the family itself.15 Families receiving
welfare may have particular difficulties keeping their children up-to-date
on immunizations due to the relative scarcity of Medicaid providers, systemic
barriers (such as restricted clinic hours), and lack of relevant information.
In fact, the Center for Law and Social Policy called for a moratorium on the
AFDC grant reduction approach "until evaluations of it and other approaches
provide information about the relative effectiveness of different approaches."16 Yet the fact remains that poor children are at higher
risk of vaccine-preventable illnesses. It is both a public health obligation
to encourage low-income parents to have their children immunized for these
diseases and a benefit to these families and to the public as a whole. There
are other advantages, as well. Although immunization rates are high at school
entry,17 there are delays in vaccine administration
before school entry that a program like this addresses. Furthermore, immunization
visits provide an opportunity for other well-baby or well-child services to
be provided. Although vaccine-preventable illnesses may pose no obvious danger
now, the recent measles outbreak suggests that the public health armamentarium
should include effective means to increase immunizations by implementing programs
similar to the PIP.
The results of our evaluation suggest that when low-income families
are given the incentive to keep their children up-to-date on immunizations
and are reminded regularly of this, they are able to do so. The threat of
a penalty and regular reminders about the important positive measure they
could take for their children provided enough of an incentive and focus for
parents to have their children immunized. Even though very few families were
actually penalized by having their welfare benefits reduced, the overall effect
of the PIP was decidedly beneficial for this population.