Impact of Community Volunteers on Immunization Rates of Children Younger Than 2 Years

Objective  To assess the effectiveness of a volunteer-driven outreach program on immunization rates in children younger than 2 years.

Design  Randomized controlled trial.

Setting  Pediatric ambulatory clinics in New York, NY.

Patients  A total of 163 children were randomly assigned to receive services from the volunteer-driven outreach program or to serve as control subjects. All children were (1) younger than 2 years, (2) no-shows for a scheduled appointment in the pediatric clinic, and (3) due or overdue for a vaccine.

Intervention  Immunization outreach, tracking, and follow-up were provided by community volunteers throughout follow-up (mean, 6.5 months). Control children were notified of immunization status at enrollment but received no further contact until the conclusion of follow-up (mean, 6.4 months).

Main Outcome Measure  Immunization status 6 months after enrollment.

Results  Significantly more intervention children were up-to-date with their vaccination series than controls (75% vs 54%; P=.03). Children in the control group were 2.8 times more likely to be late for a vaccine than intervention children (odds ratio=2.8;P=.02). In addition, an immunization delay of longer than 30 days at enrollment was a significant predictor of final immunization delay (odds ratio=2.6;P=.02).

Conclusions  This volunteer-driven program significantly improved immunization rates among intervention children compared with controls. Results confirm previous findings that indicate an increased risk of an incomplete immunization series by 2 years of age among children who fall behind early in their primary vaccination series. However, control children were almost 3 times more likely to be late (for ≥1 vaccines) than intervention children, regardless of whether an earlier immunization delay was present.

OUR PRESENT immunization delivery system does not adequately vaccinate preschool children, and the negative impact is borne unevenly. Children at greatest risk for incomplete immunization are those from urban areas, minority populations, and families living in poverty. Other risk factors include parental level of education, family size and structure, presence of social problems within the family, age at first vaccination, and lack of insurance coverage.^1-14 Responsibility for the incomplete immunization of preschool children also extends to pediatric health care providers because missed opportunities to vaccinate and organizational barriers in clinics have been widely implicated as contributing to the less than complete immunization status of this vulnerable group.^15-26 Activities to improve low rates have included computer-generated tracking, vaccine reminder systems, and clinic personnel reserved for outreach or escort services.^27-33 Although these interventions have often positively affected immunization rates, the associated costs, complexity, or lack of significant impact have precluded their wider implementation. The use of community volunteers as a resource to assist in immunization completion efforts may be a feasible alternative, but evaluations of the effectiveness of this approach have received only limited study. A volunteer-based outreach, referral, and follow-up trial³⁴ conducted with illiterate and semiliterate volunteers from poor communities in urban Bangladesh resulted in 96% and 87% immunization completion rates of referred women and children, respectively. In addition, use of community volunteers in Kampala, Uganda, was found to be cost-effective and practical for immunization outreach and follow-up during a measles epidemic.³⁵ Of note is the paucity of data regarding the impact of US voluntary agencies on childhood immunization rates.

In 1993, a metropolitan New York, NY, volunteer group (part of an international charitable organization) initiated a volunteer-driven program in response to the high rates of underimmunization in certain areas of New York. The project was designed to raise childhood immunization rates by identifying families who require immunization services, facilitating the family's receipt of services, and subsequently tracking the family to ensure that the immunization series was completed.³⁶ This study's primary objective was to assess the impact of the volunteer-driven program on immunization rates in children younger than 2 years. The study hypothesis was that children at risk for underimmunization whose families received the volunteer-driven intervention (ie, immunization outreach and referral, immunization tracking and reminders, and continued immunization follow-up from community volunteers) would have a significantly higher percentage of up-to-date immunizations than a control group receiving no such intervention.

All children and their families resided in northwestern Manhattan, NY. Children were enrolled patients in 1 of 2 ambulatory pediatric clinics of a major medical center. Both clinics were located within 4.8 km of families' homes. The clinics were selected as sites for identification of eligible children and as referral clinics for immunization services. The clinics provide primary care to predominantly low-income children who are part of a large, highly mobile, immigrant community originating from the Dominican Republic. Comprehensive health care services are provided by a combination of pediatric attending staff, pediatric nurse practitioners, pediatric residency staff, and registered pediatric nurses. The area also contains several small, private medical offices that likewise offer vaccination services. Many of the local area providers are also of Dominican Republic origin, and families often use both the medical center and private providers.

The institutional review boards of Columbia-Presbyterian Medical Center and the New York City Department of Health, both in New York, approved this study before its implementation.

Considerations involved in the determination of sample size included the number of volunteers to be involved in the study and their outreach capacity. In addition, because of intricacies related to the funding source, there was a restricted time line for study completion. Given these factors, sample size was calculated on the assumption that 40% of children would be up-to-date with their immunizations at enrollment and that 60% of intervention children would be up-to-date at the final visit. To detect the 20% difference between the groups (with a significance level of 5% and power of 80%) the estimated sample size was 107 children in each group.³⁷

A total of 434 consecutive children met the eligibility requirements for study inclusion: (1) younger than 24 months and residing in northwestern Manhattan; (2) immunization deficient by clinic chart review; and (3) a no-show for a scheduled appointment at either of the 2 ambulatory clinics. Children were considered immunization deficient when they exceeded by 30 days the due date for receipt of diphtheria and tetanus toxoids and pertussis, polio,Haemophilus influenzae type B, hepatitis B, or measles-mumps-rubella vaccines (given their age), based on the schedule announced by the Bureau of Immunization, New York City Department of Health. This schedule is consistent with the immunization schedule recommended by the Advisory Committee on Immunization Practices.

During a 7-month enrollment period (December 1995 to July 1996), children who did not attend appointments were identified through weekly review of the pediatric clinic appointment logs. Age and immunization information was obtained from the patient chart, and (after the child was identified as eligible) data regarding address, insurance, and clinic visit history were obtained from the hospital database. Children meeting the eligibility requirements subsequently received assignment (to intervention or control groups) based on a list of computer-generated random numbers.

Because children were eligible for study inclusion only if they had not attended their clinic appointment, it was not possible to secure consent for study participation until a home visit was made. Furthermore, because the home visit was to be conducted by either control group interviewers or community volunteers, randomization was necessary before the family's consent for study participation. However, before a home visit was attempted, a bilingual letter of study introduction was mailed to all eligible families. Families that declined study participation in response to the introductory letter (or at the initial home visit) received no further contact from study personnel. If the family did not decline a visit, contact was attempted by the respective interviewers.

At the initial home visit, intervention group families received basic immunization education and referral from the community volunteers. During subsequent contacts (home visits or telephone calls) throughout the remainder of follow-up (a maximum of 6 months), families were reminded of upcoming vaccinations and were recontacted to ensure that requisite vaccines were received. In addition, if a family required support or assistance to obtain immunization services (such as contacting the clinic or an escort for appointments), the volunteer would oblige. The individual volunteers were organized by the coordinator from the local branch of the larger international charitable organization. The coordinator contacted the research study director regularly to obtain locating information and data collection forms for eligible families (that she subsequently disseminated and retrieved from the volunteers) and to review data collected. In addition, the coordinator maintained a system of record keeping that included (1) families referred to the volunteer group; (2) individual volunteers assigned to referred families; and (3) referrals that were outstanding or closed or that had information needing clarification.

Control families, by contrast, were informed of their child's immunization status at the enrollment visit by the control group interviewer and were instructed to reschedule the missed appointment. Control families received no further intervention or contact from study personnel until the conclusion of follow-up (approximately 6 months after the enrollment visit). Control group interviewers were paid research personnel who were organized and supervised by the research study director (K.B.).

Control group and intervention group interviewers were trained by the study director and followed scripted interview formats. Immunization data were collected from families at the initial home visit and the final home visit and, for intervention families, throughout the study. All interviews were obtained by either the control group interviewers or members of the volunteer group. A minimum of 5 months was allotted for follow-up after each family's initial visit. As such, attempts to contact control families for their final visit began a minimum of 5 months after their enrollment date. The final visit for intervention families was defined as the last visit before the conclusion of follow-up (a minimum of 5 months) or the date the child completed the full preschool vaccination series. Control group and volunteer interviewers recorded the amount of time spent with families at each contact and the activities required to contact families. All data collection forms were reviewed for completeness, accuracy, and reliability. The control group interviews were not focused specifically on immunization but, rather, relevant immunization information was collected as part of a more general "healthy child" visit (basic growth and development and nutrition questions were included in the interview).

Determination of each child's immunization status (at enrollment and final visits) was ascertained from the home-based immunization record. In a small pilot before the study's full implementation, it had been determined that the clinic-based immunization record was not an adequate proxy for the home-based card. In addition, because a child's immunization card served as "proof" of vaccination for any day-care, school, or summer camp attendance, it was believed that this study should adopt the same pragmatic philosophy. All participating families, except 3, produced their child's immunization card at enrollment. These 3 families reported that their immunization cards were irretrievably lost (although, reportedly, immunizations had only been administered at the study-affiliated pediatric clinic). Therefore, at their enrollment visits, the clinic-based immunization record was used for data collection (with the clinic subsequently generating a "new" card). At the final visit, data were collected from the home-based records because all families possessed an immunization record. Children were classified as up-to-date (for age) if they had the appropriate dosing, number, and spacing of diphtheria and tetanus toxoids and pertussis, Haemophilus influenzae type B, polio, measles-mumps-rubella, and hepatitis B vaccines. Children were classified as late for a vaccine if they were 30 days beyond the due date for any vaccine (given their age).

Data were entered into Epi Info (version 6.04; Centers for Disease Control and Prevention, Atlanta, Ga) with subsequent export to SPSS for Windows (version 6.1; SPSS Inc, Chicago, Ill) for statistical analysis. Comparisons between groups were performed with Student t tests of independent sample means for continuous variables and χ² comparisons of categorical variables with continuity correction. Multivariate logistic regression modeling was used to determine the independent effect of group membership while adjusting for immunization status at enrollment.

Of the 434 children eligible for study inclusion, 50.2% (218/434) were randomized to the intervention group and 49.8% (216/434) to the control group. There were no significant differences between the groups regarding measured demographic and immunization-related variables. Among families eligible for study inclusion, 37.5% (163/434) consented to participation; 21.2% (92/434) refused participation; 14.3% (62/434) were currently living out of the country or New York State; and 27.0% (117/434) were not able to be located or contacted to discuss study participation. Student t test and χ² comparisons of data from enrolled (n=163) and nonenrolled (n=271) children revealed no significant differences except in insurance coverage. A significantly greater percentage of nonenrolled children were covered by Medicaid insurance than were enrolled children (83% vs 73%; P =.02).

Descriptive analysis of enrolled children revealed that most (69.9%) were between 2 and 12 months of age at enrollment, of Hispanic origin (82.2%), boys (55.2%), and insured by Medicaid at the time of their last clinic visit (73.0%). The average number of clinic visits attended and length of time followed up in the clinic (before the missed appointment) was 4 visits and slightly less than 5 months. Age of the cohort group at the final visit was slightly older than 15 months, and they had participated in the study (calculated as the length of time between enrollment and the final visit) for just longer than 6 months. Assessment of immunization status at enrollment (using the home-based record) revealed that 36.8% of children were up-to-date (for age) with immunizations, 42.6% were late (ie, beyond the 1-month grace period for vaccination based on the child's age), and 20.6% were due for an immunization (ie, within the 1-month grace period). Eight children had completed the full immunization series (ie, 4 diphtheria and tetanus toxoids and pertussis/ Haemophilus influenzae type B, 3 polio, 3 hepatitis B, and 1 measles-mumps-rubella) at the enrollment visit. Because there was no opportunity for change regarding the outcome variable for these children, they were excluded from analyses of final visit variables. At the final visit, 62.9% of children were up-to-date for age with their immunizations, whereas 29.5% were late (ie, not up-to-date for age) with their vaccination series. This represented a 26.1% improvement in the percentage of children up-to-date for age between their enrollment and final visits.

Comparative analyses of intervention and control children are summarized in Table 1. Of note is the lack of significant difference between the groups regarding measured demographic variables. Immunization data are summarized in Table 2. By the final visit, significantly more intervention children were up-to-date for age with vaccines compared with controls (75% vs 54%; P =.03), and significantly fewer intervention children were late for 1 or more vaccinations (18% vs 38%; P =.03). In the intervention group, there was an increase of 41% in the number of children who were up-to-date and a 23% decrease in the number of children who were late for a vaccine compared with enrollment figures. This is in contrast to control children, whose percentage of "late" was only slightly changed at the final visit (38%) compared with enrollment levels (44%). There was, however, a 15% increase in the percentage of children up-to-date for age at the final visit compared with enrollment levels (39% vs 54%).

In a multivariate analysis of predictors of immunization delay, the effect of group membership was adjusted for immunization status at enrollment. As summarized in Table 3, late immunization status and group membership were significant predictors of an immunization delay (>30 days) at the final visit. Children in the control group were 2.8 times more likely to be late with their immunizations at the final visit than children in the intervention group (odds ratio=2.8; P =.02). Moreover, children who were late for a vaccine at enrollment were also at significant risk of an immunization delay at the final visit (odds ratio=2.6; P =.02). However, it is important to recognize that the positive effect of the intervention was independent of the delay present at enrollment.

We examined the differences between cohort children (n=132) and those lost to follow-up (n=23). There were no significant differences between the 2 groups except for the length of time followed up in the clinic (before the missed appointment). Cohort children were registered with the clinic significantly longer than children who were lost to follow-up (4.8 vs 2.9 months; P =.04). In addition, although 85.2% (132/155) of the original cohort of enrolled children received a final visit, the follow-up rate for intervention children was 79% and for control children was 90% (Table 1). Last, within-group comparisons (intervention: cohort vs lost; and control: cohort vs lost) revealed no significant differences, with 1 exception: within the control group, cohort children were significantly older at enrollment than those lost to follow-up (9.0 vs 6.5 months; P =.03).

In summary, most eligible children who were enrolled did not seem to differ significantly from those who were not enrolled regarding measured demographic and immunization variables. Likewise, intervention and control children did not differ significantly regarding the same variables at eligibility and enrollment. By the study's conclusion, however, intervention children had significantly better immunization completion rates than controls who were 2.8 times more likely to to be late for 1 or more vaccines (given their age). Slightly more than 85% of children who were originally enrolled in the study received a final visit (79% of intervention children and 90% of controls). Children who were lost to follow-up, on the whole, did not differ significantly from cohort children.

Results of this study demonstrate the significantly positive impact that a volunteer-driven, community-based immunization outreach, referral, and follow-up intervention can have on the immunization rates of a vulnerable preschool population. Community volunteers were able to significantly increase the percentage of up-to-date immunizations (and, thus, decrease the percentage of late immunizations) among their randomly assigned group of children compared with control children.

The results confirm previous findings^4,13,17 that indicate an increased risk of immunization delay at 2 years of age for children who fall behind early in their vaccination series. However, intervention group membership significantly affected the odds of immunization delay independent of a delay at enrollment. Thus, a child who did not receive the intervention had nearly 3 times the odds of having an incomplete immunization series at the study's conclusion (given their age), regardless of whether an earlier immunization delay was present.

The study results are further strengthened by the fact that all children (intervention and control) received an initial home visit. Although the control group visit was not specifically immunization focused, families did receive immunization-related activities that are typical of outreach programs: (1) discussion of vaccinations as an important part of preventive health care, (2) assessment of each child's immunization status (with that information communicated to the caregiver), and (3) instructions to arrange immunization services. In contrast, intervention families received all the above but in addition were reminded, tracked, and revisited by community volunteers. It seems that simply informing a family about the immunization status of their child and the timing of the next vaccine (with referral back to the provider) was not sufficient to avoid (or correct) an immunization delay to a significant extent, although it is acknowledged that many factors are involved in a child's receipt of immunization services. Our results, however, call into question the role of patient education (as a single strategy) and its impact on childhood immunization rates.

Concern regarding the generalizability of our findings is warranted, given the study design. Although children were randomly assigned to either group, there was an attempt to enroll all children from the ambulatory clinics who were deemed eligible. As such, our study population is not representative of underimmunized preschool children in New York City as a whole.

Another important consideration is selection bias. Specifically, 21.2% of the 434 eligible children declined study participation, whereas 27.0% were not able to be contacted and 14.3% were not currently living in the study area. This nonparticipation rate was not totally unexpected, given that study enrollment required a minimum of 2 home visits and the study population was a predominantly immigrant population that historically is highly mobile, distrustful of large institutions, and concerned with immigration and benefit issues. Although comparisons between enrollees and nonenrollees did not reveal any major differences, there exists the possibility that a degree of selection bias may have gone undetected by the general demographic and immunization-related variables on which the 2 groups were compared. Regarding future study with a similar population, prospective researchers may want to consider measures (eg, allowing a longer time line for study enrollment) to potentially reduce nonparticipation rates.

The same concerns apply to enrolled children who were lost to follow-up. Although the overall percentage who were eligible for a follow-up visit was only 14.8% (23/155), cohort children were registered in the clinic (before a missed appointment) for significantly longer than children lost to follow-up (cohort mean, 4.8 months; lost to follow-up mean, 2.9 months; P =.04). It is conceivable that cohort children had a more established relationship with the clinic and were, therefore, more likely to continue their participation in the study and to improve their immunization status, regardless of group membership. However, it is very unlikely that the observed difference between the immunization rates of intervention and control children can be adequately explained by a patient-clinic relationship.

The intervention children who were lost to follow-up, however, are a concern, especially regarding the possibility of a spuriously elevated effect of the intervention. Although the percentage of intervention children lost to follow-up was not significantly different from the percentage of control children lost to follow-up (21% vs 9.5%; P =.07), there were slightly more than twice as many intervention children who did not receive a final visit compared with controls. If all the intervention children lost to follow-up were also immunization delayed, the effect of the intervention would be artificially magnified. The potential effect of this scenario was evaluated by repeating the multivariate analysis with the assumption that all children lost to follow-up (intervention and control) were immunization delayed. The effect of the intervention subsequently lost significance (Wald=1.13; P =.29); however, even in this unlikely situation, control children (6 months after enrollment) were almost 1.5 times more likely to be late with their vaccines compared with intervention children (odds ratio=1.4). An argument could be made that a lower threshold of intervention effectiveness would be acceptable given the advantages of a volunteer-driven program. The study results, therefore, must be interpreted within this context.

Although further research is necessary to confirm our results, the success of the volunteer group in improving childhood immunizations among intervention children is thought to be related to several factors. First, the community volunteer group was exceptionally well organized, self-contained, and committed. In addition, most of its members (60%) lived in the same neighborhood as participating families and were bilingual (85%). The volunteer group had a coordinator who managed the volunteer tracking of referred families, volunteers involved, and immunizations outstanding. The coordinator also contacted the research team regularly to exchange information for the referred families; effective and accurate data exchange was essential. Likewise, important to the success of the research project was (1) the commitment of ambulatory clinics to the volunteer group collaboration, (2) good communication with the volunteer coordinator, and (3) matching of the clinic outreach needs and volunteer group capacity.

Drawbacks of the service provider–volunteer partnership include the need for a high degree of self-direction, organization, and commitment from the coordinators and volunteers. In addition, another potential drawback to a volunteer-driven collaboration are unavoidable shifts in volunteer efforts as members juggle their volunteer activities with other responsibilities, such as work, school, and family. During this study, the volunteer group experienced organizational difficulties at the end of enrollment and follow-up periods that may have contributed to the slightly lower (although nonsignificant) percentage of children enrolled and followed up compared with the control group—which did not have such fluctuations. Last, a discussion of collaboration limitations would not be complete without mentioning the possibility of secondary objectives from the volunteer group. It is imperative that volunteers not abuse their access to families as a way to advance an agenda that is not shared by the health care organization, including solicitation of funds or commercial opportunities from participating families, distribution of literature not approved by the health care organization, and recruitment into the volunteer organization itself (or in the case of volunteers from a religious group, proselytizing). In this study, volunteer services were assessed twice for a secondary agenda, and every volunteer signed a contract that documented their understanding of, and agreement with, outreach activities that were and were not acceptable. Periodic monitoring of volunteer activities would be a responsibility of the provider organization.

The major benefit of a provider organization and volunteer group collaboration is the absence of expenditures for outreach services provided by volunteers. This might afford clinics that would not otherwise provide any type of outreach services an opportunity for limited community involvement of documented benefit. In addition, a health care agency is likely to receive information that allows for improvement of patient care and clinic efficiency (ie, accurate information regarding a family's whereabouts, a likely decrease in no-show appointments, and the opportunity to provide home-based health education). Few clinics serving poor populations can afford patient-centered services such as home visiting, clinic reminders, and community follow-up. For example, home visits in the pediatric clinics involved in this study were reserved for evaluation of only the most complex and threatening situations; rarely was there any outreach or follow-up after the failure of a routine appointment. Without volunteer collaboration, it would have been impossible to speak with families in their homes regarding the importance of timely immunization, and it is likely that there would have been no significant improvement in the immunization rate of some of the clinic's most vulnerable children during the 6-month period.

Last, the potential applications for a provider organization and volunteer group collaboration are almost unlimited; the model can be adapted to any program in which outreach services would be beneficial. A similar collaboration in a specialty clinic or local health department that addresses a variety of health concerns is easily envisioned.

Opportunities for further study include testing the model in other populations and on a larger scale to confirm our results. In addition, it would be an important next step to determine the specific administrative efforts of the volunteer group and provider organization that are required in the collaboration and to examine additional significant outcomes of the volunteer group. As part of this work, a cost-benefit analysis of the volunteer model would logically follow.

In conclusion, the volunteer-driven outreach intervention evaluated in this study was successful in significantly improving the immunization rates of a vulnerable group of preschool children during a 6-month period. In addition, infants who were immunization delayed early in their vaccination series seemed to be at risk for ongoing delays and should be considered for focused follow-up. The success of the program was no small feat, given that the study population was highly mobile and the program was largely driven by volunteers and implemented in a short time. The collaborative model is easily adapted to a variety of clinical settings and health concerns and is low cost (although not cost free). It seems, therefore, to be a practical and innovative method for increasing a health care provider's effect on immunization rates that reaches beyond the clinic doors. These returns, however, need to be balanced against the prerequisites of an exceptionally well-organized and committed volunteer group and vigilance on the part of the service provider to protect families from competing agendas. Thus, the programmatic scale is tipped by the health care problem that the outreach program is attempting to address and the availability of resources that would allow the collaboration to function.

Accepted for publication October 9, 1998.

This project was supported by a grant from the New York City Department of Health, Immunization Action Plan, New York, NY.

Presented at the Association of Public Health Annual Conference, Lancaster, England, March 25, 1998, and the American Public Health Association conference in Indianapolis, Ind, November 11, 1997.

We thank the staff of the Bureau of Immunization, New York City Department of Health, and the staff of the Ambulatory Care Network Corp, Audubon and Broadway Pediatric Clinics, New York, for their patience and cooperation during the study. Ms Barnes particularly thanks Julie Sochalski, PhD, Diana Romero, PhD, Lucy Reyes, and Anna Small, MSc, for their help throughout, and she especially thanks Jonathan Carr, MSc, without whose support this project would never have made it to completion.

Corresponding author: Katie Barnes, CPNP, MSc, MPH, School of Health and Applied Sciences, Suffolk College, Education Centre, Ipswich Hospital NHS Trust, Ipswich, Suffolk IP4 5PD, England (e-mail: KBarnes574@aol.com).

Editor's Note: This volunteer-driven study shows much promise, especially considering that if you can do it in New York, you can do it anywhere.—Catherine D. DeAngelis, MD