Effectiveness of a Practice-Based Intervention to Increase Vaccination Rates and Reduce Missed Opportunities

Background  Although provider feedback and recall/reminder systems have been shown to increase vaccination rates for children, little is known about the effectiveness of less intensive interventions.

Objective  To determine whether provider prompting at acute care visits in an urban hospital-based outpatient clinic can increase vaccination rates and decrease missed opportunities.

Design and Methods  Study participants, 3 years or younger, were identified from a managed care organization as receiving primary care at the clinic. Eligibility criteria included 1 or more visits to the clinic without regard to continuity of enrollment. Patients' vaccination records were generated at nursing triage and attached to the encounter sheet. Vaccination and visit data were abstracted from medical records, and comparisons were made between baseline (n = 521) and postintervention (n = 642) groups for up-to-date vaccination rates, missed opportunity rates, and mean numbers of visits.

Results  Up-to-date rates at the age of 24 months for 4 diphtheria and tetanus toxoids and pertussis, 3 polio, 1 measles-mumps-rubella, 3 hepatitis B, and 3 Haemophilus influenzae type b vaccines changed from 70% to 78% (P = .07). Up-to-date rates increased significantly to 87% among the subset of children continuously enrolled in the managed care organization and the practice (P<.01). Overall, mean numbers of visits were similar. Missed opportunity rates among children not up-to-date for 4 diphtheria and tetanus toxoids and pertussis, 3 polio, 1 measles-mumps-rubella, 3 hepatitis B, and 3 Haemophilus influenzae type b vaccines at the age of 24 months declined from 65% to 45% (P = .04). Similar trends were noted at the age of 10 months.

Conclusions  In the absence of increased funding, minor changes in standard operating procedures may improve vaccination delivery. Further improvements may require efforts to ensure continuity of provider and plan assignment.

INCREASING THE proportion of children with up-to-date (UTD) vaccinations to 90% by the age of 2 years remains a national objective of Healthy People 2010.¹ Failure to achieve this goal has been attributed, in part, to ineffective provider practices, which lead to missed opportunities.^2-9 Provider recall/reminder and assessment and feedback interventions have been shown to improve childhood vaccination rates¹⁰ and have been strongly recommended by the US Task Force on Community Preventive Services.¹¹ However, limitations of these established interventions include high resource intensity and the retrospective nature of assessment and feedback once the patient has left the office.

Variation in immunization status measures also contributes to the difficulty of interpreting results across intervention studies. These variations include differences in selected combination measures, acceptable timing for receipt of vaccinations, ages used to calculate measures, and sources of vaccination histories.¹² In addition, prior studies often limit evaluations to active patients with a minimum number of visits and continuous enrollment in a health plan or practice. The degree to which such inclusion criteria impact vaccination rates remains unknown. However, such inclusion criteria may lead to an overestimate of program effectiveness and vaccination rates by excluding children who are at high risk for being not UTD.

This study determines (1) whether low-intensity provider prompting at acute care visits can increase vaccination rates and decrease missed opportunities for young children receiving care at an urban primary care pediatric clinic and (2) the impact of continuous enrollment in a health plan and pediatric practice on vaccination rates and effectiveness of the intervention.

This prospective study took place in an urban hospital-based pediatric clinic in which acute care is provided by a team of residents, medical students, nurse practitioners, and preceptors, including general pediatrics fellows and faculty. The residents and students providing acute care rotate on a monthly basis. Patients receiving care at the clinic were predominantly low-income children who qualified for mandatory Medicaid managed care and were enrolled in 1 of several health plans.

The predominant managed care organization (MCO) serving the clinic provided a list of names and dates of birth for the 654 children at baseline who were 3 years or younger as of March 1, 1998, and were assigned to receive care at the Harriet Lane Pediatric Clinic (HLPC), The Johns Hopkins Hospital, Baltimore, Md. The MCO also provided a postintervention list of 930 enrollees who were 3 years or younger as of June 30, 1999, and were assigned to receive care at HLPC. Baseline and postintervention lists were manually examined and compared to identify duplicate entries within and between the lists. Medical record numbers for each child were sought through a systematic search of the computerized patient record that documents all patient encounters.

Since 1995, all vaccinations delivered in the clinic were ordered electronically, with dates and types of vaccines entered into a computerized patient record. Vaccination histories for individual children are printed on the Maryland Vaccination History form, which records vaccination type and date of administration. As part of a city immunization registry initiative, local health department staff entered historical vaccination records into the computerized patient record for children younger than 6 years. This back data entry included vaccinations provided in HLPC before 1996 and those documented (with vaccine type and date) as being given by other providers.

The study was approved by the Committee on Human Research at The Johns Hopkins University School of Hygiene and Public Health.

The intervention took place between May 1998 and July 1999. Triage nurses generated computerized printouts of each child's vaccination record and prominently attached them to the encounter form during each acute care visit. As part of this intervention, the acute care team was instructed to review the vaccination history during the visit and offer age-appropriate vaccinations as appropriate. Study investigators (C.S.M. and J.R.S.) educated the acute care team monthly regarding this policy and the need to assess vaccination status at each visit. During these sessions, clinicians and staff received chocolate bars labeled "Immunize On Time, Every Time" to reinforce compliance. In addition, clinic vaccination rates were reviewed with staff and providers at several monthly staff meetings and at quarterly preceptor meetings.

Medical records were abstracted from the outpatient clinic at baseline and postintervention. Paper and electronic medical records were used to gather and verify immunization data. Four trained abstractors recorded demographic information, vaccinations, and visit data, including date, diagnoses, and temperature. Diagnoses and temperature were recorded to identify contraindications to vaccination. Up to 3 attempts were made to locate each missing record.

Medical record data were entered, verified, and checked for internal consistency. Several outcome measures were defined.

Receipt of timely vaccinations included 3 diphtheria and tetanus toxoids and pertussis, 2 polio, 3 hepatitis B, and 3 Haemophilus influenzae type b vaccines by the age of 10 months, and 4 diphtheria and tetanus toxoids and pertussis, 3 polio, 1 measles-mumps-rubella, 3 hepatitis B, and 3 Haemophilus influenzae type b vaccines by the age of 24 months. Definitions of UTD are based on the Red Book recommendations of the American Academy of Pediatrics.¹³

A missed opportunity occurred when the child was age eligible for a vaccine and had no contraindication for the vaccine, but the vaccine was not administered. Missed opportunities may occur at any type of visit, including sick visits, and were counted for those children not UTD for immunizations. Age eligibility and contraindications also were based on the Red Book recommendations.⁶

Well-child visits included visits for health supervision, and acute care visits included visits (first time and follow-up) for injury or illness. Visits for completion of forms or prescription refills were excluded. The percentage of visits for well-child care also was determined.

Children were continuously enrolled in the MCO and clinic if they were included in the baseline and postintervention MCO enrollment files and assigned to the clinic for primary care. Analysis was limited to those children with at least 1 visit to the clinic during the intervention period.

Data analyses were conducted using the Clinical Assessment Software Application and EPI-Info⁶ (both from the Centers for Disease Control and Prevention, Atlanta, Ga) and commercially available software (Microsoft Access 1997; Microsoft Corporation, Seattle, Wash). Proportions were compared using χ² analyses, and means were compared using analysis of variance. Comparisons were made between baseline and postintervention groups for 2 groups of children: those aged 10 to 23 months and those aged 24 months and older. The postintervention sample was identified as the total sample and the subset of children continuously enrolled.

Of the 654 baseline and 930 postintervention children identified by the MCO as receiving primary care at the outpatient hospital-based clinic, records were reviewed for 521 baseline and 642 postintervention children (Figure 1). Children excluded from analyses included those with duplicate listings in the MCO enrollment file, those without a documented visit to the institution, those not assigned to HLPC but assigned to an alternate source of care within the institution, and those for whom medical records were unable to be located.

The baseline and postintervention samples were similar with regard to gender, race, number of siblings, and birth weight (Table 1). However, the groups differed regarding age, with the baseline group having a greater proportion of older children than the total postintervention group. Only children aged 12 months or older are eligible for inclusion in the continuously enrolled group.

For children aged 24 months and older and continuously enrolled in the MCO and the clinic, there is an increase in 4 diphtheria and tetanus toxoids and pertussis, 3 polio, 1 measles-mumps-rubella, 3 hepatitis B, and 3 Haemophilus influenzae type b vaccination rates from 70% to 87% (Table 2). As expected, this group experienced a higher number of mean visits, with no difference in the proportion of visits for well-child care and a decrease in missed opportunities among those not UTD. Similar trends of increased vaccination rate and decreased missed opportunities from baseline to postintervention were found for all children in the postintervention group, although the change in vaccination rate was not significant. Among children aged 24 months and older, the proportion completing a visit within the first 2 months of life was higher in the postintervention than in the baseline group. However, there was no difference in UTD rates for the first diphtheria and tetanus toxoids and pertussis vaccine at 3 months between the baseline (79.3%) and postintervention (80.7%) groups (P>.05).

For children aged 10 to 23 months, only missed opportunities were significantly changed in the total postintervention group, with a decrease from 60% at baseline to 42% postintervention. However, trends of increased vaccination rates, without a change in mean visits or percentage well-child visits, were similar to those for the aged 24 months and older group (Table 3).

Our study demonstrates the effectiveness of a low-intensity intervention, provider prompting during acute care visits, in an urban hospital-based pediatric clinic among continuously enrolled children. Such an intervention may be adopted more readily than previously described efforts. Rodewald¹⁴ and Wood¹⁵ and colleagues described hiring of nonclinical personnel to enhance vaccination delivery; however, the use of outreach workers or case managers was associated with high costs. External audits of practice vaccination rates by local health officers also have been shown to be effective,¹⁶ but may not be generalizable in localities with minimal public health infrastructure. Several interventions have relied on reallocation of responsibilities of existing clinical staff to assess vaccination status and remind providers about needed vaccinations^14,17; foregone tasks and the impact on remaining personnel are not identified. Other investigators^18,19 have used financial incentives to improve vaccination delivery among adults; however, such practices require additional resources and have not been shown to be effective in at least 1 pediatric setting.²⁰ Our proposed intervention was implemented easily by existing staff with minimal extra time.

Our study reveals that limiting analyses to those children continuously enrolled in the managed care plan and assigned to the clinic for primary care leads to higher vaccination rates and a larger reported intervention effect. As such, we suspect that performance measures, such as those adopted by the Health Plan Employer Data Information Set, which examines vaccination rates among children without lapses in coverage or receipt of care, may overestimate population-based receipt of vaccinations by excluding those children at highest risk of undervaccination.

We found a substantial proportion of children who had not received any care at HLPC despite MCO administrative records documenting their assignment to the clinic for primary care. Some of these children, in fact, were noted to receive care through other hospital-based facilities, including specialty clinics and other sites for primary care. However, we are unable to account for primary care sites for many of these children. We suspect that many practices and managed care plans lack systems and capacity to verify whether children presumed to be assigned to a particular practice are, in fact, receiving care at that site. While outreach plans are required by many state Medicaid managed care contracts for identifying children not receiving primary care, effective mechanisms for accountability may not be in place.

Several limitations of the study should be noted. First, the use of a computerized immunization data system to generate printouts of vaccination records at nursing triage may limit the generalizability of this study. However, manual health maintenance checklists, including vaccinations, have been shown to increase adult vaccinations.^21-25 In addition, computer reminders have been shown to be more effective than manual reminders²⁶ and are likely to be more time efficient and sustained beyond an initial evaluation period. Use of a computerized system also minimizes the likelihood of intervention results being attributable to increased documentation rather than improved performance and actual delivery of vaccinations.

Second, the baseline vs postintervention design does not enable us to determine causal relationships. The proportion of well-child visits did not change during baseline and postintervention assessments, suggesting that increasing numbers of health maintenance visits during which vaccination status was usually assessed did not account for this change. In addition, National Immunization Survey vaccination rates in Baltimore City were not shown to increase during this period.²⁷

Third, the higher proportion of children in the postintervention group seen in the first 2 months of life also may contribute to increased vaccination rates among those aged 24 months and older. Other studies⁷ have documented the importance of timely receipt of 2-month vaccinations for achieving UTD status at the age of 2 years. However, we found no difference in the timeliness of receipt of 2-month vaccinations among children aged 24 months and older. Moreover, early visits to pediatricians would not account for the decrease in missed opportunities by the age of 24 months observed in this study. Our observation of decreased missed opportunities without a significant increase in vaccination rates among children aged 10 to 23 months may reflect small sample sizes and expectations for fewer total vaccines than for the group aged 24 months and older.

Although previously studied provider-based interventions have been shown to improve vaccination rates, such interventions frequently require excess capacity within practices or public health systems to audit performance. In the absence of new dollars or extensive computerized systems, minor changes in standard operating procedures may improve vaccination delivery, as we found in our study. It is likely that the high degree of motivation of clinicians and staff, reinforced through educational sessions and receipt of chocolate bars, enhanced the effectiveness of these minor changes. Further improvements to increase vaccination delivery to young children may require efforts to ensure continuity of provider and plan assignment.

Accepted for publication November 3, 2000.

This study was funded in part by the Pew Charitable Trusts, Philadelphia, Pa.

Presented at the annual meeting of the Pediatric Academic Societies, Boston, Mass, May 14, 2000.

We thank the managed care organization for providing the lists of study children; Lee Bone, RN, MPH, for her contributions to the development of the intervention; Bernard Guyer, MD, MPH, for his review of an earlier draft of the manuscript; and Jeanne Wehage for her assistance with data abstraction.

Corresponding author and reprints: Cynthia S. Minkovitz, MD, MPP, Department of Population and Family Health Sciences, The Johns Hopkins University School of Hygiene and Public Health, 624 N Broadway, Baltimore, MD 21205.