Strategies to Improve Immunization Rates and Well-Child Care in a Disadvantaged Population: A Cluster Randomized Controlled Trial

Objective  To measure the effect of a multimodal intervention on well-child care visit (WCV) and immunization rates in an inner-city population.

Design  Cluster randomized controlled trial.

Setting and Participants  One-year cohort of 2843 infants born at a hospital in an integrated inner-city health care system.

Interventions  Eleven clinics were randomly allocated to 1 of 3 study arms: WCV intervention (n = 3), immunization intervention (n = 4), and controls (n = 4). Interventions to improve immunization and WCV rates included both patient-based and clinic-based activities.

Main Outcome Measures  Up-to-date status with childhood immunizations and WCVs by age 12 months (primary) and health care utilization and charges (secondary).

Results  Compared with the control arm, the WCV and immunization arms had 5% to 6% higher immunization rates and 7% to 8% higher WCV rates. In multivariate analyses that accounted for the clustered nature of the data, the number of immunizations received was greater in the WCV arm than in controls. However, neither the WCV nor the immunization intervention increased WCV or immunization up-to-date rates. The WCV arm had slightly higher health care charges. Neither intervention affected emergency, urgent care or inpatient utilization.

Conclusions  This multimodal intervention produced a small increase in the number of childhood immunizations delivered. However, patient- and clinic-based methods did not lead to significant increases in WCV or immunization up-to-date rates after controlling for other factors. Methods found in some settings to increase immunization up-to-date rates may not be as effective in a population of inner-city socioeconomically disadvantaged children.

SINCE THE measles epidemic in the United States in the early 1990s, local, state, and federal research efforts and policy decisions have improved childhood immunization rates. Today, many areas of the country are approaching the immunization goals set forth in Healthy People 2010.^1-4 Despite these successes, pockets of underimmunization continue in many of the same inner-city areas that were the primary sites of the measles epidemic.^5-10 Given that immunizations are a marker for receipt of other childhood preventive services,^11-15 many inner-city children who are underimmunized are probably not receiving other health services, such as lead and anemia screening and preventive counseling.

Numerous studies have identified risk factors for childhood underimmunization^6,16-20 and have tested interventions to increase immunization rates at the level of the individual child (eg, reminder/recall)^21,22 and at the level of the clinic (eg, AFIX [Assessment, Feedback, Incentives, and eXchange of information]).^23,24 Reminder/recall systems have been effective in a variety of settings and have increased immunization rates 5% to 20%.²¹ The AFIX-based interventions have shown an increase in childhood immunization rates of 5% to 6% per year.²⁵ The Task Force on Community Preventive Services strongly recommends both methods to increase vaccination coverage.²⁶

Fewer studies have examined risk factors for lack of receipt of well-child care visits (WCVs)¹⁴ or the interrelationship of immunization status and well-child care receipt.^27,28 Studies that examined outcomes of increased well-child care have almost universally focused on the process of care (eg, how many children were screened for anemia)^11-13 rather than on the impact of that process on child health (eg, how many cases of anemia were prevented).

To examine outcomes associated with improved receipt of immunizations and WCVs, we asked 2 research questions: Can interventions at the level of the clinic and the patient improve delivery of immunizations and WCVs in an economically disadvantaged inner-city population? Do interventions to improve components of primary care for children, such as immunizations and WCVs, have an impact on other health care utilization (eg, hospitalizations and emergency department or urgent care visits) or on health care charges? To address these questions, we conducted a cluster randomized controlled trial, with randomization at the level of the clinic, to increase immunizations and WCVs in an inner-city population of disadvantaged, largely minority children.

Denver Health (DH) is the largest vertically integrated community health center system funded by the Bureau of Primary Health Care (Section 330) in the United States. This system comprises 11 community health centers, DH Medical Center (the city hospital, with 17 000 admissions per year), the DH emergency department (50 000 visits per year), 12 school-based clinics, and a public health department. Denver Health provides health care to more than 140 000 persons, or more than 20% of the population of the city and county of Denver, per year. Its community health centers, located in areas with the lowest health status and economic indices, have more than 500 000 patient visits per year. Patient medical records are shared among all DH facilities. All records associated with a patient are scanned into an electronic imaging system accessible across the citywide network; inpatient records are filed at the hospital, and outpatient encounters are filed in one unified patient chart. When transferring care from one network community health center to another, the patient's outpatient medical chart is also transferred.

All patients born at DH Medical Center between July 1, 1998, and June 30, 1999, were enrolled and followed for 12 months; a flow diagram of study participants and clinics is shown in Figure 1. Randomization was conducted at the clinic level rather than at the patient level because a substantial component of the intervention focused on changes in the clinic site as a whole, which would result in contamination within each site if the child were used as the unit of randomization. Randomization was achieved using a standard random-number table to assign each of the 11 clinics a number between 1 and 11 (1-4, 5-8, and 9-11 having been previously assigned to a study arm). The identity of individual clinics was concealed until study arms were assigned. Clinics were not matched on baseline characteristics such as racial and ethnic distribution or baseline immunization rates because matching in cluster randomized trials with small numbers of randomization units may reduce statistical power compared with simple randomization.²⁹

All infants born at DH Medical Center were given an appointment at one of the community health centers before hospital discharge; the choice of clinic was determined by the family's site of residence. The clinic at which the infant attended the 2-week WCV determined the intervention group assignment. For the few infants who did not attend a 2-week WCV, research group allocation was determined by the clinic chosen by the family in the newborn nursery logbook. Because research staff acted on behalf of the clinics to carry out enhanced clinic activities such as reminder/recall, we did not obtain informed consent from individual families. This study was approved by the Colorado Multiple Institutional Review Board and by the review board at the Centers for Disease Control and Prevention.

This study aimed to increase WCVs and immunizations using a multimodal intervention targeted at the individual patient and the clinic where infants received preventive health care. Some components of the intervention were specific to the WCV arm, and others were specific to the immunization arm. In addition, some systemwide interventions were occurring simultaneously that were not related to this study. A summary of all the interventions across the DH system during the period of the study is given in Table 1. Overall, the key elements of the intervention were intensive reminder/recall at the level of the patient and the process of AFIX at the level of the clinic. Briefly, AFIX is an intervention promoted by the Centers for Disease and Control and Prevention that was found to improve immunization rates at public health immunization sites.^23,30,31 In our study, AFIX was applied to WCVs at clinics assigned to that intervention arm or to immunizations at sites assigned to the immunization intervention. This study, to our knowledge, is the first application of AFIX methods, which were originally developed to boost immunization rates, to increase WCVs.

Research staff performed weekly assessments of infants in the birth cohort to maintain a "real-time" overview of up-to-date rates for WCVs and immunizations. Before the monthly clinic AFIX meetings, research staff compiled detailed lists of children with delayed preventive care. In the WCV intervention clinics, monthly recall lists were generated for children needing a WCV who did not have an appointment for a future visit. Similarly, in the immunization intervention clinics, children in need of immunizations were listed for recall efforts. Research staff supplemented basic clinic recall efforts (see Table 1) with telephone calls, postcards, letters, and referrals to Early Periodic Screening and Developmental Testing, a federally financed resource that provides individual case management, home visitation, and transportation to WCVs for children enrolled in Medicaid.

This study used data from 3 DH sources: (1) an electronic immunization registry, (2) medical chart review, and (3) administrative and billing records. In addition, a second Denver hospital, which provided pediatric care to a subset of the birth cohort in this study, provided utilization (but not charge) data on those children.

The DH electronic immunization registry serves as the systemwide repository for any pediatric immunization given in the system; it meets all Centers for Disease Control and Prevention criteria for electronic registries.³² This registry has been shown to be more accurate than the conventional medical record³³; by January 1, 2001, it contained 551 622 immunizations for 64 329 children.

Five trained abstractors reviewed all infant medical records to determine the reliability of electronic data sources, search for missing data, and categorize encounter visit types (inpatient, outpatient, and well-child care) for all DH encounters. In total, 97.3% (29 748/ 30 575) of the billed encounters received a medical chart review. Ongoing review of a sample of medical charts by other project staff assessed reliability and ensured consistent reviewer interpretation. Maternal antenatal and postnatal forms were also reviewed to abstract maternal covariates (eg, prenatal care use, maternal risk behaviors, and postnatal breastfeeding status) that might affect study outcomes.

All billing and diagnosis data for the study cohort were downloaded from the DH computer system. The DH registration and administrative data provided information on infant date of birth, race and ethnicity, percentage above the federal poverty level, family size, insurance status, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnoses,³⁴ and hospital service codes and charges. After matching on name, sex, and date of birth, administrative data for infants in the study cohort were obtained from a second hospital used by a subset of DH patients. Using hospital service codes and ICD-9-CM diagnosis codes, visits were then classified into 3 categories: inpatient, emergency department or urgent care, and primary care.

The primary outcome measures for this study included measures of WCV and immunization receipt. Well-child care visits were defined as visits in which a systemwide age-appropriate WCV form was used, corresponding to American Academy of Pediatrics WCV guidelines, which recommend 5 WCVs by age 12 months (at 2 weeks and 2, 4, 6, and 9 months). Use of the WCV form to define a WCV was found to be more accurate than use of the ICD-9-CM code for WCVs (V20.2) from administrative claims. The WCV outcomes were defined as (1) attendance at 4 or more WCVs and (2) number of WCVs attended (range, 0-5).

Immunization data from medical chart review and the electronic registry were merged into the final database. Only valid doses of immunizations in accordance with standard minimum ages and intervals set by the Centers for Disease Control and Prevention³⁵ were counted. The immunization outcomes were (1) percentage of children up-to-date with the 3:2:2:2 series (3 diphtheria-tetanus-pertussis, 2 poliovirus, 2 Haemophilus influenzae type B, and 2 hepatitis B vaccines) at age 12 months, (2) number of vaccines received by age 12 months, and (3) mean number of days underimmunized.

The secondary outcome measures were health care utilization and charges. Utilization outcomes were dichotomous variables for (1) emergency or urgent care service utilization and (2) inpatient utilization. Charges incurred by children in this birth cohort at a second hospital in Denver and its outpatient clinics were imputed using DH charges for similar services.

The following covariates were abstracted from maternal antepartum records: maternal education, language, parity, gravity, estimated date of delivery, birth country, total number of prenatal visits, date of first prenatal visit, psychiatric diagnoses, history of domestic violence, and use of tobacco, alcohol, or illicit drugs. The maternal postnatal visit provided information about breastfeeding. Administrative data, supplemented and validated by medical chart review, provided an income surrogate (percentage of federal poverty guidelines), family size, prenatal care visits, maternal risk factors, infant race and ethnicity, insurance status, date of birth, and birth weight. Pediatric chronic health conditions were defined using the rubric of the National Association of Children's Hospitals and Related Institutions,³⁶ not including asthma diagnosis codes.

All analyses were conducted using an intent-to-treat model, based on the clinic at which each infant planned to obtain primary care after discharge from the hospital at birth. Comparisons of the intervention and control groups were assessed using nonlinear mixed model methods adjusted for the clustered design of the study, with the 11 clinics entered as random effects, but unadjusted for other covariates.

We used nonlinear mixed models to assess the impact of the interventions on each of the outcome variables (7 models in total). In all cases, the clinic was considered a random effect, as it was the unit of randomization in the study design. The exact type of model was determined by the distribution of the outcome variable (dichotomous assumed a binomial distribution, continuous assumed a normal distribution, and discrete counts assumed a Poisson distribution). Covariates were entered into the full regression models if they differed between study intervention arms and were related to the outcome of interest (P<.20 for both). A log transformation was applied to health care charges.³⁷ For the only normally distributed continuous outcome variable (log charges for health care), the intraclass correlation coefficient was zero. We used statistical software to conduct all analyses (SAS version 8.01; SAS Institute Inc, Cary, NC).

Denver Health had 2843 live births between July 1, 1998, and June 30, 1999. Of these, 2665 (94%) were included in 1 of the 3 study groups (Figure 1). Thirty-eight infants were excluded because their parent(s) indicated that the infant would receive care outside the DH system, 97 because the infant received no care at DH after newborn discharge (this includes 7 neonatal deaths), and 43 because the infant attended a separately funded clinic that did not use DH's administrative billing system.

At baseline there were differences in infant and maternal characteristics, but none were statistically significant after controlling for clinic clustering (Table 2). The immunization study arm consisted of 4 clinics (mean number of study cohort children per clinic, 258; range, 65-697), the WCV arm had 3 clinics (mean number of study cohort children per clinic, 158; range, 45-273), and the control arm had 4 clinics (mean number of study cohort children per clinic, 290; range, 72-718). Each study arm had 1 clinic with resident continuity practices, except for the immunization arm, which had 2 such clinics. Baseline immunization up-to-date status was higher in the immunization (59%) and control (57%) clinics than in the WCV clinics (49%). These up-to-date estimates were calculated in 1997, based on a sample of patients from each clinic, and assessed 1-year immunization status at age 2 years. Thus, the percentages are not directly comparable to the numbers in this study but are indicative of the previous immunization delivery of the clinics.

Research staff completed more than 10 000 patient contacts for the 2 intervention arms. A total of 150 clinic-level AFIX meetings took place during the 2-year intervention period. Patient-level interventions included more than 500 Early Periodic Screening and Developmental Testing referrals, 2000 appointment confirmations by telephone, 1000 other telephone messages left, and 8000 postcards to remind people of scheduled appointments and to recall patients for missed appointments. Nearly 2000 unsuccessful attempts were made to contact patients, including calls to disconnected telephone numbers, returned postcards, and unanswered telephone calls. The average number of successful contacts per child in the 2 intervention arms was 7 (10 542 contacts per 1505 children); 2757 of these were telephone contacts (average, 2 per child). Ninety-nine percent of the children in the 2 intervention arms (1485 of 1505) received at least 1 contact by either telephone or postcard.

Compared with control infants, 5% to 6% more infants at the intervention sites were up-to-date with immunizations and 7% to 8% more attended at least 4 of 5 recommended WCVs by age 1 year (Table 3). Children in the 2 intervention arms were underimmunized a median of 57 and 62 days, compared with 67 days in the control arm. After adjusting for clinic clustering, however, none of these differences were statistically significant.

In the immunization, WCV, and control arms of the study, children averaged 6.0, 6.8, and 5.3 primary care visits, and 2.1, 2.0, and 2.2 emergency department and urgent care visits, respectively (Table 4). The median total health care charges for the children in the 3 study arms were $1136; $1236; and $1079, respectively. None of the health care utilization and charges outcomes were statistically significant, although the intervention arms did show a trend toward more primary care visits (P = .10).

After adjusting for clinics as random effects and significant covariates, the immunization intervention did not demonstrate a statistically significant increase in immunization or WCV receipt or with health care utilization or charges (Table 5). This intervention was associated with a trend (P<.10) toward higher immunization receipt, a greater number of immunizations and WCVs, and higher total charges for health care. The WCV intervention was associated with an increased number of immunizations (risk ratio, 1.06; 95% confidence interval [CI], 1.02-1.10) and greater total charges for health care (risk ratio for the log of total charges, 1.02; 95% CI, 1.01-1.04), as well as trends toward a greater number of WCVs (risk ratio, 1.07; 95% CI, 0.98-1.16) and increased immunization up-to-date rates (rate ratio, 1.14; 95% CI, 0.99-1.29). There was no difference in the use of urgent or emergency health care or inpatient health services.

We assessed the impact of patient-based (intensive reminder/recall) and clinic-based (AFIX) interventions on immunization and well-child care receipt in a clinic-based cluster randomized controlled trial with a prospectively enrolled birth cohort of disadvantaged inner-city children. The interventions resulted in a modest improvement in the number of immunizations delivered and a small but not statistically significant improvement in WCV and immunization up-to-date rates. The interventions had no effect on health care utilization and charges, except for a modest increase in the log of total health care charges in the WCV arm.

In other studies,^22,26 assessment and feedback at the level of the clinic and reminder/recall systems at the level of the patient have improved childhood immunization rates. In this trial, we used both strategies, which were associated with 5% to 6% higher receipt of adequate immunizations and 7% to 8% higher receipt of WCVs in the intervention arms. This finding is consistent with previous effects for AFIX-based interventions²⁵ and is at the lower end of published effects for reminder/recall-based interventions.²¹ What were the reasons for the modest effect seen in this trial? We tracked each child in the intervention arms of the study until age 12 months. Although 1 year may not be adequate time to see the full impact of clinic-based AFIX interventions,²⁵ it should be sufficient to observe a positive effect of reminder/recall interventions.²¹ In addition, we exposed every clinic and 99% of the families in the intervention groups to at least 1 intervention (see the "Results" section).

First, this was a real-world trial. Our interventions took place against a backdrop of rising immunization rates in the DH system. The percentage of 2-year-olds up-to-date with all infant vaccinations rose from 38% in 1995 to 67% in the summer of 1999, the midpoint of our study. In addition, there were parallel nonstudy interventions occurring concomitantly at sites randomized to receive no intervention through our study. For example, an independent, nonstudy, clinic-based AFIX immunization intervention was active in the largest of the control clinics during the study, impacting 60% (601/1003) of the infants in the control arm.

Second, there may be a ceiling effect on immunization rates in this study. The challenge of boosting up-to-date immunization status from 70% to 90% is different from boosting it from 50% to 70%, as the population that is most amenable to intervention has already been reached. We used a strict intent-to-treat analysis and did not exclude children from the denominator in the intervention arms who were identified as having moved or gone elsewhere for care by study personnel, as this would result in selective removal of children and bias favoring the intervention. However, in the immunization arm, study personnel found that children who could be defined as having moved or gone elsewhere for care by standard definitions^38,39 accounted for approximately 20% of the cohort (data not shown). In addition, clinics in the immunization intervention arm had the highest baseline immunization rates and thus may have had less margin for improvement.

In addition to children who have moved or gone elsewhere for care, in a low socioeconomic inner-city population a proportion of children are likely to be in highly mobile or homeless families, to have no telephone service, or to have parents with substantial physical or mental health burdens of their own. Such children will be difficult to reach even with the intensive interventions described herein. Disadvantaged inner-city populations may be particularly difficult to reach with such interventions,⁴⁰ despite intensive case management strategies.^41,42

Finally, by evaluating immunization coverage for the 6-month vaccines at age 12 months, we may have minimized chances to detect significant differences between intervention and control arms because of the window of catch-up time available to vaccinate underimmunized children. These differences may well have been more pronounced after age 1 year, once the 1-year vaccinations are due and the "toddler gap" in immunization coverage becomes apparent.²

This study has several limitations. First, statistical significance is difficult to achieve in a cluster randomized controlled trial with a relatively small number of clinics. This study design was believed to be essential to carry out interventions at the level of the clinic and the individual family to avoid contamination within each clinic. Second, because the WCV arm of the intervention had lower baseline immunization rates than the control and immunization arms, we might have missed a positive impact of the WCV intervention on immunization rates. Third, because of the size of the association of the intervention with WCV (7%-8% increase) and immunization receipt (5%-6% increase), we could have missed an impact of improved childhood preventive care on health care utilization and charges. For example, a recent study⁴³ in the DH system demonstrates that children enrolled in the Child Health Insurance Plan are more likely to have a WCV and less likely to have a visit to the emergency department compared with uninsured children. However, we detected an increase in the number of immunizations delivered and in the log of total charges for health care. Finally, we analyzed a cohort of children born at DH based on data from the DH system and one other hospital in Denver. Approximately 20% of our cohort was lost to follow-up. We did not capture health care utilization data for children who had moved away from central Denver. However, more than 70% of this birth cohort still actively used the DH system at age 1 year, and 10% of the cohort had additional health care utilization captured at a second Denver hospital. For immunization rates, true population-based data will require citywide or statewide registries.

In conclusion, we found that a combination of patient- and< clinic-based interventions targeted at increasing preventive health visits could increase the number of immunizations delivered to a cohort of children. However, this study also suggests that methods found in some settings to increase immunization up-to-date rates may not be as effective in a population of inner-city socioeconomically disadvantaged children. The challenge remains to find ways to provide full pediatric preventive health services to children in the greatest pockets of need.

Article

Despite rising childhood immunization rates in the United States, underimmunization remains a problem in some inner-city populations. Patient- and clinic-based interventions have been shown to increase childhood immunization rates in a variety of settings. This study showed that patient- and clinic-based interventions targeted at increasing well-child care can be used to increase immunization delivery in an inner-city population. However, patient- and clinic-based interventions used to increase childhood immunizations in other settings may not be as effective in a population of inner-city socioeconomically disadvantaged children.

Corresponding author: Simon J. Hambidge, MD, PhD, Denver Health Medical Center, 777 Bannock St, Mail Code 0132, Denver, CO 80204 (e-mail: simon.hambidge@uchsc.edu).

Accepted for publication July 18, 2003.

This study was funded by cooperative agreement TS 252-13/15 from the Association of Teachers of Preventive Medicine and the Centers for Disease Control and Prevention.

We are indebted to Luis Espinosa Organista, Kristen Miller, BA, and Susan Macaskill-Deloach, BA, for their hard work and creativity in implementing the interventions and in data collection.