Effectiveness of a Citywide Patient Immunization Navigator Program on Improving Adolescent Immunizations and Preventive Care Visit Rates

Objective  To assess the impact of a tiered patient immunization navigator intervention (immunization tracking, reminder/recall, and outreach) on improving immunization and preventive care visit rates in urban adolescents.

Design  Randomized clinical trial allocating adolescents (aged 11-15 years) to intervention vs standard of care control.

Setting  Eight primary care practices.

Participants  Population-based sample of adolescents (N = 7546).

Intervention  Immunization navigators at each practice implemented a tiered protocol: immunization tracking, telephone or mail reminder/recall, and home visits if participants remained unimmunized or behind on preventive care visits.

Main Outcome Measures  Immunization rates at study end. Secondary outcomes were preventive care visit rates during the previous 12 months and costs.

Results  The intervention and control groups were similar at baseline for demographics (mean age, 13.5 years; 63% black, 14% white, and 23% Hispanic adolescents; and 74% receiving Medicaid), immunization rates, and preventive care visit rates. Immunization rates at the end of the study were 44.7% for the intervention group and 32.4% for the control group (adjusted risk ratio, 1.4; 95% confidence interval, 1.3-1.5); preventive care visit rates were 68.0% for the intervention group and 55.2% for the control group (1.2; 1.2-1.3). Findings were similar across practices, sexes, ages, and insurance providers. The number needed to treat for immunizations and preventive care visits was 9. The intervention cost was $3.81 per adolescent per month; the cost per additional adolescent fully vaccinated was $465, and the cost per additional adolescent receiving a preventive care visit was $417.

Conclusion  A tiered tracking, reminder/recall, and outreach intervention improved immunization and preventive care visit rates in urban adolescents.

Trial Registration  clinicaltrials.gov Identifier: NCT00581347

The era of adolescent immunizations has dawned, with pertussis, meningococcus, and human papillomavirus vaccines recommended for routine administration and influenza vaccine recommended annually.^1-3 However, immunization rates in adolescents remain low⁴ and disparities exist, with low-income and minority adolescents having poorer rates.⁴ One challenge is that adolescents have few preventive health care visits,⁵ which is when most vaccines are administered.^6,7 Timely receipt of adolescent vaccines requires most adolescents to make additional visits to their primary care practices.^8,9

Primary care practices have the burden of tracking and identifying those in need of vaccinations and implementing strategies that encourage adolescents to come in for needed vaccine administration visits.⁷ One strategy involves tracking and reminder/recall, which has been recommended by the Task Force on Community Preventive Services¹⁰ and other experts,^11,12 for populations of any age.

Several recent reminder/recall studies^13-17 using simple protocols for sending letters or telephone messages for low-income populations have found little or no benefit in improving infant and toddler vaccinations because of difficulty in reaching families that often move or change telephone numbers. One trial¹⁸ of telephone-based reminder/recall in urban adolescents, conducted before the new vaccines, found minimal improvement in hepatitis B vaccination rates (mostly in families with stable telephone numbers).

Because of the limited success of traditional letter and telephone reminder/recall systems for urban populations, several studies of infant/toddler vaccinations have evaluated the benefit of tiered interventions. This model involves tracking immunization rates for all individuals, adding telephone or letter reminder/recall for those who are behind, and then adding more intensive outreach through home visits for individuals who remain unvaccinated despite reminder/recall.^19-21 Two studies^19,20 of tiered interventions for infant and toddler immunizations noted considerable improvements in immunization and preventive care visit rates. A similar intervention targeting older adults found significant improvements in influenza and pneumococcal vaccination rates.²² However, tiered tracking, reminder/recall, and outreach has not been evaluated for adolescents. Because many adolescents underuse primary care services and are difficult to reach, interventions that work for infants or the elderly may not work for adolescents.

We conducted a randomized controlled clinical trial in 8 urban practices in Rochester, New York, to evaluate the impact of tiered tracking, reminder/recall, and outreach on adolescent immunization and preventive health care visit rates. We also assessed the costs of the intervention. We hypothesized that the intervention increases rates of immunizations and preventive care visits in urban adolescents.

This study was performed in Rochester, a city of 250 000 residents in which the adolescent population (>10 years old) is 54% black, 22% Hispanic, and 21% white.²³ Almost 80% of adolescents in the city live below the poverty level; 52% are covered by Medicaid, and 7% are uninsured.²³

We conducted a randomized controlled trial of a tiered intervention to assist families in obtaining vaccinations and preventive care visits for their 11- to 15-year-old adolescents. We randomized adolescents in each practice, each age in years (11-15 years), and each sex to a tiered intervention using patient immunization navigators or to a standard of care control group. We conducted the intervention between October 1, 2007, and December 31, 2008, and we performed medical record reviews during the subsequent 3 months to assess outcomes. The Research Subjects Review Board of the University of Rochester approved this study. Parental informed consent was not required.

We identified the 8 largest urban practices that serve adolescents in Rochester; these sites care for 47% of city adolescents. They included 2 federally qualified community health centers, 2 pediatric hospital-based clinics, 1 family medicine teaching clinic, 1 hospital-associated medicine-pediatrics practice, and 2 urban private practices. For 7 practices, more than two-thirds of adolescents are covered by Medicaid; for the remaining practice, half of adolescents are covered by Medicaid. All the practices agreed to participate.

The target population was adolescents aged 11 to 15 years (birth dates: July 1, 1992, to June 30, 1997) who were enrolled in one of the practices. Because no practice had accurate denominators of their active patients, we used 2 methods to identify eligible adolescents. First, the 2 major insurance plans in the region (that together serve 94% of Medicaid-managed care, 100% of State Children's Health Insurance Program, and 98% of commercially insured populations)²⁴ each provided a list of all their age-appropriate patients at these 8 practices. Second, because the 2 largest practices (40% of all participants) had accessible billing systems, we added all age-appropriate adolescents who made a visit to the practice within 2 years of the start of the study and had either fee-for-service Medicaid or no insurance.

We identified all families with age-eligible adolescents, randomly selected a reference adolescent, and randomly assigned each family to the intervention or control group (using a commercially available software program [SAS, version 9.1; SAS Institute Inc, Chicago, Illinois], stratifying on practice, age, and sex). Health care providers were unaware of group assignment.

The intervention consisted of a tiered protocol. Each step was more intensive and targeted a progressively smaller proportion of adolescents who remained behind in immunizations despite the previous steps. This method, modeled after a childhood program,¹⁹ minimized the intervention needed for each adolescent.

The intervention was delivered by trained patient immunization navigators (4.5 full-time equivalents), analogous to chronic disease patient navigators²⁵ or promotoras.²⁰ The navigators were recruited from the community; 1 fluent Spanish-speaking navigator was placed in the practice with the largest Hispanic population. They received formal training on the intervention, use of a database, health promotion, and methods to assist families to navigate the health and social service systems. The navigators were provided a workspace and a computer at each practice, and all were supervised by a social worker (M.S.). Their percentage effort in each practice was determined by their caseload, which varied from 600 to nearly 1000 per full-time equivalent.

Because the study occurred before practices had incorporated the statewide immunization registry for adolescent vaccinations, we created a Web-based database for navigators to track the adolescents, record immunizations and preventive care visits, and document tasks performed.

Navigators performed reminder/recall for adolescents who were eligible for either a vaccination or a preventive care visit (with a 1-month grace period). They attempted to contact families by telephone (≥2 attempts at different times of day on different days) and mail (2 letters sent 2 weeks apart). The protocol involved 2 telephone calls (>1 week apart) and then 2 letters, and it started with letters if no telephone number was available. The navigators used a patient-centered and partnership-building approach to increase family awareness of preventive health measures and to address barriers to care. They offered transportation assistance (bus tokens and transport by car). If after the reminder/recall parents did not make and keep appointments, vaccination status was not brought up-to-date at the visit, or a subsequent human papillomavirus vaccination was needed, the navigators reinitiated the cycle. After 2 telephone calls and 2 letters, they moved to step 3.

If adolescents remained unvaccinated despite the previous steps, the navigators performed a home visit to further assess barriers, promote the importance of preventive care, and encourage families to make appointments. The number of home visits was kept low to minimize personnel costs and maximize feasibility and sustainability. Control subjects received standard of care. All the practices routinely sent letter or telephone reminders to families who had upcoming scheduled visits, but none used active reminder/recall based on vaccinations.

Patient information from insurer or practice lists included name, address, telephone number, birth date, insurance type at the start of the intervention (Medicaid managed care, fee-for-service Medicaid, State Children's Health Insurance Program, commercial, or uninsured), and primary care practice. Patient information collected from the practices' medical records for the intervention included name, address, race/ethnicity (white, black, or Hispanic), language, telephone number, contact information for a parent/guardian, vaccination history, and preventive care visits. Summaries of navigator activities (telephone calls, letters, and home visits) were entered into the study database.

After the study intervention period, we reviewed medical records (paper or electronic) using a standardized medical record abstraction form for all adolescent immunization dates and preventive care visit dates. We also searched the New York State immunization registry for any additional vaccination dates. We used these data to determine baseline and follow-up immunization dates. Quality assurance checks performed on 5% of medical record reviews demonstrated high interrater reliability (κ ≥ 0.89).

The primary outcome was receipt of each recommended routine vaccination after age 11 years (meningococcus, pertussis, and 3 human papillomavirus vaccines [girls] and all vaccines combined). We did not include influenza vaccine because this study started before the introduction of universal influenza vaccination recommendations.²⁶ We excluded varicella vaccine because varicella disease status or vaccine eligibility is difficult to assess by medical record review.²⁷

Because it is recommended that all adolescents receive an annual preventive care visit, we assessed the receipt of a preventive care visit during the 12-month period before the end of the study (after age 11 years). We measured the costs of the intervention by summing total personnel costs (salaries of navigators [80%] and supervisory personnel [16%]) and nonpersonnel costs (including office supplies, cell phone costs, travel expenses for home visits and transports, and costs of the database [4%]).

To determine the relative risk (RR) of the intervention at the end of the study, we used multiple Poisson regression, with vaccination status and receipt of preventive care visits as the outcome and group status as the main explanatory variable and controlling for stratification variables (practice, age, and sex). To consider the lack of independence among siblings, we used the clustered Huber/White variance estimator for all analyses (STATA/SE, version 11.0; StataCorp LP, College Station, Texas).²⁸

Because the intervention could affect only those not up-to-date at the beginning of the study, we excluded adolescents who were up-to-date at the study onset for each vaccine; for example, if an adolescent was up-to-date for meningococcus but not pertussis before the start of the study, he or she was excluded from the meningococcus analysis but was included in the pertussis and composite outcome analyses. For the preventive care analysis, we included an independent variable indicating whether the child had a preventive care visit within 12 months before the start of the study. We performed a prespecified subgroup analysis on age, sex, race, insurance, and practice subgroups for the composite immunization and preventive care outcomes. To control for the multiple testing, we adjusted confidence intervals for the 22 tests for the composite immunization outcome and 24 tests for the preventive care outcome using the Šidák method.²⁹

We also assessed process and cost measures. We measured the number of reminder/recall messages and home visits. For the cost analyses, we included all the participants. We calculated the cost for each additional outcome (becoming vaccinated or receiving a preventive care visit) as the total cost of the intervention divided by ([No. of subjects] × [difference in the % of the outcome between the study and control groups]).³⁰

Altogether, 7546 adolescents from 6682 families were randomized (Figure); 5910 families (88.4%) had 1 adolescent, 690 (10.3%) had 2, 74 (1.1%) had 3, 7 (0.1%) had 4, and 1 (0.01%) had 5. No controls received the intervention, and all the participants were included in an intent-to-treat analysis.

The control and intervention groups had similar demographic characteristics and baseline immunization and preventive care visit rates (Table 1). The mean age at the start of the study was 13.5 years; half the participants were male; 63% were black, 14% white, and 23% Hispanic; and 74% had Medicaid and 6% were uninsured.

Immunization rates at the end of the study for individual vaccines and for all 3 vaccines combined (including human papillomavirus for girls) were 12 to 16 percentage points higher for the intervention group than for the control group, with adjusted RR ratios ranging from 1.2 to 1.5 (P < .001 for all) (Table 2). Preventive care visit rates for each age group were 9 to 17 percentage points higher for the intervention group, with adjusted RR ratios ranging from 1.1 to 1.3 for each age group (P < .01 for all) (Table 3). The intervention had a substantial effect on most subgroups analyzed, including age, sex, race/ethnicity, insurance type, and practice (Table 4). Tests for interactions revealed that the intervention had greater effects on immunizations for girls than for boys (26% higher RR, P = .004) and for black or Hispanic adolescents than for white adolescents (40% and 27% higher RR, P = .009) and a greater effect on preventive care visits for those without a previous preventive care visit (13% higher RR, P = .02). Finally, we reexamined the results by clustering on practice and found no differences in any outcomes.

Altogether, 71% of the intervention group received either telephone or mail reminders, and 12% had a home visit. There were no adverse events from outreach; parents of 15 of 3839 participants declined navigator assistance but were included in the analyses. The total personnel plus nonpersonnel cost (subtracting research-related costs) was $45.74 per adolescent per year ($3.81 per month). The number needed to treat for an additional adolescent vaccinated was 9; the number needed to treat for an additional preventive care visit was also 9. The intervention cost per additional adolescent fully vaccinated was $465, and the cost per additional adolescent receiving a preventive care visit was $417.

In this clinical trial, which included more than 40% of adolescents in Rochester, we demonstrated that a tiered intervention of tracking, reminder/recall, and outreach provided by navigators for adolescent immunizations improved immunization rates by 12% to 16% depending on the vaccine and improved receipt of an annual preventive care visit by 9% to 17% depending on age. The intervention had a similar effect across all practices and irrespective of patient characteristics. These outcomes were accomplished primarily through telephone reminder/recall, with home visits for 12% of adolescents. The immunization navigator intervention cost was $3.81 per adolescent per month.

To our knowledge, this is the first large-scale study since introduction of the recommendations for new adolescent vaccines to demonstrate the benefit of a practice-based immunization intervention for urban adolescents. In fact, we found few published studies that have improved immunization or preventive care visit rates in urban adolescents. One randomized clinical trial¹⁸ investigated the effect of an autodialer-based telephone reminder/recall intervention (without patient navigators) for urban adolescents aged 11 to 14 years. Although that intervention was quite intensive, with up to 5 reminder calls per month, it had only a minimal effect on hepatitis B and tetanus immunization rates and no effect on preventive care visit rates.¹⁸ The key barrier was inaccurate telephone numbers for a relatively transient population, which experts postulate explains the limited success of telephone or mail reminder/recall interventions for low-income populations.^{11,14,16,17,31,32}

The success of the present intervention raises 2 important lessons regarding reminder/recall for low-income urban populations. First, for populations that have not tended to benefit from traditional telephone or mail reminder/recall messages,^13-17 a tiered intervention that includes traditional reminder/recall plus more intense outreach may be beneficial. To maintain efficiency, more intense intervention should be reserved for adolescents who do not respond to lower-level interventions. The present results mirror the impact noted by similar tiered interventions targeting infants^19-21 and older adults.²²

The second lesson is that the intervention has a cost at $3.81 per adolescent per month. This is similar to the cost of reminder/recall and outreach for infants or toddlers in the Rochester area,¹⁹ less than one-sixth the cost of a similar program targeting infants in Denver,²⁰ and only a fraction of the cost of a home visitation and case management program to improve childhood immunization rates in Los Angeles.³³ Furthermore, the present intervention produced benefits beyond immunizations by improving preventive care visit rates. This occurred because immunizations are typically bundled with preventive care visits.^6,7 Although we did not measure receipt of preventive care services such as screening tests, anticipatory guidance, or tune-up of chronic conditions, these clinical preventive care services also tend to be coupled with preventive care visits.⁶ Furthermore, we could not find any other peer-reviewed publications of interventions that have successfully increased preventive care visits in urban adolescents, and this population has low rates of preventive care visits.^34,35 Despite these benefits, the costs of the present intervention represented approximately 2.5% of total health care costs for adolescents in the Rochester region (Howard Brill, PhD, Monroe Plan for Medical Care, written communication, June 23, 2010).

Study strengths include the use of a clinically relevant intervention, a large and diverse sample, inclusion of a broad spectrum of primary care practices, ascertainment of important outcomes (immunization and preventive care visit rates), and an intention-to-treat design.

A limitation is the inability to distinguish the relative impact of different parts of the tiered intervention, such as reminders vs recall vs outreach. This is an inherent limitation of the multipart interventions that are now recommended for immunization delivery³⁶ and other practice-based quality improvement interventions.³⁷ Second, baseline immunization rates were low; benefits of the intervention may wane as rates rise. However, baseline preventive care visit rates mirrored national rates.^5,7 To estimate the potential role of baseline rates, we performed a post hoc analysis comparing the odds of being up-to-date on overall immunization rates at the end of the study in children who had low (<5%), medium (5%-39%), and high (≥40%) baseline immunization rates and found no significant differences (odds ratios, 1.9, 1.7, and 1.8, respectively; P = .60). Third, this study had substantial research support, and we consider it an efficacy trial.³⁸ Further implementation and dissemination studies are needed to assess continued effectiveness. Fourth, although studies have evaluated the cost-effectiveness of adolescent vaccinations,^39-41 none have evaluated costs of practice-based interventions to either raise immunization rates or improve adolescent preventive care visits. Also, we could not find cost-effectiveness studies of other adolescent preventive services to assess the relative value of the present intervention. Furthermore, because this intervention was applied across nearly half the city, costs per adolescent were likely lower than if the intervention were implemented in a single practice. Finally, the intervention may not be generalizable to other urban settings, suburban settings, or practices with high baseline rates.

In conclusion, a tiered patient navigator intervention consisting of immunization tracking, patient reminder/recall, and outreach based in inner-city primary care practices substantially increased adolescent immunization rates, with a spillover benefit of increased preventive care visit rates. Physicians and leaders in public health, managed care, and integrated health systems should consider this tiered navigator system for urban adolescents. Further research should assess the effectiveness and cost-effectiveness of the program in other settings.

Correspondence: Peter G. Szilagyi, MD, MPH, Department of Pediatrics, University of Rochester School of Medicine and Dentistry, PO Box 777, 601 Elmwood Ave, Rochester, NY 14642 (peter_szilagyi@urmc.rochester.edu).

Accepted for Publication: December 30, 2010.

Author Contributions: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Szilagyi, Humiston, Albertin, and Sandler. Acquisition of data: Szilagyi, Humiston, Gallivan, Albertin, and Blumkin. Analysis and interpretation of data: Szilagyi, Humiston, and Blumkin. Drafting of the manuscript: Szilagyi, Gallivan, and Blumkin. Critical revision of the manuscript for important intellectual content: Szilagyi, Humiston, Albertin, Sandler, and Blumkin. Statistical analysis: Szilagyi and Blumkin. Obtained funding: Szilagyi, Humiston, Albertin, and Sandler. Administrative, technical, and material support: Szilagyi, Humiston, Gallivan, Albertin, and Sandler. Study supervision: Szilagyi and Sandler.

Financial Disclosure: None reported.

Funding/Support: The study was funded by the Greater Rochester Health Foundation.

Role of the Sponsor: The Greater Rochester Health Foundation had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.

Previous Presentation: This study was presented in part at the Pediatric Academic Societies meetings; May 2, 2010; Vancouver, British Columbia, Canada.

Additional Contributions: We appreciate the collaboration of the physicians and staff at the 8 primary care practices in Rochester.