Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
June 06, 2011

Effectiveness of a Citywide Patient Immunization Navigator Program on Improving Adolescent Immunizations and Preventive Care Visit Rates

Author Affiliations

Author Affiliations: Departments of Pediatrics (Dr Szilagyi; Mss Gallivan, Albertin, and Sandler; and Mr Blumkin) and Social Work (Mss Gallivan and Sandler), University of Rochester School of Medicine and Dentistry, Rochester, New York; and Emergency Medical Services, Department of Pediatrics, Children's Mercy Hospitals and Clinics, Kansas City, Missouri (Dr Humiston).


Copyright 2011 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2011

Arch Pediatr Adolesc Med. 2011;165(6):547-553. doi:10.1001/archpediatrics.2011.73

Objective  To assess the impact of a tiered patient immunization navigator intervention (immunization tracking, reminder/recall, and outreach) on improving immunization and preventive care visit rates in urban adolescents.

Design  Randomized clinical trial allocating adolescents (aged 11-15 years) to intervention vs standard of care control.

Setting  Eight primary care practices.

Participants  Population-based sample of adolescents (N = 7546).

Intervention  Immunization navigators at each practice implemented a tiered protocol: immunization tracking, telephone or mail reminder/recall, and home visits if participants remained unimmunized or behind on preventive care visits.

Main Outcome Measures  Immunization rates at study end. Secondary outcomes were preventive care visit rates during the previous 12 months and costs.

Results  The intervention and control groups were similar at baseline for demographics (mean age, 13.5 years; 63% black, 14% white, and 23% Hispanic adolescents; and 74% receiving Medicaid), immunization rates, and preventive care visit rates. Immunization rates at the end of the study were 44.7% for the intervention group and 32.4% for the control group (adjusted risk ratio, 1.4; 95% confidence interval, 1.3-1.5); preventive care visit rates were 68.0% for the intervention group and 55.2% for the control group (1.2; 1.2-1.3). Findings were similar across practices, sexes, ages, and insurance providers. The number needed to treat for immunizations and preventive care visits was 9. The intervention cost was $3.81 per adolescent per month; the cost per additional adolescent fully vaccinated was $465, and the cost per additional adolescent receiving a preventive care visit was $417.

Conclusion  A tiered tracking, reminder/recall, and outreach intervention improved immunization and preventive care visit rates in urban adolescents.

Trial Registration Identifier: NCT00581347

The era of adolescent immunizations has dawned, with pertussis, meningococcus, and human papillomavirus vaccines recommended for routine administration and influenza vaccine recommended annually.13 However, immunization rates in adolescents remain low4 and disparities exist, with low-income and minority adolescents having poorer rates.4 One challenge is that adolescents have few preventive health care visits,5 which is when most vaccines are administered.6,7 Timely receipt of adolescent vaccines requires most adolescents to make additional visits to their primary care practices.8,9

Primary care practices have the burden of tracking and identifying those in need of vaccinations and implementing strategies that encourage adolescents to come in for needed vaccine administration visits.7 One strategy involves tracking and reminder/recall, which has been recommended by the Task Force on Community Preventive Services10 and other experts,11,12 for populations of any age.

Several recent reminder/recall studies1317 using simple protocols for sending letters or telephone messages for low-income populations have found little or no benefit in improving infant and toddler vaccinations because of difficulty in reaching families that often move or change telephone numbers. One trial18 of telephone-based reminder/recall in urban adolescents, conducted before the new vaccines, found minimal improvement in hepatitis B vaccination rates (mostly in families with stable telephone numbers).

Because of the limited success of traditional letter and telephone reminder/recall systems for urban populations, several studies of infant/toddler vaccinations have evaluated the benefit of tiered interventions. This model involves tracking immunization rates for all individuals, adding telephone or letter reminder/recall for those who are behind, and then adding more intensive outreach through home visits for individuals who remain unvaccinated despite reminder/recall.1921 Two studies19,20 of tiered interventions for infant and toddler immunizations noted considerable improvements in immunization and preventive care visit rates. A similar intervention targeting older adults found significant improvements in influenza and pneumococcal vaccination rates.22 However, tiered tracking, reminder/recall, and outreach has not been evaluated for adolescents. Because many adolescents underuse primary care services and are difficult to reach, interventions that work for infants or the elderly may not work for adolescents.

We conducted a randomized controlled clinical trial in 8 urban practices in Rochester, New York, to evaluate the impact of tiered tracking, reminder/recall, and outreach on adolescent immunization and preventive health care visit rates. We also assessed the costs of the intervention. We hypothesized that the intervention increases rates of immunizations and preventive care visits in urban adolescents.


This study was performed in Rochester, a city of 250 000 residents in which the adolescent population (>10 years old) is 54% black, 22% Hispanic, and 21% white.23 Almost 80% of adolescents in the city live below the poverty level; 52% are covered by Medicaid, and 7% are uninsured.23

We conducted a randomized controlled trial of a tiered intervention to assist families in obtaining vaccinations and preventive care visits for their 11- to 15-year-old adolescents. We randomized adolescents in each practice, each age in years (11-15 years), and each sex to a tiered intervention using patient immunization navigators or to a standard of care control group. We conducted the intervention between October 1, 2007, and December 31, 2008, and we performed medical record reviews during the subsequent 3 months to assess outcomes. The Research Subjects Review Board of the University of Rochester approved this study. Parental informed consent was not required.

Primary Care Practices

We identified the 8 largest urban practices that serve adolescents in Rochester; these sites care for 47% of city adolescents. They included 2 federally qualified community health centers, 2 pediatric hospital-based clinics, 1 family medicine teaching clinic, 1 hospital-associated medicine-pediatrics practice, and 2 urban private practices. For 7 practices, more than two-thirds of adolescents are covered by Medicaid; for the remaining practice, half of adolescents are covered by Medicaid. All the practices agreed to participate.


The target population was adolescents aged 11 to 15 years (birth dates: July 1, 1992, to June 30, 1997) who were enrolled in one of the practices. Because no practice had accurate denominators of their active patients, we used 2 methods to identify eligible adolescents. First, the 2 major insurance plans in the region (that together serve 94% of Medicaid-managed care, 100% of State Children's Health Insurance Program, and 98% of commercially insured populations)24 each provided a list of all their age-appropriate patients at these 8 practices. Second, because the 2 largest practices (40% of all participants) had accessible billing systems, we added all age-appropriate adolescents who made a visit to the practice within 2 years of the start of the study and had either fee-for-service Medicaid or no insurance.


We identified all families with age-eligible adolescents, randomly selected a reference adolescent, and randomly assigned each family to the intervention or control group (using a commercially available software program [SAS, version 9.1; SAS Institute Inc, Chicago, Illinois], stratifying on practice, age, and sex). Health care providers were unaware of group assignment.

The intervention consisted of a tiered protocol. Each step was more intensive and targeted a progressively smaller proportion of adolescents who remained behind in immunizations despite the previous steps. This method, modeled after a childhood program,19 minimized the intervention needed for each adolescent.

The intervention was delivered by trained patient immunization navigators (4.5 full-time equivalents), analogous to chronic disease patient navigators25 or promotoras.20 The navigators were recruited from the community; 1 fluent Spanish-speaking navigator was placed in the practice with the largest Hispanic population. They received formal training on the intervention, use of a database, health promotion, and methods to assist families to navigate the health and social service systems. The navigators were provided a workspace and a computer at each practice, and all were supervised by a social worker (M.S.). Their percentage effort in each practice was determined by their caseload, which varied from 600 to nearly 1000 per full-time equivalent.

Step 1: Patient Tracking

Because the study occurred before practices had incorporated the statewide immunization registry for adolescent vaccinations, we created a Web-based database for navigators to track the adolescents, record immunizations and preventive care visits, and document tasks performed.

Step 2: Reminders/Recall

Navigators performed reminder/recall for adolescents who were eligible for either a vaccination or a preventive care visit (with a 1-month grace period). They attempted to contact families by telephone (≥2 attempts at different times of day on different days) and mail (2 letters sent 2 weeks apart). The protocol involved 2 telephone calls (>1 week apart) and then 2 letters, and it started with letters if no telephone number was available. The navigators used a patient-centered and partnership-building approach to increase family awareness of preventive health measures and to address barriers to care. They offered transportation assistance (bus tokens and transport by car). If after the reminder/recall parents did not make and keep appointments, vaccination status was not brought up-to-date at the visit, or a subsequent human papillomavirus vaccination was needed, the navigators reinitiated the cycle. After 2 telephone calls and 2 letters, they moved to step 3.

Step 3: Home Visits

If adolescents remained unvaccinated despite the previous steps, the navigators performed a home visit to further assess barriers, promote the importance of preventive care, and encourage families to make appointments. The number of home visits was kept low to minimize personnel costs and maximize feasibility and sustainability. Control subjects received standard of care. All the practices routinely sent letter or telephone reminders to families who had upcoming scheduled visits, but none used active reminder/recall based on vaccinations.

Data Sources

Patient information from insurer or practice lists included name, address, telephone number, birth date, insurance type at the start of the intervention (Medicaid managed care, fee-for-service Medicaid, State Children's Health Insurance Program, commercial, or uninsured), and primary care practice. Patient information collected from the practices' medical records for the intervention included name, address, race/ethnicity (white, black, or Hispanic), language, telephone number, contact information for a parent/guardian, vaccination history, and preventive care visits. Summaries of navigator activities (telephone calls, letters, and home visits) were entered into the study database.

After the study intervention period, we reviewed medical records (paper or electronic) using a standardized medical record abstraction form for all adolescent immunization dates and preventive care visit dates. We also searched the New York State immunization registry for any additional vaccination dates. We used these data to determine baseline and follow-up immunization dates. Quality assurance checks performed on 5% of medical record reviews demonstrated high interrater reliability (κ ≥ 0.89).


The primary outcome was receipt of each recommended routine vaccination after age 11 years (meningococcus, pertussis, and 3 human papillomavirus vaccines [girls] and all vaccines combined). We did not include influenza vaccine because this study started before the introduction of universal influenza vaccination recommendations.26 We excluded varicella vaccine because varicella disease status or vaccine eligibility is difficult to assess by medical record review.27


Because it is recommended that all adolescents receive an annual preventive care visit, we assessed the receipt of a preventive care visit during the 12-month period before the end of the study (after age 11 years). We measured the costs of the intervention by summing total personnel costs (salaries of navigators [80%] and supervisory personnel [16%]) and nonpersonnel costs (including office supplies, cell phone costs, travel expenses for home visits and transports, and costs of the database [4%]).


To determine the relative risk (RR) of the intervention at the end of the study, we used multiple Poisson regression, with vaccination status and receipt of preventive care visits as the outcome and group status as the main explanatory variable and controlling for stratification variables (practice, age, and sex). To consider the lack of independence among siblings, we used the clustered Huber/White variance estimator for all analyses (STATA/SE, version 11.0; StataCorp LP, College Station, Texas).28

Because the intervention could affect only those not up-to-date at the beginning of the study, we excluded adolescents who were up-to-date at the study onset for each vaccine; for example, if an adolescent was up-to-date for meningococcus but not pertussis before the start of the study, he or she was excluded from the meningococcus analysis but was included in the pertussis and composite outcome analyses. For the preventive care analysis, we included an independent variable indicating whether the child had a preventive care visit within 12 months before the start of the study. We performed a prespecified subgroup analysis on age, sex, race, insurance, and practice subgroups for the composite immunization and preventive care outcomes. To control for the multiple testing, we adjusted confidence intervals for the 22 tests for the composite immunization outcome and 24 tests for the preventive care outcome using the Šidák method.29

We also assessed process and cost measures. We measured the number of reminder/recall messages and home visits. For the cost analyses, we included all the participants. We calculated the cost for each additional outcome (becoming vaccinated or receiving a preventive care visit) as the total cost of the intervention divided by ([No. of subjects] × [difference in the % of the outcome between the study and control groups]).30


Altogether, 7546 adolescents from 6682 families were randomized (Figure); 5910 families (88.4%) had 1 adolescent, 690 (10.3%) had 2, 74 (1.1%) had 3, 7 (0.1%) had 4, and 1 (0.01%) had 5. No controls received the intervention, and all the participants were included in an intent-to-treat analysis.

Image not available

Flow of participants through the study. Siblings were allocated to the same group.

The control and intervention groups had similar demographic characteristics and baseline immunization and preventive care visit rates (Table 1). The mean age at the start of the study was 13.5 years; half the participants were male; 63% were black, 14% white, and 23% Hispanic; and 74% had Medicaid and 6% were uninsured.

Table 1. 
Image not available
Baseline Characteristics, Immunization Rates, and Preventive Care Visit Rates by Randomization Group

Immunization rates at the end of the study for individual vaccines and for all 3 vaccines combined (including human papillomavirus for girls) were 12 to 16 percentage points higher for the intervention group than for the control group, with adjusted RR ratios ranging from 1.2 to 1.5 (P < .001 for all) (Table 2). Preventive care visit rates for each age group were 9 to 17 percentage points higher for the intervention group, with adjusted RR ratios ranging from 1.1 to 1.3 for each age group (P < .01 for all) (Table 3). The intervention had a substantial effect on most subgroups analyzed, including age, sex, race/ethnicity, insurance type, and practice (Table 4). Tests for interactions revealed that the intervention had greater effects on immunizations for girls than for boys (26% higher RR, P = .004) and for black or Hispanic adolescents than for white adolescents (40% and 27% higher RR, P = .009) and a greater effect on preventive care visits for those without a previous preventive care visit (13% higher RR, P = .02). Finally, we reexamined the results by clustering on practice and found no differences in any outcomes.

Table 2. 
Image not available
Immunization Rates at the End of the Study
Table 3. 
Image not available
Preventive Care Visit Rates at the End of the Study
Table 4. 
Image not available
Impact of the Intervention by Adolescent Characteristics and Practice

Altogether, 71% of the intervention group received either telephone or mail reminders, and 12% had a home visit. There were no adverse events from outreach; parents of 15 of 3839 participants declined navigator assistance but were included in the analyses. The total personnel plus nonpersonnel cost (subtracting research-related costs) was $45.74 per adolescent per year ($3.81 per month). The number needed to treat for an additional adolescent vaccinated was 9; the number needed to treat for an additional preventive care visit was also 9. The intervention cost per additional adolescent fully vaccinated was $465, and the cost per additional adolescent receiving a preventive care visit was $417.


In this clinical trial, which included more than 40% of adolescents in Rochester, we demonstrated that a tiered intervention of tracking, reminder/recall, and outreach provided by navigators for adolescent immunizations improved immunization rates by 12% to 16% depending on the vaccine and improved receipt of an annual preventive care visit by 9% to 17% depending on age. The intervention had a similar effect across all practices and irrespective of patient characteristics. These outcomes were accomplished primarily through telephone reminder/recall, with home visits for 12% of adolescents. The immunization navigator intervention cost was $3.81 per adolescent per month.

To our knowledge, this is the first large-scale study since introduction of the recommendations for new adolescent vaccines to demonstrate the benefit of a practice-based immunization intervention for urban adolescents. In fact, we found few published studies that have improved immunization or preventive care visit rates in urban adolescents. One randomized clinical trial18 investigated the effect of an autodialer-based telephone reminder/recall intervention (without patient navigators) for urban adolescents aged 11 to 14 years. Although that intervention was quite intensive, with up to 5 reminder calls per month, it had only a minimal effect on hepatitis B and tetanus immunization rates and no effect on preventive care visit rates.18 The key barrier was inaccurate telephone numbers for a relatively transient population, which experts postulate explains the limited success of telephone or mail reminder/recall interventions for low-income populations.11,14,16,17,31,32

The success of the present intervention raises 2 important lessons regarding reminder/recall for low-income urban populations. First, for populations that have not tended to benefit from traditional telephone or mail reminder/recall messages,1317 a tiered intervention that includes traditional reminder/recall plus more intense outreach may be beneficial. To maintain efficiency, more intense intervention should be reserved for adolescents who do not respond to lower-level interventions. The present results mirror the impact noted by similar tiered interventions targeting infants1921 and older adults.22

The second lesson is that the intervention has a cost at $3.81 per adolescent per month. This is similar to the cost of reminder/recall and outreach for infants or toddlers in the Rochester area,19 less than one-sixth the cost of a similar program targeting infants in Denver,20 and only a fraction of the cost of a home visitation and case management program to improve childhood immunization rates in Los Angeles.33 Furthermore, the present intervention produced benefits beyond immunizations by improving preventive care visit rates. This occurred because immunizations are typically bundled with preventive care visits.6,7 Although we did not measure receipt of preventive care services such as screening tests, anticipatory guidance, or tune-up of chronic conditions, these clinical preventive care services also tend to be coupled with preventive care visits.6 Furthermore, we could not find any other peer-reviewed publications of interventions that have successfully increased preventive care visits in urban adolescents, and this population has low rates of preventive care visits.34,35 Despite these benefits, the costs of the present intervention represented approximately 2.5% of total health care costs for adolescents in the Rochester region (Howard Brill, PhD, Monroe Plan for Medical Care, written communication, June 23, 2010).

Study strengths include the use of a clinically relevant intervention, a large and diverse sample, inclusion of a broad spectrum of primary care practices, ascertainment of important outcomes (immunization and preventive care visit rates), and an intention-to-treat design.

A limitation is the inability to distinguish the relative impact of different parts of the tiered intervention, such as reminders vs recall vs outreach. This is an inherent limitation of the multipart interventions that are now recommended for immunization delivery36 and other practice-based quality improvement interventions.37 Second, baseline immunization rates were low; benefits of the intervention may wane as rates rise. However, baseline preventive care visit rates mirrored national rates.5,7 To estimate the potential role of baseline rates, we performed a post hoc analysis comparing the odds of being up-to-date on overall immunization rates at the end of the study in children who had low (<5%), medium (5%-39%), and high (≥40%) baseline immunization rates and found no significant differences (odds ratios, 1.9, 1.7, and 1.8, respectively; P = .60). Third, this study had substantial research support, and we consider it an efficacy trial.38 Further implementation and dissemination studies are needed to assess continued effectiveness. Fourth, although studies have evaluated the cost-effectiveness of adolescent vaccinations,3941 none have evaluated costs of practice-based interventions to either raise immunization rates or improve adolescent preventive care visits. Also, we could not find cost-effectiveness studies of other adolescent preventive services to assess the relative value of the present intervention. Furthermore, because this intervention was applied across nearly half the city, costs per adolescent were likely lower than if the intervention were implemented in a single practice. Finally, the intervention may not be generalizable to other urban settings, suburban settings, or practices with high baseline rates.

In conclusion, a tiered patient navigator intervention consisting of immunization tracking, patient reminder/recall, and outreach based in inner-city primary care practices substantially increased adolescent immunization rates, with a spillover benefit of increased preventive care visit rates. Physicians and leaders in public health, managed care, and integrated health systems should consider this tiered navigator system for urban adolescents. Further research should assess the effectiveness and cost-effectiveness of the program in other settings.

Back to top
Article Information

Correspondence: Peter G. Szilagyi, MD, MPH, Department of Pediatrics, University of Rochester School of Medicine and Dentistry, PO Box 777, 601 Elmwood Ave, Rochester, NY 14642 (

Accepted for Publication: December 30, 2010.

Author Contributions: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Szilagyi, Humiston, Albertin, and Sandler. Acquisition of data: Szilagyi, Humiston, Gallivan, Albertin, and Blumkin. Analysis and interpretation of data: Szilagyi, Humiston, and Blumkin. Drafting of the manuscript: Szilagyi, Gallivan, and Blumkin. Critical revision of the manuscript for important intellectual content: Szilagyi, Humiston, Albertin, Sandler, and Blumkin. Statistical analysis: Szilagyi and Blumkin. Obtained funding: Szilagyi, Humiston, Albertin, and Sandler. Administrative, technical, and material support: Szilagyi, Humiston, Gallivan, Albertin, and Sandler. Study supervision: Szilagyi and Sandler.

Financial Disclosure: None reported.

Funding/Support: The study was funded by the Greater Rochester Health Foundation.

Role of the Sponsor: The Greater Rochester Health Foundation had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.

Previous Presentation: This study was presented in part at the Pediatric Academic Societies meetings; May 2, 2010; Vancouver, British Columbia, Canada.

Additional Contributions: We appreciate the collaboration of the physicians and staff at the 8 primary care practices in Rochester.

Bilukha  OORosenstein  NNational Center for Infectious Diseases, Centers for Disease Control and Prevention (CDC), Prevention and control of meningococcal disease: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2005;54 (RR-7) 1- 21
Markowitz  LEDunne  EFSaraiya  MLawson  HWChesson  HUnger  ERCenters for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP), Quadrivalent human papillomavirus vaccine: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2007;56 (RR-2) 1- 24
Broder  KRCortese  MMIskander  JK  et al. Advisory Committee on Immunization Practices (ACIP), Preventing tetanus, diphtheria, and pertussis among adolescents: use of tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2006;55 (RR-3) 1- 34
Centers for Disease Control and Prevention (CDC), National, state, and local area vaccination coverage among adolescents aged 13-17 years—United States, 2008. MMWR Morb Mortal Wkly Rep 2009;58 (36) 997- 1001
Ziv  ABoulet  JRSlap  GB Utilization of physician offices by adolescents in the United States. Pediatrics 1999;104 (1, pt 1) 35- 42
Broder  KRCohn  ACSchwartz  B  et al. Working Group on Adolescent Prevention Priorities, Adolescent immunizations and other clinical preventive services: a needle and a hook? Pediatrics 2008;121 ((suppl 1)) S25- S34
Szilagyi  PGRand  CMMcLaurin  J  et al. Working Group on Adolescent Vaccination in the Medical Home, Delivering adolescent vaccinations in the medical home: a new era? Pediatrics 2008;121 ((suppl 1)) S15- S24
Rand  CMSzilagyi  PGAlbertin  CAuinger  P Additional health care visits needed among adolescents for human papillomavirus vaccine delivery within medical homes: a national study. Pediatrics 2007;120 (3) 461- 466
Rand  CMSzilagyi  PGYoo  BKAuinger  PAlbertin  CColeman  MS Additional visit burden for universal influenza vaccination of US school-aged children and adolescents. Arch Pediatr Adolesc Med 2008;162 (11) 1048- 1055
Briss  PARodewald  LEHinman  AR  et al. The Task Force on Community Preventive Services, Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med 2000;18 (1) ((suppl)) 97- 140
Szilagyi  PVann  JBordley  C  et al.  Interventions aimed at improving immunization rates. Cochrane Database Syst Rev 2002; (4) CD003941
Szilagyi  PGBordley  CVann  JC  et al.  Effect of patient reminder/recall interventions on immunization rates: a review. JAMA 2000;284 (14) 1820- 1827
Kempe  ALowery  NEPearson  KA  et al.  Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr 2001;139 (5) 630- 635
Daley  MFSteiner  JFKempe  A  et al.  Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr 2004;4 (3) 217- 223
Davis  MMSzilagyi  PG Can quality improvement reach into pockets of need for childhood immunizations? Ambul Pediatr 2004;4 (3) 224- 225
Hambidge  SJDavidson  AJPhibbs  SL  et al.  Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial. Arch Pediatr Adolesc Med 2004;158 (2) 162- 169
LeBaron  CWStarnes  DMRask  KJ The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Arch Pediatr Adolesc Med 2004;158 (3) 255- 261
Szilagyi  PGSchaffer  SBarth  R  et al.  Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med 2006;160 (2) 157- 163
Rodewald  LESzilagyi  PGHumiston  SGBarth  RKraus  RRaubertas  RF A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics 1999;103 (1) 31- 38
Hambidge  SJPhibbs  SLChandramouli  VFairclough  DSteiner  JF A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124 (2) 455- 464
Szilagyi  PGSchaffer  SShone  L  et al.  Reducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices. Pediatrics 2002;110 (5) e58
Kicera  TJDouglas  MGuerra  FA Best-practice models that work: the CDC's Racial and Ethnic Adult Disparities Immunization Initiative (READII) programs. Ethn Dis 2005;2 ((suppl 3)) S317- S320
US Census Bureau, Monroe County, New York.  Washington, DC US Census Bureau2000;
New York State Dept of Health, Managed Care Plan Enrollment Report—2009.  Albany, NY New York State Department of Health2009;
Franks  PFiscella  K Reducing disparities downstream: prospects and challenges. J Gen Intern Med 2008;23 (5) 672- 677
Fiore  AEShay  DKBroder  K  et al. Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP), Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008. MMWR Recomm Rep 2008;57 (RR-7) 1- 60
Rodewald  LMaes  EStevenson  JLyons  BStokley  SSzilagyi  P Immunization performance measurement in a changing immunization environment. Pediatrics 1999;103 (4, pt 2) 889- 897
Williams  RL A note on robust variance estimation for cluster-correlated data. Biometrics 2000;56 (2) 645- 646
Lagakos  SW The challenge of subgroup analyses—reporting without distorting. N Engl J Med 2006;354 (16) 1667- 1669
Hoch  JSBlume  JD Measuring and illustrating statistical evidence in a cost-effectiveness analysis. J Health Econ 2008;27 (2) 476- 495
Daley  MFBarrow  JPearson  K  et al.  Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics 2004;113 (1, pt 1) e26- e33
Kempe  ABeaty  BLSteiner  JF  et al.  The regional immunization registry as a public health tool for improving clinical practice and guiding immunization delivery policy. Am J Public Health 2004;94 (6) 967- 972
Wood  DHalfon  NDonald-Sherbourne  C  et al.  Increasing immunization rates among inner-city, African American children. A randomized trial of case management. JAMA 1998;279 (1) 29- 34
Poehling  KASzilagyi  PG Not just for kids: new paradigms for vaccine delivery in pediatrics. Acad Pediatr 2009;9 (5) 293- 294
Dempsey  AFSinger  DClark  SJDavis  MM Parents' views on 3 shot-related visits: implications for use of adolescent vaccines like human papillomavirus vaccine. Acad Pediatr 2009;9 (5) 348- 352
Ndiaye  SMHopkins  DPSmith  SJHinman  ARBriss  PATask Force on Community Preventive Services, Methods for conducting systematic reviews of targeted vaccination strategies for the guide to community preventive services. Am J Prev Med 2005;28 (5) ((suppl)) 238- 247
Homer  CJForbes  PHorvitz  LPeterson  LEWypij  DHeinrich  P Impact of a quality improvement program on care and outcomes for children with asthma. Arch Pediatr Adolesc Med 2005;159 (5) 464- 469
Szilagyi  PG Translational research and pediatrics. Acad Pediatr 2009;9 (2) 71- 80
Kulasingam  SLMyers  ER Potential health and economic impact of adding a human papillomavirus vaccine to screening programs. JAMA 2003;290 (6) 781- 789
Shepard  CWOrtega-Sanchez  IRScott  RD  IIRosenstein  NEABCs Team, Cost-effectiveness of conjugate meningococcal vaccination strategies in the United States. Pediatrics 2005;115 (5) 1220- 1232
Ortega-Sanchez  IRLee  GMJacobs  RJ  et al. Working Group on Leading Economic Issues for New Vaccine for Adolescents, Projected cost-effectiveness of new vaccines for adolescents in the United States. Pediatrics 2008;121 ((suppl 1)) S63- S78