Effect of Telephone Reminder/Recall on Adolescent Immunization and Preventive Visits: Results From a Randomized Clinical Trial

Objective  To measure the effect of telephone-based reminder/recall on immunization and well-child care (WCC) visit rates among adolescents in urban practices.

Design  Randomized clinical trial of telephone-based reminder/recall over 18 months.

Setting  Four urban primary care practices.

Participants  Adolescents aged 11 to 14 years.

Intervention  Adolescents within practices were randomized to study (n = 1496) or control groups (n = 1510). The study group was sent audiotaped telephone reminders about a scheduled or needed immunization or WCC visit. Households were called weekly if there was no response; telephone numbers were updated weekly. Controls received standard care.

Results  Baseline demographics and immunization and WCC visit rates were similar for study and control groups. The intervention was largely ineffective in improving immunization or WCC visit rates. Although at the end of the study, the study group had slightly higher hepatitis B coverage (3 vaccinations) (62% vs 57.8%; P = .02), WCC visits were the same (53% and 54%), and impact on other vaccinations was minimal. The effect of reminder/recall was equivalent across demographic subgroups (eg, age, race/ethnicity, insurance). The major factor limiting intervention effectiveness was inaccurate telephone numbers. Seventy-one percent of study subjects with single telephone numbers throughout the study had a WCC visit vs 25% of study subjects with multiple/changed telephone numbers and 54% of controls (P<.001).

Conclusions  An intensive telephone reminder and recall system was only minimally successful in improving immunization and WCC visit rates among urban adolescents. Lack of success was largely owing to changed or inaccurate telephone numbers.

A national health priority is to improve immunization rates to reduce vaccine-preventable diseases.¹ Studies have documented effective strategies for improving childhood and adult immunization rates.^2-12

In spite of the plethora of studies on young children and older adults, surprisingly little attention has been paid to adolescent immunizations.¹³ Until recently, adolescent immunizations involved only catch-up and booster doses of tetanus vaccine and measles-containing vaccine. During the past decade, new hepatitis B,^14-16 varicella,¹⁷ and influenza vaccination¹⁸ guidelines heightened attention to adolescent vaccinations. Adolescent vaccinations will soon become a major public health and pediatric focus because of new policies regarding pertussis^19-21 and meningococcal^22-25 vaccines; the development of vaccines to prevent sexually transmitted infections, including human papillomavirus,^26,27 herpes simplex,²⁸ and chlamydia²⁹; and increasing emphasis on underused vaccines such as influenza,^30,31 varicella,³² and hepatitis A³³ and B.^34-36

In 1997, the Advisory Committee on Immunization Practices (Atlanta, Ga), the American Academy of Pediatrics (Elk Grove Village, Ill), the American Academy of Family Physicians (Leawood, Kan), and the American Medical Association issued recommendations to improve the delivery of adolescent immunizations.³⁷ These guidelines highlighted barriers to adolescent immunizations: inability of health care professionals to track and recall adolescents who need vaccinations, missed opportunities for vaccinations within health care settings, and record scattering.^35,38 Additional barriers can involve parental consent, because parents are frequently absent during adolescent health visits³⁹; lack of health insurance⁴⁰; and problems with access to health care.^41-45

Few studies have evaluated interventions for adolescent immunizations. Studies of school-based adolescent immunization programs have noted mixed success.⁴⁶ Systematic telephone or mail reminder/recall systems, which are often,^4,5,47 but not always,^48-50 effective for increasing immunization rates among young children or adults, are recommended for adolescents but have not been evaluated for this vulnerable^33,37,39 population. A study of letter reminders to adolescents or parents noted poor response rates⁵¹ and had no comparison group.

Our study evaluated the effectiveness of a systematic telephone-based reminder/recall system on immunization rates and well-child care (WCC) visits among adolescents attending 4 urban primary care practices that serve predominantly city residents.

The University of Rochester (Rochester, NY) Research Subjects Review Board approved this study. The study was performed at 4 urban primary care practices located in Rochester, a city with a population of 250 000 and high rates of child poverty⁵² within a county of 750 000 residents. The practices included 2 pediatric group practices, a hospital-based pediatric clinic, and a family medicine–based neighborhood health center. These large practices are representative of urban primary care practices and together serve 19% of the county's children.

We analyzed each practice's billing database for adolescents who had 1 or more visits at each site. From 5902 potential subjects with a birth date between June 1, 1983, and May 31, 1987 (aged 11-14 years at the start of the intervention), we excluded (1) siblings (randomly selecting 1 child per family), (2) those with no practice visits within 24 months, (3) those residing outside the county, and (4) those with no telephone number in the database. Prior power calculations to detect a 10% improvement in baseline immunization rates of 50% (power of 0.80; α = .05) within each practice required more than 750 adolescents per practice. From 5902 potential subjects, 3006 were eligible for randomization.

The 3006 subjects were stratified into 2 equal age groups (11-12 years and 13-14 years at the start of the study) and randomly allocated into a study group (n = 1496) or control group (n = 1510) using a random-number generator with the child as the unit of randomization. Health care professionals were unaware of group allocation for specific subjects because the intervention used research personnel and reminders from a central office.

We used an automated telephone message reminder system (autodialer⁵³), previously noted to improve preschool immunization rates in 2 health clinics^54,55 and a health maintenance organization setting.⁵⁶ A medical record review at the beginning of the study identified telephone numbers for the autodialers, a database tracked adolescents needing reminders, and a research assistant verified weekly upcoming appointments, recent WCC visits or immunizations, and changes in telephone numbers. The intervention mimicked an appointment-scheduling module that is linked to a telephone-reminder system. The intervention occurred between August 8, 1998, and February 29, 2000.

Adolescents were called if they were due for an annual WCC visit, a tetanus booster (>5 years since diphtheria and tetanus toxoids and pertussis vaccination), or a hepatitis B vaccination according to Advisory Committee on Immunization Practices guidelines. A variable number of calls was placed depending on the need for immunizations or WCC visits and prior response to reminder calls. The calls were voice recordings in English to request a vaccination appointment or WCC visit or to remind families of upcoming scheduled appointments. Calls were made 6 days per week during the day or early evening.

During the initial 11 months of the 18-month clinical trial, telephone calls were stopped if (1) recipients indicated from a telephone menu option that the telephone number was incorrect, the adolescent had left the practice, or the parent requested calls to be stopped or (2) 5 calls were placed within 30 days and no appointment was scheduled (“unresponsive numbers”). After 11 months, the autodialer telephone reminder calls were restarted for those subjects with “unresponsive numbers” to give families a second opportunity to respond to subsequent reminders.

Based on medical record review at the end of the study, subjects who had moved or subjects for whom no medical record was found were considered “inactive” (132 study subjects and 168 controls). Because we performed an intention-to-treat analysis, these subjects were considered “behind in immunizations” or “not having a WCC visit.”

The key independent measure was group assignment (study vs control). Key demographic subgroups^57-59 (Table 1) delineated from billing files included: subject's age (11-14 years), sex, residence (inner city, rest of the city of Rochester, and suburbs), primary care practice (each of 4), insurance (6 categories), and race/ethnicity (white non-Hispanic, black non-Hispanic, Hispanic, or other/unknown). Analyses by residence, health insurance, and race/ethnicity combined subjects across practices and age groups.

Key dependent measures were obtained by blinded medical record reviews at the end of the study using a standardized medical record review form. Quality assurance checks for 5% of subjects noted more than 98% reliability.

Dependent measures were immunization-related outcomes: (1) up-to-date rates (yes/no) for hepatitis B (first, second, and third doses) and tetanus-diphtheria toxoids booster (Td); and (2) mean number of days eligible for each vaccine during the study time frame. Health care visit measures included (1) receipt (yes/no) of WCC within the prior year and (2) mean number of days eligible for WCC visits during the study time frame.

Intention-to-treat analyses were performed for the 1496 study and 1510 control subjects using χ² tests for categorical variables and t tests or analysis of variance for numeric variables.

For 2 key measures (the initial hepatitis B vaccination and a WCC visit), Kaplan-Meier survival curves were generated to determine the number of days eligible for a vaccine or WCC visit throughout the 18-month study. Results for study subjects were compared with results for control subjects using a log-rank test.

Because lack of telephones, incorrect telephone numbers, or changed numbers are common among impoverished families,^47,48 we anticipated that many study subjects would never receive the telephone messages. Thus, a single post hoc analysis compared results for subjects for whom only 1 telephone number was noted throughout the study vs subjects for whom multiple telephone numbers were noted throughout the study. We surmised that subjects with a single telephone number represented a less mobile⁶⁰ subgroup of families who may have had greater exposure to the intervention.

Study and control groups were similar with respect to age group, sex, practice, insurance, and race/ethnicity (Table 1). Most were from the city of Rochester, many had Medicaid coverage, and a high proportion were black or Hispanic.

At baseline, study and control groups were similar for hepatitis B and Td coverage and WCC visit rates. At the end of the intervention, the study group had higher hepatitis B coverage (3 vaccinations) (62.0% vs 57.8%; P = .02), but no difference was noted in coverage for Td or WCC visit rates. Although both study and control groups experienced increases in coverage for hepatitis B and Td vaccinations, there was a statistically significant difference between study and control groups in the change in hepatitis B coverage but not in the change in Td or WCC rates (Table 2).

Table 3 presents more detailed immunization and health care visit information. Since some subjects were never eligible for some immunizations, we assessed whether eligible adolescents received immunizations or WCC visits during the study. No statistically significant differences were noted between study and control groups, although there was a trend toward higher hepatitis B rates (3 vaccinations) among study subjects. The mean number of days eligible for a vaccination or a WCC visit and the mean number of WCC or other visits were also similar for study and control groups, except for slightly higher rates of Td vaccination among study subjects.

We examined immunization and WCC visits measures for study vs control groups for the key demographic characteristics in Table 1, and no substantive differences between study and control groups were noted. This suggests that the reminder/recall intervention did not perform differently for subgroups that varied by these demographic characteristics.

The intervention appeared to be marginally beneficial during the first few months of the study, with benefit waning afterward. For example, during month 1, 87.8% of study and 88.4% of control subjects needed a vaccination or WCC visit, 81% of study subjects were sent a telephone reminder, and study subjects were more likely than controls to have a visit within 3 months (study subjects, 40.4%; controls, 34.4%; P = .001) or a vaccination (study subjects, 18.1%; controls, 11.6%; P<.001). For month 2, eligible study subjects again were more likely than controls to have a visit (P<.05) or vaccination (P<.001). Thereafter, benefits for WCC visits were no longer noted, and after 4 months, benefits for vaccinations were no longer noted.

The Figure shows Kaplan-Meier survival curves for the proportion of adolescents eligible for hepatitis B vaccination (Figure A) or WCC visit (Figure B). The y-axis shows the proportion eligible (starting with 1.0), and the x-axis shows the days during the 18-month study. As seen by comparing the “all study subjects” vs “all control subjects” groups, more control subjects than study subjects remained eligible for hepatitis B vaccination or WCC visits in the early months of the study, but the effect waned after several months.

Since the benefit of reminder/recall was minimal and limited to the initial months of the study, we examined factors that may have limited its effectiveness. Two factors that accounted for some reminder calls being discontinued were families (1) responding to the autodialer message that the adolescent was no longer a patient of the practice (3.4%) or (2) requesting calls to be discontinued for no stated reason (9.8%). The major group involved parents simply not responding to the reminders with no explanation (62.8%), which eventually resulted in the calls being “turned off.” We cannot determine the proportion of these families that simply ignored reminders vs wrong telephone numbers.

The Figure also shows Kaplan-Meier survival curves for the proportion of subjects eligible for hepatitis B vaccination (Figure A) or WCC visit (Figure B) among subjects who had 1 vs more than 1 telephone number during the study period. Fully 69% (n = 1034) of study subjects had 2 or more different telephone numbers during the 18-month study period. The one third of study subjects with a single telephone number throughout the 18-month period were far more likely (P<.001) to receive a hepatitis B vaccination and a WCC visit than were the study subjects who had multiple telephone numbers. Results for other outcomes followed a similar pattern.

An intensive autodialer-based telephone reminder and recall system was only marginally successful in improving immunization rates and WCC visits among adolescents attending 4 urban primary care practices. An initial effect waned after several months. The small effect (4-7 percentage points) appears consistent with the small effect noted in many reminder/recall studies.^5,61 The intervention did not appear to be any more or less effective for demographic subgroups stratified by age, geographic residence, insurance status, and race/ethnicity. The major factor limiting effectiveness of reminder/recall was inaccurate telephone numbers. The initial improvements were probably owing to the reminders reaching those families with accurate telephone numbers or those who were going to respond. Continued intervention had no effect.

Patient reminder/recall can improve immunization rates^4,5,54 and preventive visit rates⁶² in children or adults, but several recent studies,^47,48 including 1 autodialer-based study⁶³ focusing on urban infants, have failed to find benefit. The most commonly cited reason for the lack of effectiveness is difficulty in successfully contacting the patient population, a concern regarding adolescents who make few regular health care visits. In our study, lack of an accurate telephone number was the major problem. Of 1496 study subjects, 69% had at least 2 distinct telephone numbers over 18 months. The subgroup that had a single telephone number had markedly greater response to the telephone reminders than did the subgroup with 2 or more telephone numbers. Our findings confirm that the effectiveness of the patient reminder/recall is strongly associated with the accuracy of telephone numbers or addresses, which was low among this population.

Unfortunately, practices cannot discern beforehand which families are likely to change their telephone number (or addresses). Practices should update telephone numbers during all patient encounters and obtain alternate contact information such as mobile telephone numbers. Our study also suggests that new vaccines requiring multiple doses (eg, human papillomavirus vaccine) may pose a challenge for vaccinating the urban adolescent population. Given that our intervention had some initial benefit, probably because it reached those families that could be reached, health care professionals serving urban populations may consider short-term reminder/recall interventions instead of prolonged and intensive interventions.

We expect that the use of autodialers or similar technology will increase as computerized appointment scheduling modules are implemented.⁵⁸ The costs of autodialer-based reminder systems are relatively low if (1) telephone numbers are updated electronically using links with billing systems and (2) the systems are used by hospital or health systems that apply them to populations beyond adolescents.⁵⁴ As an example, 1 of the study practices has just begun using an autodialer reminder system for scheduled appointments as part of a hospitalwide autodialer reminder system. Costs to the practice are only a few hundred dollars per year because costs are distributed hospitalwide. Thus, even with very small increases in show rates, this reminder system is probably revenue generating.

Our study highlights the importance of checking adolescents' immunization status at all visits and using visits other than WCC visits as opportunities to immunize adolescents. More than half the study group had at least 1 visit with a missed opportunity for vaccination with hepatitis B or Td vaccine.

Our study has several limitations. The major threat to generalizability is the study setting—urban practices in a metropolitan area. Patient characteristics, the accuracy of telephone numbers, baseline immunization coverage, and practice characteristics may differ in other settings. Baseline vaccination rates were within the wide spectrum of rates noted nationally.^14,34,64 Demographic characteristics of our population were similar to characteristics of other urban settings. Also, although hepatitis B vaccination among adolescents will become less important (because of infant vaccination), it serves as a model for human papillomavirus vaccination because that schedule will be similar to the hepatitis B schedule.

One threat to internal validity is that we undoubtedly included adolescents who were not current patients of the practices because there was no perfect method to select eligible adolescents. Some adolescents may have received vaccinations elsewhere. The major limitation to the intervention was the inability to determine accurate telephone numbers in spite of the research assistant checking medical records and web-based directories weekly. Coordination of practice-based interventions with Medicaid managed care may improve effectiveness since the requirements to reenroll into Medicaid managed care plans provide opportunities for managed care organizations to update patient contact information.

It is possible that the autodialer was not as effective as a direct telephone call from a person would have been. Studies are lacking comparing autodialer-based reminders with person-to-person reminders. Further, we did not attempt to ascertain vaccinations received outside of the practice; however, in this community, few vaccinations are provided to adolescents in schools or public health clinics.⁶¹ Finally, we did not perform cost analyses because we were using a labor-intensive method to update kept appointments and changes in telephone numbers.

We conclude that an intensive primary care practice-based telephone reminder/recall intervention was only marginally effective by slightly improving the receipt of adolescent immunizations visits within these urban primary care practices. The critical factor associated with success of the intervention was the accuracy of telephone numbers, which was low among our population.

An implication for health care professionals is that practices are more likely to have success with patient reminder/recall if they have accurate patient telephone numbers. In these cases, even small practices may consider using reminder systems. For practices within hospital or health systems, practice-level costs of reminder/recall systems may be low because costs would be distributed across an entire system, and large numbers of patients beyond adolescents could benefit. In these settings, an autodialer-based reminder/recall system that has only a small effect should still be considered. However, to achieve high levels of preventive care among urban adolescents, more aggressive strategies (such as outreach^13,65 or combinations of interventions) may be needed.

An implication for communities and public health is that in some settings, practice-based patient reminder/recall might be only marginally effective in improving adolescent vaccinations and preventive visits. These interventions may be more effective if combined with updated contact information from managed care organizations, immunization registries, or integrated delivery systems. With emerging new adolescent vaccines, aggressive strategies will be needed to ensure high adolescent vaccination coverage as well as receipt of preventive adolescent services.

Correspondence: Peter G. Szilagyi, MD, MPH, Department of Pediatrics, Strong Memorial Hospital, Box 632, Rochester, NY 14642 (peter_szilagyi@urmc.rochester.edu).

Accepted for Publication: September 6, 2005.

Funding/Support: This work was funded by the Centers for Disease Control and Prevention, Atlanta, Ga, and Cooperative Agreement TS-622 from the Association for Teachers of Preventive Medicine, Washington, DC.

Acknowledgment: We gratefully acknowledge the statistical consultation of Jason Roy, PhD.