Objective
To measure the effect of telephone-based reminder/recall on immunization and well-child care (WCC) visit rates among adolescents in urban practices.
Design
Randomized clinical trial of telephone-based reminder/recall over 18 months.
Setting
Four urban primary care practices.
Participants
Adolescents aged 11 to 14 years.
Intervention
Adolescents within practices were randomized to study (n = 1496) or control groups (n = 1510). The study group was sent audiotaped telephone reminders about a scheduled or needed immunization or WCC visit. Households were called weekly if there was no response; telephone numbers were updated weekly. Controls received standard care.
Results
Baseline demographics and immunization and WCC visit rates were similar for study and control groups. The intervention was largely ineffective in improving immunization or WCC visit rates. Although at the end of the study, the study group had slightly higher hepatitis B coverage (3 vaccinations) (62% vs 57.8%; P = .02), WCC visits were the same (53% and 54%), and impact on other vaccinations was minimal. The effect of reminder/recall was equivalent across demographic subgroups (eg, age, race/ethnicity, insurance). The major factor limiting intervention effectiveness was inaccurate telephone numbers. Seventy-one percent of study subjects with single telephone numbers throughout the study had a WCC visit vs 25% of study subjects with multiple/changed telephone numbers and 54% of controls (P<.001).
Conclusions
An intensive telephone reminder and recall system was only minimally successful in improving immunization and WCC visit rates among urban adolescents. Lack of success was largely owing to changed or inaccurate telephone numbers.
A national health priority is to improve immunization rates to reduce vaccine-preventable diseases.1 Studies have documented effective strategies for improving childhood and adult immunization rates.2-12
In spite of the plethora of studies on young children and older adults, surprisingly little attention has been paid to adolescent immunizations.13 Until recently, adolescent immunizations involved only catch-up and booster doses of tetanus vaccine and measles-containing vaccine. During the past decade, new hepatitis B,14-16 varicella,17 and influenza vaccination18 guidelines heightened attention to adolescent vaccinations. Adolescent vaccinations will soon become a major public health and pediatric focus because of new policies regarding pertussis19-21 and meningococcal22-25 vaccines; the development of vaccines to prevent sexually transmitted infections, including human papillomavirus,26,27 herpes simplex,28 and chlamydia29; and increasing emphasis on underused vaccines such as influenza,30,31 varicella,32 and hepatitis A33 and B.34-36
In 1997, the Advisory Committee on Immunization Practices (Atlanta, Ga), the American Academy of Pediatrics (Elk Grove Village, Ill), the American Academy of Family Physicians (Leawood, Kan), and the American Medical Association issued recommendations to improve the delivery of adolescent immunizations.37 These guidelines highlighted barriers to adolescent immunizations: inability of health care professionals to track and recall adolescents who need vaccinations, missed opportunities for vaccinations within health care settings, and record scattering.35,38 Additional barriers can involve parental consent, because parents are frequently absent during adolescent health visits39; lack of health insurance40; and problems with access to health care.41-45
Few studies have evaluated interventions for adolescent immunizations. Studies of school-based adolescent immunization programs have noted mixed success.46 Systematic telephone or mail reminder/recall systems, which are often,4,5,47 but not always,48-50 effective for increasing immunization rates among young children or adults, are recommended for adolescents but have not been evaluated for this vulnerable33,37,39 population. A study of letter reminders to adolescents or parents noted poor response rates51 and had no comparison group.
Our study evaluated the effectiveness of a systematic telephone-based reminder/recall system on immunization rates and well-child care (WCC) visits among adolescents attending 4 urban primary care practices that serve predominantly city residents.
The University of Rochester (Rochester, NY) Research Subjects Review Board approved this study. The study was performed at 4 urban primary care practices located in Rochester, a city with a population of 250 000 and high rates of child poverty52 within a county of 750 000 residents. The practices included 2 pediatric group practices, a hospital-based pediatric clinic, and a family medicine–based neighborhood health center. These large practices are representative of urban primary care practices and together serve 19% of the county's children.
We analyzed each practice's billing database for adolescents who had 1 or more visits at each site. From 5902 potential subjects with a birth date between June 1, 1983, and May 31, 1987 (aged 11-14 years at the start of the intervention), we excluded (1) siblings (randomly selecting 1 child per family), (2) those with no practice visits within 24 months, (3) those residing outside the county, and (4) those with no telephone number in the database. Prior power calculations to detect a 10% improvement in baseline immunization rates of 50% (power of 0.80; α = .05) within each practice required more than 750 adolescents per practice. From 5902 potential subjects, 3006 were eligible for randomization.
The 3006 subjects were stratified into 2 equal age groups (11-12 years and 13-14 years at the start of the study) and randomly allocated into a study group (n = 1496) or control group (n = 1510) using a random-number generator with the child as the unit of randomization. Health care professionals were unaware of group allocation for specific subjects because the intervention used research personnel and reminders from a central office.
Intervention (study group)
We used an automated telephone message reminder system (autodialer53), previously noted to improve preschool immunization rates in 2 health clinics54,55 and a health maintenance organization setting.56 A medical record review at the beginning of the study identified telephone numbers for the autodialers, a database tracked adolescents needing reminders, and a research assistant verified weekly upcoming appointments, recent WCC visits or immunizations, and changes in telephone numbers. The intervention mimicked an appointment-scheduling module that is linked to a telephone-reminder system. The intervention occurred between August 8, 1998, and February 29, 2000.
Algorithms for telephone calls
Adolescents were called if they were due for an annual WCC visit, a tetanus booster (>5 years since diphtheria and tetanus toxoids and pertussis vaccination), or a hepatitis B vaccination according to Advisory Committee on Immunization Practices guidelines. A variable number of calls was placed depending on the need for immunizations or WCC visits and prior response to reminder calls. The calls were voice recordings in English to request a vaccination appointment or WCC visit or to remind families of upcoming scheduled appointments. Calls were made 6 days per week during the day or early evening.
During the initial 11 months of the 18-month clinical trial, telephone calls were stopped if (1) recipients indicated from a telephone menu option that the telephone number was incorrect, the adolescent had left the practice, or the parent requested calls to be stopped or (2) 5 calls were placed within 30 days and no appointment was scheduled (“unresponsive numbers”). After 11 months, the autodialer telephone reminder calls were restarted for those subjects with “unresponsive numbers” to give families a second opportunity to respond to subsequent reminders.
Participant flow during the study
Based on medical record review at the end of the study, subjects who had moved or subjects for whom no medical record was found were considered “inactive” (132 study subjects and 168 controls). Because we performed an intention-to-treat analysis, these subjects were considered “behind in immunizations” or “not having a WCC visit.”
The key independent measure was group assignment (study vs control). Key demographic subgroups57-59 (Table 1) delineated from billing files included: subject's age (11-14 years), sex, residence (inner city, rest of the city of Rochester, and suburbs), primary care practice (each of 4), insurance (6 categories), and race/ethnicity (white non-Hispanic, black non-Hispanic, Hispanic, or other/unknown). Analyses by residence, health insurance, and race/ethnicity combined subjects across practices and age groups.
Key dependent measures were obtained by blinded medical record reviews at the end of the study using a standardized medical record review form. Quality assurance checks for 5% of subjects noted more than 98% reliability.
Dependent measures were immunization-related outcomes: (1) up-to-date rates (yes/no) for hepatitis B (first, second, and third doses) and tetanus-diphtheria toxoids booster (Td); and (2) mean number of days eligible for each vaccine during the study time frame. Health care visit measures included (1) receipt (yes/no) of WCC within the prior year and (2) mean number of days eligible for WCC visits during the study time frame.
Intention-to-treat analyses were performed for the 1496 study and 1510 control subjects using χ2 tests for categorical variables and t tests or analysis of variance for numeric variables.
For 2 key measures (the initial hepatitis B vaccination and a WCC visit), Kaplan-Meier survival curves were generated to determine the number of days eligible for a vaccine or WCC visit throughout the 18-month study. Results for study subjects were compared with results for control subjects using a log-rank test.
Because lack of telephones, incorrect telephone numbers, or changed numbers are common among impoverished families,47,48 we anticipated that many study subjects would never receive the telephone messages. Thus, a single post hoc analysis compared results for subjects for whom only 1 telephone number was noted throughout the study vs subjects for whom multiple telephone numbers were noted throughout the study. We surmised that subjects with a single telephone number represented a less mobile60 subgroup of families who may have had greater exposure to the intervention.
Study and control groups were similar with respect to age group, sex, practice, insurance, and race/ethnicity (Table 1). Most were from the city of Rochester, many had Medicaid coverage, and a high proportion were black or Hispanic.
Impact of reminder/recall on immunization and wcc rates
At baseline, study and control groups were similar for hepatitis B and Td coverage and WCC visit rates. At the end of the intervention, the study group had higher hepatitis B coverage (3 vaccinations) (62.0% vs 57.8%; P = .02), but no difference was noted in coverage for Td or WCC visit rates. Although both study and control groups experienced increases in coverage for hepatitis B and Td vaccinations, there was a statistically significant difference between study and control groups in the change in hepatitis B coverage but not in the change in Td or WCC rates (Table 2).
Table 3 presents more detailed immunization and health care visit information. Since some subjects were never eligible for some immunizations, we assessed whether eligible adolescents received immunizations or WCC visits during the study. No statistically significant differences were noted between study and control groups, although there was a trend toward higher hepatitis B rates (3 vaccinations) among study subjects. The mean number of days eligible for a vaccination or a WCC visit and the mean number of WCC or other visits were also similar for study and control groups, except for slightly higher rates of Td vaccination among study subjects.
Impact of reminder/recall on subgroups
We examined immunization and WCC visits measures for study vs control groups for the key demographic characteristics in Table 1, and no substantive differences between study and control groups were noted. This suggests that the reminder/recall intervention did not perform differently for subgroups that varied by these demographic characteristics.
Waning benefit of the intervention over time
The intervention appeared to be marginally beneficial during the first few months of the study, with benefit waning afterward. For example, during month 1, 87.8% of study and 88.4% of control subjects needed a vaccination or WCC visit, 81% of study subjects were sent a telephone reminder, and study subjects were more likely than controls to have a visit within 3 months (study subjects, 40.4%; controls, 34.4%; P = .001) or a vaccination (study subjects, 18.1%; controls, 11.6%; P<.001). For month 2, eligible study subjects again were more likely than controls to have a visit (P<.05) or vaccination (P<.001). Thereafter, benefits for WCC visits were no longer noted, and after 4 months, benefits for vaccinations were no longer noted.
The Figure shows Kaplan-Meier survival curves for the proportion of adolescents eligible for hepatitis B vaccination (Figure A) or WCC visit (Figure B). The y-axis shows the proportion eligible (starting with 1.0), and the x-axis shows the days during the 18-month study. As seen by comparing the “all study subjects” vs “all control subjects” groups, more control subjects than study subjects remained eligible for hepatitis B vaccination or WCC visits in the early months of the study, but the effect waned after several months.
Factors limiting effectiveness of reminder/recall
Since the benefit of reminder/recall was minimal and limited to the initial months of the study, we examined factors that may have limited its effectiveness. Two factors that accounted for some reminder calls being discontinued were families (1) responding to the autodialer message that the adolescent was no longer a patient of the practice (3.4%) or (2) requesting calls to be discontinued for no stated reason (9.8%). The major group involved parents simply not responding to the reminders with no explanation (62.8%), which eventually resulted in the calls being “turned off.” We cannot determine the proportion of these families that simply ignored reminders vs wrong telephone numbers.
Post hoc analysis by single or multiple telephone numbers
The Figure also shows Kaplan-Meier survival curves for the proportion of subjects eligible for hepatitis B vaccination (Figure A) or WCC visit (Figure B) among subjects who had 1 vs more than 1 telephone number during the study period. Fully 69% (n = 1034) of study subjects had 2 or more different telephone numbers during the 18-month study period. The one third of study subjects with a single telephone number throughout the 18-month period were far more likely (P<.001) to receive a hepatitis B vaccination and a WCC visit than were the study subjects who had multiple telephone numbers. Results for other outcomes followed a similar pattern.
An intensive autodialer-based telephone reminder and recall system was only marginally successful in improving immunization rates and WCC visits among adolescents attending 4 urban primary care practices. An initial effect waned after several months. The small effect (4-7 percentage points) appears consistent with the small effect noted in many reminder/recall studies.5,61 The intervention did not appear to be any more or less effective for demographic subgroups stratified by age, geographic residence, insurance status, and race/ethnicity. The major factor limiting effectiveness of reminder/recall was inaccurate telephone numbers. The initial improvements were probably owing to the reminders reaching those families with accurate telephone numbers or those who were going to respond. Continued intervention had no effect.
Patient reminder/recall can improve immunization rates4,5,54 and preventive visit rates62 in children or adults, but several recent studies,47,48 including 1 autodialer-based study63 focusing on urban infants, have failed to find benefit. The most commonly cited reason for the lack of effectiveness is difficulty in successfully contacting the patient population, a concern regarding adolescents who make few regular health care visits. In our study, lack of an accurate telephone number was the major problem. Of 1496 study subjects, 69% had at least 2 distinct telephone numbers over 18 months. The subgroup that had a single telephone number had markedly greater response to the telephone reminders than did the subgroup with 2 or more telephone numbers. Our findings confirm that the effectiveness of the patient reminder/recall is strongly associated with the accuracy of telephone numbers or addresses, which was low among this population.
Unfortunately, practices cannot discern beforehand which families are likely to change their telephone number (or addresses). Practices should update telephone numbers during all patient encounters and obtain alternate contact information such as mobile telephone numbers. Our study also suggests that new vaccines requiring multiple doses (eg, human papillomavirus vaccine) may pose a challenge for vaccinating the urban adolescent population. Given that our intervention had some initial benefit, probably because it reached those families that could be reached, health care professionals serving urban populations may consider short-term reminder/recall interventions instead of prolonged and intensive interventions.
We expect that the use of autodialers or similar technology will increase as computerized appointment scheduling modules are implemented.58 The costs of autodialer-based reminder systems are relatively low if (1) telephone numbers are updated electronically using links with billing systems and (2) the systems are used by hospital or health systems that apply them to populations beyond adolescents.54 As an example, 1 of the study practices has just begun using an autodialer reminder system for scheduled appointments as part of a hospitalwide autodialer reminder system. Costs to the practice are only a few hundred dollars per year because costs are distributed hospitalwide. Thus, even with very small increases in show rates, this reminder system is probably revenue generating.
Our study highlights the importance of checking adolescents' immunization status at all visits and using visits other than WCC visits as opportunities to immunize adolescents. More than half the study group had at least 1 visit with a missed opportunity for vaccination with hepatitis B or Td vaccine.
Our study has several limitations. The major threat to generalizability is the study setting—urban practices in a metropolitan area. Patient characteristics, the accuracy of telephone numbers, baseline immunization coverage, and practice characteristics may differ in other settings. Baseline vaccination rates were within the wide spectrum of rates noted nationally.14,34,64 Demographic characteristics of our population were similar to characteristics of other urban settings. Also, although hepatitis B vaccination among adolescents will become less important (because of infant vaccination), it serves as a model for human papillomavirus vaccination because that schedule will be similar to the hepatitis B schedule.
One threat to internal validity is that we undoubtedly included adolescents who were not current patients of the practices because there was no perfect method to select eligible adolescents. Some adolescents may have received vaccinations elsewhere. The major limitation to the intervention was the inability to determine accurate telephone numbers in spite of the research assistant checking medical records and web-based directories weekly. Coordination of practice-based interventions with Medicaid managed care may improve effectiveness since the requirements to reenroll into Medicaid managed care plans provide opportunities for managed care organizations to update patient contact information.
It is possible that the autodialer was not as effective as a direct telephone call from a person would have been. Studies are lacking comparing autodialer-based reminders with person-to-person reminders. Further, we did not attempt to ascertain vaccinations received outside of the practice; however, in this community, few vaccinations are provided to adolescents in schools or public health clinics.61 Finally, we did not perform cost analyses because we were using a labor-intensive method to update kept appointments and changes in telephone numbers.
We conclude that an intensive primary care practice-based telephone reminder/recall intervention was only marginally effective by slightly improving the receipt of adolescent immunizations visits within these urban primary care practices. The critical factor associated with success of the intervention was the accuracy of telephone numbers, which was low among our population.
An implication for health care professionals is that practices are more likely to have success with patient reminder/recall if they have accurate patient telephone numbers. In these cases, even small practices may consider using reminder systems. For practices within hospital or health systems, practice-level costs of reminder/recall systems may be low because costs would be distributed across an entire system, and large numbers of patients beyond adolescents could benefit. In these settings, an autodialer-based reminder/recall system that has only a small effect should still be considered. However, to achieve high levels of preventive care among urban adolescents, more aggressive strategies (such as outreach13,65 or combinations of interventions) may be needed.
An implication for communities and public health is that in some settings, practice-based patient reminder/recall might be only marginally effective in improving adolescent vaccinations and preventive visits. These interventions may be more effective if combined with updated contact information from managed care organizations, immunization registries, or integrated delivery systems. With emerging new adolescent vaccines, aggressive strategies will be needed to ensure high adolescent vaccination coverage as well as receipt of preventive adolescent services.
Correspondence: Peter G. Szilagyi, MD, MPH, Department of Pediatrics, Strong Memorial Hospital, Box 632, Rochester, NY 14642 (peter_szilagyi@urmc.rochester.edu).
Accepted for Publication: September 6, 2005.
Funding/Support: This work was funded by the Centers for Disease Control and Prevention, Atlanta, Ga, and Cooperative Agreement TS-622 from the Association for Teachers of Preventive Medicine, Washington, DC.
Acknowledgment: We gratefully acknowledge the statistical consultation of Jason Roy, PhD.
1.Centers for Disease Control and Prevention (CDC), Ten great public health achievements—United States, 1900-1999
MMWR Morb Mortal Wkly Rep 1999;48241- 243
PubMedGoogle Scholar 2.Gyorkos
TWTannenbaum
TNAbrahamowicz
M
et al. Evaluation of the effectiveness of immunization delivery methods
Can J Public Health 1994;85(suppl 1)S14- S30
PubMedGoogle Scholar 3.Shea
SDuMouchel
WBahamonde
L A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting
J Am Med Inform Assoc 1996;3399- 409
PubMedGoogle ScholarCrossref 4.Udovic
SLieu
TA Evidence on office-based interventions to improve childhood immunization delivery
Pediatr Ann 1998;27355- 361
PubMedGoogle ScholarCrossref 5.Shefer
ABriss
PRodewald
L
et al. Improving immunization coverage rates: an evidence-based review of the literature
Epidemiol Rev 1999;2196- 142
PubMedGoogle ScholarCrossref 6.Szilagyi
PGBordley
CVann
JC
et al. The effect of patient reminder/recall interventions on immunization rates: a review
JAMA 2000;2841820- 1827
PubMedGoogle ScholarCrossref 7.Centers for Disease Control and Prevention, Vaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults. a report on recommendations from the Task Force on Community Preventive Services
MMWR Morb Mortal Wkly Rep 1999;48(RR-8)1- 15
PubMedGoogle Scholar 8.Centers for Disease Control and Prevention (CDC), National, state, and urban area vaccination levels among children aged 19-35 months—United States, 2002
MMWR Morb Mortal Wkly Rep 2003;52728- 732
PubMedGoogle Scholar 9.Centers for Disease Control and Prevention (CDC), Public health and aging: influenza vaccination coverage among adults aged > or =50 years and pneumococcal vaccination coverage among adults aged > or =65 years—United States, 2002
MMWR Morb Mortal Wkly Rep 2003;52987- 992
PubMedGoogle Scholar 10.Rodewald
LESzilagyi
PGShiuh
T
et al. Is underimmunization a marker for insufficient utilization of preventive and primary care?
Arch Pediatr Adolesc Med 1995;149393- 397
PubMedGoogle ScholarCrossref 11.Fairbrother
GFriedman
SDuMont
KALobach
KS Markers for primary care: missed opportunities to immunize and screen for lead and tuberculosis by private physicians serving large numbers of inner-city Medicaid-eligible children
Pediatrics 1996;97785- 790
PubMedGoogle Scholar 12.Rodewald
LESzilagyi
PGHumiston
SG
et al. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care
Pediatrics 1999;10331- 38
PubMedGoogle ScholarCrossref 14.Centers for Disease Control and Prevention (CDC), Vaccination coverage among adolescents 1 year before the institution of a seventh grade school entry vaccination requirement—San Diego, California, 1998
MMWR Morb Mortal Wkly Rep 2000;49101- 102, 111
PubMedGoogle Scholar 15.Centers for Disease Control and Prevention (CDC), Hepatitis B vaccination—United States, 1982-2002
MMWR Morb Mortal Wkly Rep 2002;51549- 552, 563
PubMedGoogle Scholar 16.Centers for Disease Control and Prevention, Update: recommendations to prevent hepatitis B virus transmission—United States
JAMA 1999;281790
PubMedGoogle ScholarCrossref 18.Bridges
CBHarper
SAFukuda
KUyeki
TMCox
NJSingleton
JAAdvisory Committee on Immunization Practices, Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP)
MMWR Recomm Rep 2003;52(RR-8)1- 34[published correction appears in
MMWR Morb Mortal Wkly Rep. 2003;52:526]
PubMedGoogle Scholar 19.Vitek
CRPascual
FBBaughman
ALMurphy
TV Increase in deaths from pertussis among young infants in the United States in the 1990s
Pediatr Infect Dis J 2003;22628- 634
PubMedGoogle Scholar 21.Edwards
KM Is pertussis a frequent cause of cough in adolescents and adults? should routine pertussis immunization be recommended?
Clin Infect Dis 2001;321698- 1699
PubMedGoogle ScholarCrossref 22. Meningococcal disease and college students: recommendations of the Advisory Committee on Immunization Practices (ACIP)
MMWR Recomm Rep 2000;4913- 20
PubMedGoogle Scholar 23.Lingappa
JRRosenstein
NZell
ERShutt
KASchuchat
APerkins
BA Active Bacterial Core surveillance (ABCs) team: surveillance for meningococcal disease and strategies for use of conjugate meningococcal vaccines in the United States
Vaccine 2001;194566- 4575
PubMedGoogle ScholarCrossref 26.Koutsky
LAAult
KAWheeler
CM
et al. Proof of Principle Study Investigators, A controlled trial of a human papillomavirus type 16 vaccine
N Engl J Med 2002;3471645- 1651
PubMedGoogle ScholarCrossref 27.Kulasingam
SLMyers
ER Potential health and economic impact of adding a human papillomavirus vaccine to screening programs
JAMA 2003;290781- 789
PubMedGoogle ScholarCrossref 28.Skinner
GRTuryk
MEBenson
CA
et al. The efficacy and safety of Skinner herpes simplex vaccine towards modulation of herpes genitalis; report of a prospective double-blind placebo-controlled trial
Med Microbiol Immunol (Berl) 1997;18631- 36
PubMedGoogle ScholarCrossref 30.Szilagyi
PGRodewald
LESavageau
J
et al. Improving influenza vaccination rates in children with asthma: a test of a computerized reminder system and an analysis of factors predicting vaccination compliance
Pediatrics 1992;90871- 875
PubMedGoogle Scholar 34.Gonzalez
IMAverhoff
FMMassoudi
MS
et al. Vaccine Safety Datalink Team, Hepatitis B vaccination among adolescents in 3 large health maintenance organizations
Pediatrics 2002;110929- 934
PubMedGoogle ScholarCrossref 35.Centers for Disease Control and Prevention (CDC), Hepatitis B vaccination among high-risk adolescents and adults—San Diego, California, 1998-2001
MMWR Morb Mortal Wkly Rep 2002;51618- 621
PubMedGoogle Scholar 37.Centers for Disease Control and Prevention, Immunization of adolescents: recommendations of the Advisory Committee on Immunization Practices, the American Academy of Pediatrics, the American Academy of Family Physicians, and the American Medical Association
JAMA 1997;277202- 207
PubMedGoogle ScholarCrossref 38.Schaffer
SJHumiston
SGShone
LPAverhoff
FMSzilagyi
PG Adolescent immunization practices: a national survey of US physicians
Arch Pediatr Adolesc Med 2001;155566- 571
PubMedGoogle ScholarCrossref 39.Gordon
TEZook
EGAverhoff
FMWilliams
WW Consent for adolescent vaccination: issues and current practices
J Sch Health 1997;67259- 264
PubMedGoogle ScholarCrossref 42.Klein
JDWilson
KMMcNulty
MKapphahn
CCollins
KS Access to medical care for adolescents: results from the 1997 Commonwealth Fund Survey of the Health of Adolescent Girls
J Adolesc Health 1999;25120- 130
PubMedGoogle ScholarCrossref 43.Yu
SMBellamy
HASchwalberg
RHDrum
MA Factors associated with use of preventive dental and health services among US adolescents
J Adolesc Health 2001;29395- 405
PubMedGoogle ScholarCrossref 44.Zimmer-Gembeck
MJAlexander
TNystrom
RJ Adolescents report their need for and use of health care services
J Adolesc Health 1997;21388- 399
PubMedGoogle ScholarCrossref 45.Marcell
AVKlein
JDFischer
IAllan
MJKokotailo
PK Male adolescent use of health care services: where are the boys?
J Adolesc Health 2002;3035- 43
PubMedGoogle ScholarCrossref 46.Unti
LCoyle
KWoodruff
BADemonstration Project Staff, A Review of Adolescent School-Based Hepatitis B Vaccination Projects. Washington, DC US Department of Health and Human Services Public Health Service1996;
47.Briss
PARodewald
LEHinman
AR
et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults: the Task Force on Community Preventive Services
Am J Prev Med 2000;1897- 140
PubMedGoogle ScholarCrossref 48.Kempe
ALowery
NEPearson
KA
et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic
J Pediatr 2001;139630- 635
PubMedGoogle ScholarCrossref 49.Daley
MFSteiner
JFKempe
A
et al. Quality improvement in immunization delivery following an unsuccessful immunization recall
Ambul Pediatr 2004;4217- 223
PubMedGoogle ScholarCrossref 50.Davis
MMSzilagyi
PG Can quality improvement reach into pockets of need for childhood immunizations?
Ambul Pediatr 2004;4224- 225
PubMedGoogle ScholarCrossref 51.Harper
PMadlon-Kay
DJ Adolescent measles vaccination: response rates to mailings addressed to patients vs parents
Arch Fam Med 1994;3619- 622
PubMedGoogle ScholarCrossref 54.Dini
EFLinkins
RWChaney
M Effectiveness of computer-generated telephone messages in increasing clinic visits
Arch Pediatr Adolesc Med 1995;149902- 905
PubMedGoogle ScholarCrossref 55.Linkins
RWDini
EFWatson
GPatriarca
PA A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children
Arch Pediatr Adolesc Med 1994;148908- 914
PubMedGoogle ScholarCrossref 56.Lieu
TABlack
SBRay
P
et al. Computer-generated recall letters for underimmunized children: how cost-effective?
Pediatr Infect Dis J 1997;1628- 33
PubMedGoogle ScholarCrossref 58.Newacheck
PWHung
YYPark
MJBrindis
CDIrwin
CE
Jr Disparities in adolescent health and health care: does socioeconomic status matter?
Health Serv Res 2003;381235- 1252
PubMedGoogle ScholarCrossref 60.Joffe
GPRodewald
LEHerbert
TBarth
RSzilagyi
PG Doctor-switching and utilization of health care by children on fee-for-service Medicaid
J Urban Health 1999;76322- 334
PubMedGoogle ScholarCrossref 61.Jacobson Vann
JCSzilagyi
PG Patient reminder and patient recall systems to improve immunization rates Oxford, England Cochrane Database of Systematic Reviews.Updated May 30, 2005
62.Campbell
JRSzilagyi
PGRodewald
LE
et al. Intent to immunize among pediatric and family medicine residents
Arch Pediatr Adolesc Med 1994;148926- 929
PubMedGoogle ScholarCrossref 64.Fishbein
DB Immunization of adolescents: an update Paper presented at: Centers for Disease Control and Prevention National Vaccine Program Office; June 2, 2005; Washington, DC
65.Wood
DHalfon
NDonald-Sherbourne
C
et al. Increasing immunization rates among inner-city, African American children: a randomized trial of case management
JAMA 1998;27929- 34
PubMedGoogle ScholarCrossref