[Skip to Navigation]
Sign In
March 1998

Reducing Missed Opportunities to Vaccinate During Child Health Visits: How Effective Are Parent Education and Case Management?

Author Affiliations

From RAND, Santa Monica, Calif (Drs Wood, Schuster, Donald-Sherbourne, Duan, Mazel, and Halfon); Shriners Hospitals for Children, Tampa, Fla (Dr Wood); Department of Pediatrics, School of Medicine, University of California, Los Angeles (Dr Schuster); and Department of Community Health, School of Public Health, University of California, Los Angeles (Dr Halfon).

Arch Pediatr Adolesc Med. 1998;152(3):238-243. doi:10.1001/archpedi.152.3.238

Background  At child health visits, immunizations that are due are frequently not given. Increased parent understanding of and demand for immunizations may influence providers to not miss these opportunities.

Objective  To assess, as part of a larger study of effectiveness of parent education and case management (CM) in raising immunization rates, the intervention's effectiveness at reducing missed opportunities to vaccinate during child health visits.

Methods  A representative sample of African American newborns and their families from south central Los Angeles, Calif, were randomly assigned to a control or a CM group and observed during the first year of life. Case managers visited and telephoned parents, educating them on the benefits and safety of immunizations, and encouraging them to request immunizations from providers. When the children were at least 1 year of age, parents were interviewed and provider records were abstracted.

Results  Complete records were abstracted for 126 controls and 129 CM group children. For these children, 1092 visits were documented where immunizations were due. Missed opportunities to vaccinate occurred at more than 50% of visits. Case management was associated with a modest reduction in the percentage of visits with missed opportunities in the bivariate analysis but not after adjustment for other covariates. In a logistic regression model, missed opportunities were more frequent at visits with private than public physicians and at acute illness than well-child visits. Missed opportunities were less frequent among children with a history of at least 1 cancelled appointment, and for visits of children with mothers who smoked.

Conclusions  Missed opportunities were minimally influenced by a home visitation and parent education program. They are primarily determined by issues under the control of the provider. Family- and child-related characteristics, however, do influence the likelihood of a missed opportunity occurring independent of provider factors.

IMMUNIZATION RATES for certain groups of preschool children remain below the national target of 90% up to date by 2 years of age.1 A number of studies have documented that, during almost half of child health visits, immunizations are due but not given. These missed opportunities to vaccinate (MOs) significantly contribute to the problem of underimmunization of young children.2-7 Studies have associated MOs with provider knowledge of the appropriate contraindications to immunization and the immunization schedule.8-11 A number of studies have shown that a provider's compliance with preventive health screenings can be improved through interventions to educate and empower patients.12-14 Similarly, it has been demonstrated that parental perception of and demand for immunizations can influence the child's receipt of immunizations.15-18

Misconceptions held by both professionals and lay people (eg, that immunizations are dangerous or ineffective when given to a mildly ill child) inhibit the delivery of immunizations. In a previous study, we found that the majority of African American mothers believed that it was unsafe to give immunizations to a mildly ill child.19,20 More accurate parental knowledge of immunization safety, effectiveness, and timing may increase demand for timely immunizations.

Home visitation has been shown to be effective at providing parents with accurate information and supporting them in their effort to seek health services.21-23 In a previous article, we showed that case management (CM) intervention was effective at raising immunization levels among inner-city, African American infants.24 We hypothesized that a CM program could increase parental demand for immunizations during child health visits, thereby reducing MOs. In this article, we focus our analysis on the effect of the CM intervention on MOs during child health visits.

Subjects and methods


We enrolled a representative sample of 419 African American infants born into a 10–ZIP code area in inner-city Los Angeles, Calif, during a 3-month period (February through early May) in 1994 (more detail on the sample selection process is available elsewhere24,25). They were randomly assigned at entry into a CM or a control group, and they received follow-up for more than 1 year. We successfully followed up and interviewed 185 members of the intervention group and 180 members of the control group. During the interview, we asked them to identify all providers the child had seen in the first year of life. We were able to obtain and abstract the complete medical records from all reported providers for 255 children (70%)—129 intervention children and 126 control children. We compared the 255 families and children used in this analysis with the 110 for whom we did not have complete provider records. They were comparable on all demographic and health utilization characteristics except for health insurance (10% of the abstracted group vs 5.4% of others had private insurance), knowledge of contraindications scale (P=.005), and history of being breast-fed (P=.02). These factors are controlled for when the main effect of group (intervention effect) on MOs is compared in the multivariate analysis.


Both groups were given health passports produced by the State of California, which contain information on the recommended visits for preventive health care and the childhood immunization schedule approved by the Centers for Disease Control and Prevention (CDC).26 Case managers conducted an in-depth assessment in the home of the client generally by 6 weeks of age. Subsequent visits were scheduled for 2 weeks before the next preventive health visit when immunizations were due. Case managers also followed up with clients by telephone and mail after scheduled appointments. The mean (±SD) numbers of contacts of case managers with clients were 4.0±1.9 for home visits, 7.5±3.9 for telephone contacts, and 4.4±2.5 for mailings.

At each visit, the case manager documented that the client understood which immunizations were due at the next visit, and had a preventive health visit appointment at the appropriate interval. If barriers to care were expressed, the case manager helped the mother overcome them. The case managers specifically sought to assure parents that it was safe to give immunizations if the child had a cold, mild ear infection, or other mild illness. The case manager also encouraged the mothers to be proactive and request immunizations from their providers.

Data collection

At the end of the intervention, we interviewed the mothers and collected demographic information (eg, family structure, birth order, maternal education, maternal smoking history), information on health care system utilization (type of health insurance, months of health insurance in the child's first year of life, and type of regular provider), and parental knowledge of the immunization schedule (2-item scale) and of contraindications to vaccination (3-item scale). Each provider's chart was coded for the presence or absence of a separate immunization record. For children with multiple providers, visits were merged. For each visit, the following information was abstracted: whether it was a preventive health or an illness visit, date of visit, all diagnoses recorded, temperature if recorded, and immunizations administered.

Definition of mo

As defined by the CDC, an MO occurs at a visit at which "immunizations were due but not given and no contraindications were present."27 According to the CDC definition, a contraindication to vaccination can be justified on the basis of either (1) a vaccine-specific contraindication (eg, temperature higher than 40.6°C with a previous diphtheria, tetanus, and pertussis vaccination or allergy to eggs for measles, mumps, and rubella vaccination) or (2) "the child being too sick to be immunized at that visit." No vaccine-specific contraindications were recorded during the 1092 visits in this sample at which immunizations were due. To assess whether the diagnoses recorded provided evidence that the child was "too sick to immunize," each diagnosis was reviewed and rated by a panel of 3 pediatricians. The interrater reliability (κ) for the pediatrician ratings was 93%. In cases where the raters did not agree, the majority rating was assigned. The following diagnoses were deemed sufficient to conclude that the child was too sick to immunize: acute chickenpox, acute croup, pneumonia, possible pneumonia, meningitis, viral meningitis, possible bacteremia, sepsis, and possible sepsis. In addition, the pediatric panel agreed that a recorded temperature of 38.3°C or greater should qualify as a valid contraindication under the "too sick to immunize" clause. By combining the data on dates of visits, dates of immunizations, and contraindications, we could determine for each visit which, if any, immunizations were due and whether an MO occurred.

Data analysis

In the abstracted records, a total of 1938 visits were documented in the providers' charts for the study children as having occurred through the first 12 months of life. Of the 1938 recorded visits, 846 were eliminated from the analysis because immunizations had not been due or contraindications to vaccination had been present (eg, temperature >38.3°C or a diagnosis that indicated the child was too sick to immunize). The remaining 1092 visits were assigned the family, child, provider/chart, or visit characteristics with which they were associated. Statistically significant bivariate associations between independent variables (health visit, child/family, and provider/chart characteristics) and the dependent variable (the occurrence of an MO) were tested within each intervention group. We tested the association between the rate of MOs and continuous independent variables by means of the test of significance for correlation coefficients. For categorical independent variables, we tested the association with the occurrence of an MO by means of the χ2 statistic. All bivariate associations with P<.10 were included in a visit-based logistic regression model predicting visits with MOs vs visits with no MOs (all immunizations due were given). To achieve a more parsimonious model, we used a stepwise, backward elimination approach to remove nonsignificant variables. To test the hypotheses central to this analysis, we forced the CM and immunization knowledge variables into the model regardless of their level of significance. Furthermore, child health insurance and history of breast-feeding were also included in the model to adjust for potential biases in the sample selection. We adjusted for the clustering of visits within provider by means of a Huber logistic regression model.28


The 2 experimental groups (CM and control) were comparable at baseline on all demographic and health care system utilization characteristics in Table 1. The CM intervention was associated with a trend toward a lower rate of MOs (47.7% vs 53.3%; P=.07) (Table 2). Comparing the MO rates between the control and CM group across independent variable categories, the control group had MO rates higher by approximately 5 to 6 percentage points in almost every category (range, 3.4-9.3 percentage points). For within–intervention group comparisons, MOs were (1) more frequent for visits at private physicians' offices and health maintenance organization offices than for visits at public clinics (P<.001); (2) less frequent for children of women who smoked (P=.004); (3) more frequent for children who had been breast-fed (P=.001); and (4) more frequent for visits occurring from 6 to 12 months of age than 0 to 6 months of age (P<.001). There was a trend for lower MO rates among those continuously receiving Medicaid insurance vs the other insurance groups. There was no significant within-group relationship between MOs and maternal education or months of insurance coverage. For the control group, greater knowledge of the immunization schedule was significantly associated with a higher rate of MOs (P<.01). For children in the CM group, greater knowledge of immunization contraindications was associated with a lower MO rate (P<.01).

Table 1. 
Characteristics of the Case Management and Control Groups*
Characteristics of the Case Management and Control Groups*
Table 2. 
Association Between Missed Opportunities and Child/Family Demographic and Health Care System Utilization Characteristics*
Association Between Missed Opportunities and Child/Family Demographic and Health Care System Utilization Characteristics*

Missed opportunities were much more common at sick visits than well visits (P<.001) (Table 3). For the control group only, MOs were more frequent when the chart lacked an immunization record (P<.05). Within the control group, the rate of MOs was similar for visits of children with and without a history of cancelled appointments. However, within the CM group, MO rates were lower by 14.4 percentage points for children with a history of cancelled appointments compared with those with no history of cancelled appointments (P<.01). In both the control and CM groups, MOs tended to be more common during visits of children who reported having had more than 1 provider during the year (P=.09). Missed opportunities were more common at visits with any of the illness diagnoses recorded compared with visits with no illness diagnosis recorded. With the exception of pharyngitis, MO rates at visits with illness diagnoses were similar between the control and MO groups. The total number of visits in the illness categories exceeds 1092 because multiple diagnoses were recorded at some visits.

Table 3. 
Percentage of Visits With Missed Opportunity by Visit Characteristics*
Percentage of Visits With Missed Opportunity by Visit Characteristics*

The logistic regression model is presented in Table 4. The model explains more than one third of the variance in the occurrence of MOs (pseudo R2=0.37). After adjusting for the other covariates, neither the intervention group nor parental knowledge variables had significant predictive effect on the likelihood of an MO occurring. Provider, visit, and child/family characteristics were significantly associated with the likelihood of an MO occurring. Missed opportunities were more likely at visits (1) by children 7 to 12 months of age (vs 0 to 6 months of age); (2) by children with older mothers; (3) by children who were breast-fed as infants; (4) by children with more continuous health insurance coverage; (5) in private physicians' offices or health maintenance organizations (vs public clinics); and (6) in physicians' offices that did not maintain separate immunization records. Missed opportunities were less common at visits of children whose mothers smoked cigarettes or if the child had a history of cancelled appointments.

Table 4. 
Logistic Regression Predicting Occurrence of a Missed Opportunity Within a Health Visit
Logistic Regression Predicting Occurrence of a Missed Opportunity Within a Health Visit

Characteristics of the visit itself were strongly associated with the likelihood of an MO. Missed opportunities were much less likely to occur at preventive health visits than illness visits, even after adjusting for the presence of 1 or more illness diagnoses. Missed opportunities were more likely to occur when illness diagnoses of otitis media or "other" were listed. The illness diagnoses of pharyngitis, diarrhea, and cold were not independently associated with MOs and fell out of the model.


Our study is one of the first to examine predictors of MOs taking into account detailed parent/child characteristics from a parent interview as well as provider/visit characteristics derived from chart abstractions. We found that parent-directed education and CM with an emphasis on immunizations and preventive health visits, after adjustment for other factors, did not reduce MOs during child health visits. While the intervention did not have the desired effect, our study does provide important new information on issues relating to MOs to vaccinate in young, inner-city children.

At approximately 50% of the 1092 visits we analyzed, immunization opportunities were missed. This is particularly disturbing given that almost half of these children were behind in their immunizations at 1 year of age. Moreover, this inner-city area was a focus of the 1989 to 1990 measles epidemic, which resulted in considerable media attention and additional immunization outreach efforts targeted at parents and providers in our study area. Despite this context, the MOs at child health visits remained high, consistent with MO rates in other studies.2,3,7,8

Low immunization rates have been attributed, in large part, to MOs. However, quantifying the impact of MOs on immunization completion rates is not straightforward. The relationship is complicated by the number and timing of health visits in relation to the immunization schedule. As we described elsewhere,24 while this CM did not lower the rate of MOs, it did raise immunization completion rates by 13.2 percentage points. Some of the children who received CM failed to receive more than 3 well-child visits in the first year. Nonetheless, this intervention subgroup experienced a remarkable 28 percentage point rise in immunization completion (from 30% up to date in the control group to 58% up to date in the CM group). In this same group (for whom we have physician records), the CM intervention only modestly reduced the frequency of MOs. Our interpretation of these findings is that, for children with few well-child visits, MOs may have a much greater negative effect on immunization completion than for children with ample numbers of visits to physicians. While the latter children may even experience more MOs, additional visits can overcome the effect of MOs and allow them to meet the immunization completion milestones. For children with few visits, 1 MO may be adequate to cause them to fail to meet the immunization up-to-date milestone.

The most powerful predictor of an MO is the visit type; MOs were much more common at acute illness visits than at preventive health visits. These findings are consistent with previous research showing that MOs are, in large part, caused by the provider misconception that minor illnesses are contraindications to vaccination.8,9,29,30 In our model, the illness visit had a strong independent association with MOs even after accounting for the effect of the illness diagnoses listed. A possible explanation for these findings may be that most physician offices reserve well-child visits for the labor-intensive task of administering immunizations and do not give immunizations at sick visits regardless of the degree or nature of the illness.

We also found that private providers and health maintenance organization providers were more likely to miss opportunities to vaccinate than were public health providers. A number of factors may explain this finding. First, private providers in inner-city areas may concentrate more on illness care than preventive care, which is more time consuming and less well reimbursed.31 Second, private and health maintenance organization providers saw their patients more frequently than the public providers in this sample of patients, which may have led them to be more relaxed about delaying immunizations to future well-child visits, thereby missing opportunities to vaccinate.

The CDC Standards for Pediatric Immunization Practice, published in 1993, strongly promote screening for and administering immunizations at both sick and well visits.27 The standards also clearly delineate the true contraindications for each vaccine, which are rare.26 Despite having been published more than 2 years before our study, the standards were not widely adopted by the providers serving children in inner-city Los Angeles. The CDC immunization standards need to be disseminated more widely, especially in high-risk, poor areas where immunization levels are low. In addition, for high-risk areas, public health departments should identify physicians for education, audits of their records, and constructive feedback on their performance.32-34

A number of child and family characteristics in our model were associated with either a greater or lower likelihood of MOs occurring. Lower rates of MOs (ie, higher compliance by physicians) were associated with the negative characteristics of maternal smoking and a history of broken appointments. Higher rates of MOs (ie, lower compliance by physicians) were associated with the relatively positive characteristics of insurance continuity and history of breast-feeding. We hypothesize that these family characteristics are perceived by providers as indicators of risk and that providers adjust their immunization decision making in response to these perceptions. That is, provider perceptions of increased risk may prompt more aggressive ordering of immunizations, hence the lower MO rate in children of smoking mothers. For children and families perceived to be at low risk, the provider may be more willing to defer immunizations, confident that the child will return for immunizations at a later well-child visit.

The limitations of this study should be appreciated. First, we were able to obtain and abstract the complete records from all reported providers for 255 children, which is 69.7% of the 366 successfully followed up in the randomized controlled trial. However, it represents only 61% of the 419 children originally enrolled in the trial. We compared important demographic characteristics between both the 419 originally enrolled and 366 followed up in the randomized controlled trial with the 255 families/children used in this analysis; they were comparable on all demographic and health utilization characteristics except for health insurance, knowledge of contraindications scale, and history of being breast-fed. These factors are controlled for in the multivariate analysis. However, the possibility that attrition of 39% of the sample from our analyses may have biased our results cannot be completely ruled out. Second, while we were largely successful at abstracting records from all providers reported by the parents, the potential for bias resulting from inadequate provider record keeping remains. Because we were equally successful at provider record abstraction across the 2 intervention groups, we doubt that this bias affected our results. Finally, this study was conducted on a sample of inner-city, predominantly poor, African American families. The results, therefore, may not be generalizable to other populations of children.

In summary, we found that family CM was not effective at reducing MOs at child health visits. In this study, MOs were found to be associated with a number of provider-related factors (ie, presence of false contraindications, record-keeping quality) that are under the providers' control and should be addressed. Family- and child-related factors (eg, maternal smoking, cancelled appointments), however, do influence the provision of immunizations, indicating that immunization provision, like most medical decisions, is an outcome of a complex interaction between the provider, child, and family.

Accepted for publication October 30, 1997.

This study was supported by contract 200-91-0942 from the Centers for Disease Control and Prevention, Atlanta, Ga.

We thank the Los Angeles Research and Education Institute–Women-Infant-Children staff and the study case managers for their enthusiastic and dedicated efforts to improve the care of children and families in inner-city Los Angeles.

Editor's Note: So much for the effectiveness of case management—at least as it relates to immunization. Any other ideas?—Catherine D. DeAngelis, MD

Reprints: David Wood, MD, MPH, Shriners Hospitals, 2900 Rocky Point Dr, Tampa, FL 33607-1460 (e-mail: dwood@gte.net).

Centers for Disease Control and Prevention, Vaccination coverage of 2-year-old children—United States, 1995.  MMWR Morb Mortal Wkly Rep. 1996;43705- 709Google Scholar
Grabowsky  MOrenstein  WA The critical role of provider practices in undervaccination.  Pediatrics. 1996;97729- 732Google Scholar
Wood  DLPereyra  MHalfon  N  et al.  Vaccination levels in public health centers: missed opportunities and other contributing factors.  Am J Public Health. 1995;85850- 853Google ScholarCrossref
Mustin  HDHolt  VLConnell  FA Adequacy of well-child care and immunization in US infants born in 1988.  JAMA. 1994;2721111- 1115Google ScholarCrossref
Williams  ITMilton  JDFarrell  JBGraham  NMH Interaction of socioeconomic status and provider practices as predictors of immunization coverage in Virginia children.  Pediatrics. 1995;96439- 446Google Scholar
Centers for Disease Control and Prevention, Impact of missed opportunities to vaccinate preschool-aged children on vaccination coverage levels—selected US sites, 1991-1992.  MMWR Morb Mortal Wkly Rep. 1994;43709- 711717- 718Google Scholar
Guyer  BHugart  NHold  E  et al.  Immunization coverage and its relationship to preventive health care visits among inner-city children in Baltimore.  Pediatrics. 1994;9453- 58Google Scholar
Szilagyi  PGRodewald  LEHumiston  SG  et al.  Missed opportunities for childhood vaccinations in office practices and the effect on vaccination status.  Pediatrics. 1993;911- 7Google Scholar
Wood  DLHalfon  NHamlin  JGrabowsky  M Knowledge of child immunization schedule and contraindications to vaccinate by private and public providers in Los Angeles.  J Pediatr Infect Dis. 1996;15140- 145Google ScholarCrossref
Hughart  NGuyer  BStanton  B  et al.  Do provider practices conform to the new pediatric immunization standards?  Arch Pediatr Adolesc Med. 1994;148930- 935Google ScholarCrossref
Lieu  TABlack  SBSorel  MERay  PShinefield  HR Would better adherence to guidelines improve childhood immunization rates?  Pediatrics. 1996;981062- 1068Google Scholar
Davis  DAThomson  MAOxman  ADHaynes  RB Changing physician performance: a systematic review of the effect of continuing medical education strategies.  JAMA. 1995;274700- 705Google ScholarCrossref
McPhee  SJBird  JAFordham  DRodnick  JEOsborn  EH Promoting cancer prevention activities by primary care physicians.  JAMA. 1991;266538- 544Google ScholarCrossref
Rosser  WWMcDowell  INewell  C Use of reminders for preventive procedures in family medicine.  Can Med Assoc J. 1991;145807- 814Google Scholar
Bates  ASFitzgerald  JFDittus  RSWolinsky  FD Risk factors for underimmunization in poor urban infants.  JAMA. 1994;2721105- 1110Google ScholarCrossref
Szilagyi  PGRodewald  LESavageau  JYoos  LDoane  C Improving influenza vaccination rates in children with asthma: a test of a computerized reminder system and an analysis of factors predicting vaccination compliance.  Pediatrics. 1992;90871- 875Google Scholar
Bennett  PSmith  C Parents' attitudinal and social influences on childhood vaccination.  Health Educ Res. 1992;7341- 348Google ScholarCrossref
Ball  TMSerwint  JR Missed opportunities for vaccination and the delivery of preventive care.  Arch Pediatr Adolesc Med. 1996;150858- 861Google ScholarCrossref
Wood  DLHalfon  NSherbourne  C  et al.  Increasing Immunizations for African American and Latino Preschool Children in the Inner City of Los Angeles.  Atlanta, Ga Centers for Disease Control and Prevention1993;Contract 200-91-0942
Wood  DLSherbourne  CDHalfon  N  et al.  Factors related to immunization status among Latino and African American inner city preschoolers.  Pediatrics. 1995;96295- 301Google Scholar
Olds  DLKitzman  H Review of research on home visiting for pregnant women and parents of young children.  Future Child. 1993;353- 92Google ScholarCrossref
Selby-Harrington  MSorenson  JRQuade  DStearns  SCTesh  ASDonat  PLN Increasing Medicaid child health screenings: the effectiveness of mailed pamphlets, phone calls and home visits.  Am J Public Health. 1995;851412- 1417Google ScholarCrossref
Colombo  TJFreeborn  KDMullooly  JPBurnham  VR The effect of outreach workers' educational efforts on disadvantaged preschool children's use of preventive services.  Am J Public Health. 1979;69465- 468Google ScholarCrossref
Wood  DLHalfon  NDonald-Sherbourne  C  et al.  A randomized trial of case management to raise immunization rates among African American children in inner-city Los Angeles.  JAMA. 1998;27929- 34Google ScholarCrossref
Wood  DLHalfon  NSherbourne  C  et al.  A Randomized Trial of Case Management to Raise Immunization Rates Among African American Children in Inner-City Los Angeles.  Atlanta, Ga Centers for Disease Control and Prevention1996;Contract 200-91-0942
Centers for Disease Control and Prevention, General recommendations on immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP).  MMWR Morb Mortal Wkly Rep. 1994;43 ((RR-1)) 1- 38Google Scholar
Centers for Disease Control and Prevention, Standards for pediatric immunization practices.  JAMA. 1993;2691817- 1822Google ScholarCrossref
Not Available, STATA 4.0.  College Station, Tex Stata Corp1994/1995;
Hutchins  SSJansen  HAFMRobertson  SE Missed Opportunities for Immunization: The Magnitude, Reasons, Strategies and Recommendations.  Geneva, Switzerland Expanded Programme on Immunization, World Health Organization October1991;
Farizo  KMStehr-Green  TAMarkowitz  LEPatriarca  PA Vaccination levels and missed opportunities for measles vaccination: a record audit in a public pediatric clinic.  Pediatrics. 1992;89589- 592Google Scholar
Fairbrother  GFriedman  SDuMont  KALobach  KS Markers for primary care: missed opportunities to immunize and screen for lead and tuberculosis by private physicians serving large numbers of inner-city Medicaid-eligible children.  Pediatrics. 1996;97785- 790Google Scholar
Batalden  PBStoltz  PK A framework for the continual improvement of health care.  Jt Comm J Qual Improv. 1993;19424- 446Google Scholar
Kibbe  DCKaluzny  ADMcLaughlin  CP Integrating guidelines with continuous quality improvement: doing the right thing the right way to achieve the right goals.  Jt Comm J Qual Improv. 1994;20181- 191Google Scholar
Kanouse  DEKallich  JKahan  JP Dissemination of practice-relevant information to health care providers and payers.  Presented at the National Agenda Setting Conference in Outcomes and Effectiveness Research April 15, 1991 Washington, DC