Thompson JW, Ryan KW, Pinidiya SD, Bost JE. Quality of Care for Children in Commercial and Medicaid Managed Care. JAMA. 2003;290(11):1486-1493. doi:10.1001/jama.290.11.1486
Author Affiliations: Department of Pediatrics, College of Medicine, University of Arkansas for Medical Sciences (Dr Thompson), Arkansas Center for Health Improvement (Messrs Ryan and Pinidiya and Dr Thompson), Department of Biostatistics, College of Public Health, University of Arkansas for Medical Sciences (Dr Bost), Little Rock.
Context Many states have turned to commercial health plans to serve Medicaid
beneficiaries and to achieve cost-containment goals. Assumptions that the
quality of care provided to Medicaid beneficiaries through these programs
is acceptable have not been tested.
Objective To compare the quality of care provided to children and adolescents
in commercial and Medicaid managed care in the United States.
Design, Setting, and Population Using 1999 data collected through the Health Plan Employer Data and
Information Set, we examined reported quality-of-care indicators for children
and adolescents. Results from 423 commercial and 169 Medicaid plans were compared.
Matched pairs analyses were performed using data from each of the 81 companies
serving both populations to control for corporate differences. Correlation
coefficients and regression procedures were used to examine observed variations
in health plan performance.
Main Outcome Measures Quality indicators including prenatal care, childhood immunizations,
well-child visits, adolescent immunizations, and myringotomy and tonsillectomy
Results Using standard indicators of clinical performance, children and adolescents
enrolled in Medicaid received worse care compared with their commercial counterparts.
For most of the 81 health plans serving both populations, Medicaid enrollees
had statistically significantly (P<.001) lower
rates than commercial plans for clinical quality indicators (eg, childhood
immunization rates of 69% vs 54%); for clinical access indicators (eg, well-child
visits in the first 15 months of life, 53% vs 31%); and for common procedures
(eg, myringotomies for children aged 0-4 years, 35 vs 2 per 1000 members).
Conversely, some plans demonstrated equal and high-quality care for both populations.
Regression models failed to identify consistent plan characteristics that
explained the observed differences in quality of care.
Conclusions Most commercial health plans do not deliver high-quality care on a number
of performance indicators for children enrolled in Medicaid. Policy makers
and the public need plan-specific quality information to inform purchasing
Over the past 3 decades, commercial managed care entities have become
the primary source of health insurance for privately insured children.1 State Medicaid programs also have turned to managed
care to achieve cost-containment goals and expand services2,3 through
federal waivers4,5 using commercial
entities under contractual arrangements.6,7 Because
states recently expanded insurance coverage for children through State Children's
Health Insurance Programs,8 they have enrolled
previously uninsured children in commercial delivery systems.9,10
Although many policy makers herald the efficiencies and abilities of
the commercial sector to provide high-quality health care,11,12 concerns
about quality of care (QOC) have led to national efforts to measure and report
information on QOC provided by managed care organizations (MCOs).13- 19 Of
particular interest has been the QOC provided to vulnerable populations, including
children and adolescents.20- 23
The managed care industry has developed a set of reporting requirements
for quality assessment—the Health Plan Employer Data and Information
Set (HEDIS).24,25 Maintained by
the National Committee for Quality Assurance (NCQA),25 HEDIS
contains specified measures and data-collection procedures to provide comparative
data on health care quality.26 Quality of care
information has been collected by NCQA from commercial MCOs since 1995,27 and from Medicaid and Medicare MCOs since 1997.28- 30
In its recent report to Congress, the Institute of Medicine called for
an examination of health care quality across all governmental programs using
such standardized performance indicators.31 The
National Report on Healthcare Quality,32 required
annually by Congress through the 1999 Healthcare Research and Quality Act,33 with the first report scheduled to be issued this
year, will present an opportunity to examine and track improvements in QOC.
We undertook this study to evaluate what existing health plan performance
assessments reveal about the QOC provided to children and adolescents in the
United States. Of particular interest was the health care quality provided
to children and adolescents enrolled in MCOs through the state and federally
funded Medicaid programs. The principal question of interest was whether the
QOC provided to children in Medicaid MCOs is equivalent to the QOC provided
to children in commercial MCOs.
Health plans report their self-assessed and audited HEDIS data to the
NCQA using electronic submission tools and attest to the accuracy of the final
submission. These data are maintained in NCQA's Quality Compass database.25,26 Clinical performance data were reported
by each plan for the 12-month period preceding the submission year.
We examined health plan performance data for selected child and adolescent
health indicators for 1999 reported in 2000 by both commercial and Medicaid
MCO plans. Many MCOs offer both a Medicaid product and a commercial product
and submit HEDIS data separately for both.
From the approximately 80 QOC indicators contained in HEDIS, we selected
all indicators relevant and unique to child or adolescent health care. These
included 3 clinical quality, 3 access, and 4 procedure indicators. The clinical
quality indicators were early initiation of prenatal care, childhood immunization
combination rate (4 diphtheria-tetanus-pertussis, 3 polio, 1 measles-mumps-rubella,
>1 Haemophilus influenzae type B, and 2 hepatitis
B vaccinations before 2 years of age), and adolescent immunization for measles-mumps-rubella.
The access indicators were well-child visits in the first 15 months of life,
annual well-child visits 3 through 6 years of age, and adolescent well visits.
The procedure indicators were number of myringotomies per 1000 members aged
0 to 4 years, number of myringotomies per 1000 members aged 5 to 19 years,
number of tonsillectomies per 1000 members aged 0 to 9 years, and number of
tonsillectomies per 1000 members aged 10 to 19 years. Details on the numerator
and denominator for each HEDIS measure are summarized in Table 1.
Many of our selected performance indicators corresponded to recommendations
and goals of the US Preventive Services Task Force and Healthy People 2000.34 While high-quality performance is represented by
higher levels of achievement for clinical and access indicators, higher rates
of use of specific procedures may represent better QOC or overutilization.
Thus variation in the utilization rates of common procedures observed between
plans can be used to compare performance, but conclusions regarding QOC cannot
The NCQA publishes specifications and acceptable methodologies for data
collection to ensure standardized reporting. These include defining usable
data sources, systematic sampling strategies, and verifiable data collection
procedures. Most clinical indicators of quality use information from administrative
data with supplementation by medical record abstraction for plans with incomplete
information or for indicators requiring more specific information (eg, childhood
immunizations). For both Medicaid and commercial plans, administrative data
were supplemented with record review in a majority of plans for the clinical
quality indicators (ie, 87% of commercial and 78% of Medicaid for adolescent
immunization) and in approximately one third for the access indicators. Through
such efforts, data quality for clinical quality and access indicators is optimized
while statistical power is maintained. Point estimates are generated to compare
performance between the 2 plans and provide 80% power to detect a difference
within ± 5%; usually a sample of 411 enrolled and eligible children
is used with separate samples required for each Medicaid and commercial plan.
Measures of procedure rates were based on administrative data for all children
and have no associated sample error.
Results for each QOC indicator were calculated separately for Medicaid
and commercially enrolled children in each health plan. The national average
for plan performance was calculated as the unweighted plan average for each
indicator. Standard deviations and ranges are provided as measures of performance
distribution across plans. After descriptive results for all reporting plans
were determined, analyses were restricted to those plans reporting both commercial
and Medicaid results for children. This controlled for potential variations
between plans on corporate philosophy, structures, management strategies,
and types of delivery networks.
For each indicator, performance results for commercial and Medicaid
enrollees within each plan were calculated. Then Medicaid rates were subtracted
from their paired commercial rates. For plans that achieved equivalent care
for their Medicaid and commercial beneficiaries, no difference was observed;
for those that achieved higher-quality scores for the commercial beneficiaries,
a positive score was observed; for those in which Medicaid quality exceeded
commercial quality, a negative score was generated. Paired t tests were conducted to see if the differences were statistically
different from zero.
To determine whether the relationships among measures were consistent
for observed differences between commercial and Medicaid enrollees, Pearson
product moment correlation coefficients were calculated for the commercial
rates, Medicaid rates, and the difference scores.
Finally, to identify potential explanatory variables, we modeled the
adjusted log-transformed difference scores (to achieve a more normal distribution)
using linear regression against the following MCO characteristics that have
been shown to be associated with performance: (1) whether or not the commercial
plan publicly reported its data, (2) years in business for the commercial
product line, (3) years in business for the Medicaid product line, (4) number
of commercial enrollees, (5) number of Medicaid enrollees, (6) corporate profit
(tax) status, (7) whether the commercial plan's results were in the top quartile,
(8) whether the Medicaid plan's results were in the top quartile, (9) whether
both commercial and Medicaid results were in the top quartile of reported
results, and (10) location in 4 US regions (northeast includes Department
of Health and Human Services regions 1-3; southeast, regions 4, 6, and 7;
midwest, region 5; and west, regions 8-10). Analyses were performed using
In 1999, 423 commercial and 169 Medicaid MCOs submitted quality performance
data; 81 reported information on both groups of enrollees. Compared with existing
commercial MCO plans in operation that year, approximately 75% reported and
were similar in model type, geographic location, and tax status to all commercial
MCOs; however, small MCOs (<10 000 enrollees) were underrepresented.
The 169 Medicaid MCO submissions were from 29 of the 40 states with Medicaid
MCOs and represented 50% of the 337 Medicaid MCOs in 1999. Almost half (48%)
of the Medicaid plans in our sample also offered commercial products, compared
with 62% nationwide. Regional distributions across the 10 Department of Health
and Human Services regions were similar between the 169 Medicaid plans in
this sample and the 337 existing nationwide.
The mean performance across all clinical quality and access indicators
for plans with commercially enrolled children was significantly higher than
that of plans with Medicaid-enrolled beneficiaries with the exception of adolescent
well visits (Table 2). For example,
across all commercial plans, the mean (10th-90th percentile) plan performance
on childhood immunizations was 64% (43%-81%) of 2-year-olds completely immunized;
for Medicaid plans the mean performance was 49% (15%-69%). Assessed procedures
had consistently higher utilization rates in the commercial populations than
in their Medicaid counterparts (P<.001).
From 81 health plans that reported information on both Medicaid and
commercial beneficiaries, performance indicator data availability ranged from
56 plans reporting on well-child visits for children aged 15 months or younger
to 72 plans reporting information on childhood immunizations. For Medicaid
and commercially enrolled children served by the same health plan, the mean
performance scores for commercially enrolled children statistically significantly
exceeded that for their Medicaid counterparts (P<.05)
for each clinical quality and access indicator except adolescent well visits
(Table 3). Performance differences
were greatest for indicators assessing multiple coordinated contacts with
the health care system such as the combined childhood immunization rate (mean
difference of 15%) or well-child visits in the first 15 months of life (mean
difference of 22%). Indicators requiring a single point of service delivery
revealed less disparities—annual well-child visits from 3 to 6 years
of age (mean difference 5%) or the single vaccine for adolescents (mean difference
Consistent differences were observed when rates of commonly utilized
childhood procedures were examined (Table
3). For commercially enrolled children younger than 5 years, 35
per 1000 children underwent myringotomy, compared with a mean rate of 2 per
1000 for Medicaid enrolled children. Similar differences were observed for
the rate of tonsillectomy, with a mean rate for children younger than 10 years
of 7 per 1000 for commercially enrolled children and 0.5 per 1000 for Medicaid
For most plans, QOC provided to commercially enrolled children significantly
exceeded QOC provided to Medicaid enrollees on all available HEDIS child and
adolescent clinical and access indicators except adolescent well visits. On
a scatterplot of performance on the child immunizations indicator for the
72 plans serving both populations (Figure
1), the worse performance for the Medicaid population of many plans
is evident. However, a subgroup of plans achieved high rates of immunizations
(>75%) for both their Medicaid and commercial populations.
For indicators in both the commercial and Medicaid plans, within each
group—clinical, access, and procedural—the rates were highly correlated
(data available from the authors on request). In addition, examining the correlations
between difference scores (commercial minus Medicaid performance), moderate
to high correlations within each group of indicators were present. However,
similar findings between these groups of indicators were not observed, suggesting
unique contributions by each of the 3 indicator groups to the evaluation of
In the linear regression models, few significant effects across the
10 performance indicators were observed (Table 4). However, for both commercial and Medicaid plans, increased
years in business were associated with a convergence in rates (reduced difference
scores) for most well-child visit indicators and myringotomy and tonsillectomy
measures. In addition, increased Medicaid enrollment was associated with reduced
differences for all procedure indicators in both the younger and older age
groups. For-profit plans were associated with greater differences between
Medicaid and commercial enrollees for both childhood immunizations and well-child
visits for 3- to 6-year-olds.
For the clinical quality indicators, no consistent patterns emerged
in the linear regression models with the exception of the top performing group
of Medicaid plans. Only members of this group demonstrated statistically significant
and consistently reduced differences between Medicaid and commercial enrollees
for clinical quality indicators.
Increasingly, consumers, regulatory bodies, and state and federal governments
are asking for comparisons of health care quality performance across providers
of care.36 As the federal and state governments
have turned to commercial health care systems to provide care to children
in the Medicaid and newly formed State Children's Health Insurance Programs,
assumptions have been made that equivalent QOC will be achieved for these
newly enrolled populations when compared with commercially enrolled counterparts.
Our study is the first to our knowledge to document that children enrolled
in Medicaid served by commercial MCOs frequently receive lower QOC than their
commercially enrolled counterparts in the same MCO on a number of performance
indicators. A few select MCOs do achieve high-quality and equivalent care
for both Medicaid and commercially enrolled children.
Observed differences in procedure utilization for common conditions
including myringotomy and tonsillectomy clearly indicate wide variation in
practice between commercial and Medicaid enrolled children. Despite guideline
development and dissemination of treatment standards, 20-fold differences
were observed between the 2 enrollment groups for myringotomies in younger
children.37 While absolute performance goals
such as those for clinical quality and access indicators are not established
for these procedures, clearly the observed variation in utilization rates
cannot be explained solely based on clinical indications.
By examining the QOC results for both Medicaid and commercially insured
children within the same plan, we controlled for variations in performance
due to corporate structure, management strategies, and provider networks.
Within most of these plans, Medicaid enrollees received worse-quality service
as measured by specified clinical quality and access HEDIS indicators, with
the exception being adolescent immunization for which adequate performance
was not demonstrated for either population. Similarly, higher rates of utilization
for common procedures were present in the commercial population compared with
the Medicaid populations, suggesting, either differences in primary care referral
rates, access barriers to specialists, or other unidentified practice variations
affecting utilization of these services.
Importantly, several plans serving both populations did achieve equivalent
and high-quality care for their Medicaid enrollees. Efforts to statistically
model the difference between commercial and Medicaid rates across the 10 performance
indicators using characteristics of plans were unsuccessful. Characteristics
previously found to be associated with higher quality care—nonprofit
status, regional location, publicly reporting results, plan size and years
in business—were not consistently associated with equivalent QOC for
commercial and Medicaid enrollees. Managed care organizations operating a
plan for more years and/or with larger enrollments attenuated the differences
in observed variations in well-child visits and utilization rates for procedures,
potentially suggesting the successful development of the network and services
for Medicaid enrollees over time.
However, similar results by larger or older MCOs were not observed for
the clinical quality indicators. Only top performance in delivery of care
for Medicaid enrollees was associated with reduced differences between commercial
and Medicaid plans. These findings are important in that plan characteristics
and commercial sector performance results cannot be used to infer QOC delivered
to Medicaid enrollees. Thus, these analyses suggest that only Medicaid-specific
performance information can serve to inform beneficiaries and policy makers
on the QOC of services delivered.
Although plan characteristics associated with higher QOC were not identified,
it is clear that providing care to a Medicaid population is difficult. To
try to gain insight into those difficulties and how they might be addressed,
on completion of quantitative analyses described above, key informant interviews
were undertaken with medical directors in health plans that achieved high
quality for both Medicaid and commercial enrollees. Corporate commitment to
serve the community was a uniform response. In addition, specific challenges
faced in serving Medicaid beneficiaries identified by most informants included
a lack of reliable transportation, geographic maldistribution of existing
network providers, language barriers, inflexible parent work hours, and a
lack of continuity in primary care resulting in high utilization of emergency
department services. Responses to these challenges included geographic location
of providers near target populations and/or on public transportation routes,
incorporation of traditional providers for Medicaid beneficiaries (eg, community
health centers) into MCO networks, extension of provider hours into evenings
and weekends, and efforts to address deficits in continuity of care through
various education strategies. Continued challenges reported by several MCOs
involved the interface with state government and timely enrollment and/or
changes in eligibility information. Importantly, despite the challenges in
delivering care to Medicaid beneficiaries, none of the respondents anticipated
withdrawal from the Medicaid market.
While HEDIS is the industry standard for performance measurement, several
important limitations must be considered. Reported QOC represents both the
clinical services delivered and the completeness of the data sources documenting
care. For most indicators, administrative data is supplemented by individual
chart review to ascertain whether specific services have been provided. Failure
to document provision of services is considered a deficit in QOC. For indicators
that allow samples of plan members, sampling error may have contributed to
select associations in the regression models or the findings of some plans
achieving parity in outcomes.
These performance data are not adjusted for sociodemographic characteristics
of the populations. Efforts to quantify and adjust health system performance
based on underlying social, demographic, or economic characteristics of the
populations suggest that the influence of these characteristics on reported
health plan quality may be frequently overstated. We have published findings
of adjusted and unadjusted HEDIS rates demonstrating some differences in the
absolute results but minimal change in health plans' relative performance.38
This study represents the largest comparative study of Medicaid and
commercial performance results to date; however, it does not include a small
subset of health plans that chose not to report data. We previously demonstrated
that plans restricting public access to quality information provide poorer
quality care to their enrollees.39 Therefore,
our national estimates of performance results for MCOs may be slightly higher
than if all plans had reported. Importantly, for these reporting plans neither
Medicaid nor commercial plans consistently demonstrated high performance on
any of the indicators (eg, childhood immunization rates of 64% and 49% for
commercial and Medicaid plans, respectively). Finally, only a limited number
of indicators, mostly reflecting preventive care, were available for study.
Whether these findings would extend to care for acute or chronic conditions
Annual measurements of health care quality information and public reporting
of HEDIS indicators are both associated with improved performance.40 While most plans in this study had reported HEDIS
data for at least 2 years, ongoing quality monitoring efforts are underway
that may improve both clinical performance and data quality. It is likely
that if Medicaid MCOs continue to measure and report quality information they
will achieve quality improvements. Future reexaminations of plan performance
are warranted to ensure improvement of Medicaid quality and guard against
erosion of commercial QOC.
These indicators may represent the current industry standard, but they
do not reflect the complexity of needed information for complete assessment
of health care quality. Continued performance measure development and deployment
must include not only indicators for health care delivery systems but also
assessments of hospitals and individual clinical providers and a wider range
of acute and chronic care measures to achieve a more complete QOC assessment.
New HEDIS indicators for asthma QOC41 and children
with special health care needs42 and recent
assessments of provider level guideline adherence43 represent
opportunities for more robust QOC assessments.
Immediate policy implications become apparent from our study. State
and federal policy makers must monitor health care systems to ensure that
quality services are being provided for children and adolescents enrolled
in commercial, Medicaid, and new State Children's Health Insurance Program
plans. Importantly, while these analyses document performance problems in
managed systems of care, no similar QOC information is available on care provided
through traditional fee-for-service programs serving many beneficiaries across
the United States. It is not known, for example, whether children served by
Medicaid MCOs are receiving better or worse care than Medicaid children not
enrolled in MCOs. These gaps in information must be addressed to accurately
assess the quality performance provided by different components of our health
This study documents worse QOC on clinical quality and access indicators
for children and adolescents in Medicaid programs served by commercial health
plans and dramatic differences in utilization rates for common procedures.
Demonstrated high-quality and equivalent care by a few health plans, however,
calls for more active policy and management decisions about poor-quality programs.
Our findings suggest that identification of Medicaid plans providing high
QOC requires plan-specific performance information and that neither plan characteristics
nor commercial sector performance can be used to identify high-quality Medicaid