Ma J, Stafford RS. Quality of US Outpatient CareTemporal Changes and Racial/Ethnic Disparities. Arch Intern Med. 2005;165(12):1354-1361. doi:10.1001/archinte.165.12.1354
The current national measure set for the quality of health care underrepresents the spectrum of outpatient care and makes limited use of readily available national ambulatory care survey data.
We examined 23 outpatient quality indicators in 1992 and again in 2002 to measure overall performance and racial/ethnic disparities in outpatient care in the United States. The National Ambulatory Medical Care Survey and the National Hospital Ambulatory Medical Care Survey yielded information about ambulatory services provided in private physician offices and hospital outpatient departments, respectively. Quality indicator performance was defined as the percentage of applicable visits receiving appropriate care.
In 2002, mean performance was 50% or more of applicable visits for 12 quality indicators, 7 of which were in the areas of appropriate antibiotic use and avoiding unnecessary routine screening. The performance of the remaining 11 indicators ranged from 15% to 42%. Overall, changes between 1992 and 2002 were modest, with significant improvements in 6 indicators: treatment of depression (47% vs 83%), statin use for hyperlipidemia (10% vs 37%), inhaled corticosteroid use for asthma in adults (25% vs 42%) and children (11% vs 36%), avoiding routine urinalysis during general medical examinations (63% vs 73%), and avoiding inappropriate medications in the elderly (92% vs 95%). After adjusting for potential confounders, race/ethnicity did not seem to affect quality indicator performance, except for greater angiotensin-converting enzyme inhibitor use for congestive health failure among blacks and less unnecessary antibiotic use for uncomplicated upper respiratory tract infections among whites.
Measurable quality deficits and modest improvements across time call for greater adherence to evidence-based medicine in US ambulatory settings. Although significant racial disparities have been described in a variety of settings, we observed that similar, although less than optimal, care is being provided on a per-visit basis regardless of patient racial/ethnic background.
Quality of health care is prominent on the nation’s health policy agenda and in the current health care debate.Quality can be evaluated at 3 distinct yet interrelated levels: structure, process, and outcomes.1 It is the Institute of Medicine’s view that a balanced national measure set should avoid structural measures because they have failed to consistently reflect the quality of care and desired outcomes.2 Multiple technical and practical difficulties exist in assessing all but the most common outcomes. As a result, quality is most often measured in the form of process indicators.3
In the past 2 decades, tremendous progress has been made in providing better quality measurement and reporting.4,5 Most recently, the Agency for Healthcare Research and Quality released the first edition of 2 comprehensive annual reports about health care in the United States—the National Healthcare Quality Report (NHQR)6 and the National Healthcare Disparities Report.7 The NHQR identified a variety of areas where health care has markedly improved across time and is now reaching or surpassing national performance goals. However, the report also identified many more areas where the quality of health care delivery is suboptimal. As its companion document, the National Healthcare Disparities Report demonstrated that suboptimal health care does not affect the US population uniformly. Instead, racial, ethnic, and socioeconomic disparities are national problems. These 2 reports represent the most comprehensive and in-depth documentation available to date on the quality of US health care. Nonetheless, both reports acknowledged many existing limitations and emphasized the need for further research on additional data sources and new quality measures.
The measure set examined in the NHQR and the National Healthcare Disparities Report underrepresents the spectrum of outpatient care, particularly for recommended medication use in chronic conditions where the most rigorous evidence exists. Also, the measure set makes limited use of readily available national surveys of patient care physicians in private offices (National Ambulatory Medical Care Survey [NAMCS]) and hospital outpatient departments (National Hospital Ambulatory Medical Care Survey [NHAMCS]). The current literature on quality of ambulatory care can be enriched with more data regarding changes across time and racial/ethnic disparities. To fill in these gaps, we constructed 23 process measures of outpatient care, with a focus on pharmacotherapy for chronic conditions. These measures are not intended to be exhaustive of outpatient activities but, rather, to represent those domains that can be measured with reasonable reliability using the 2 national surveys.
This analysis was based on the NAMCS (a survey of private office–based physicians) and the Outpatient Department component of the NHAMCS. Complete survey descriptions can be viewed at the National Center for Health Statistics Web site (http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm). Both surveys have been conducted annually since 1992, and 2002 data were the latest to be released at the time of this study. Both surveys used multistage stratified probability sampling procedures, which enable nationally representative estimates to be generated. Between 1992 and 2002, annual participation rates among selected physicians for the NAMCS averaged 70%, whereas, on average, 90% of the selected hospitals participated in the NHAMCS. The surveys, particularly the NAMCS, have been validated against other national data sources.8,9
The unit of analysis in both surveys is the patient visit. In-office physicians (with staff assistance) or hospital staff completed standard encounter forms for a systematic random sample of patient visits during randomly assigned reporting periods. Yearly standard forms varied slightly between the NAMCS and the NHAMCS, and they underwent revisions every 2 years. Variables common to NAMCS and NHAMCS encounter forms across time included patient visit characteristics, diagnoses (up to 3), new and continuing medications (up to 5 before 1995 and up to 6 thereafter), and other medical services (eg, surgical and counseling services) provided at the visit. Item nonresponse rates were mostly 5% or less in both surveys across years.
We applied general criteria, which were defined in accordance with the Institute of Medicine’s broad criteria of clinical importance, scientific soundness, and feasibility for indicator selection,2 and criteria specific to limitations of the data sources (Table 1). To the extent possible, each selected indicator must meet both sets of criteria. We constructed 23 quality indicators that fall into 5 distinct categories: medicinal management of common chronic diseases (10 measures), appropriate antibiotic use (3 measures), preventive counseling (5 measures), screening tests (4 measures), and inappropriate prescribing in elderly patients (1 measure). Table 2 details the composition and a key source of supporting evidence for each chosen quality indicator. The evidence base for our set of measures consists of current, formal recommendations in the form of practice guidelines or, in the absence of authoritative guidelines, consensus expert statements. Although we acknowledge the limitations of retrospectively applying current recommendations, this approach allows a longitudinal assessment of potential improvements in the quality of care.
The performance of quality indicators was computed as the percentage of applicable visits receiving appropriate care (the number of eligible visits receiving recommended care divided by the number of all eligible visits). Visits found to have clinical contraindications to a recommended treatment were excluded from the numerator and the denominator. Attempts were carefully made to exclude prominent contraindications from quality measurement (Table 2). In most cases, disease conditions and prominent exclusions were identified by using International Classification of Diseases, Ninth Revision, Clinical Modification, diagnostic codes; NAMCS/NHAMCS-specific reason-for-visit codes; and diagnostic checkboxes if available.
The National Center for Health Statistics analytical guidelines establish the legitimacy of combining multiple years of data from the NAMCS and the NHAMCS. Such data merging was necessary in this study to attain adequate sample size and stable estimates. Also, combined NAMCS and NHAMCS data allow for the inclusion of a wider range of outpatient settings and a broader socioeconomic spectrum of patients seeking ambulatory care. Quality measures on which the 2 data sources differed significantly were noted in the results and were controlled for when examining racial/ethnic disparities.
All analyses took into account the sampling weights and sample design variables available in the NAMCS and the NHAMCS for the generation of nationally representative point and variance estimates. Mean performance and 99% confidence intervals based on combined NAMCS and NHAMCS data were generated for 1992 and 2002 using the SURVEYMEANS procedure in SAS for Windows (SAS Institute Inc, Cary, NC). Nonoverlapping confidence intervals indicate significant differences in performance between the 2 years.
For assessment of racial/ethnic disparities, NAMCS and NHAMCS data from 1995 through 2002 were combined so that reliable national estimates (based on a sample size >30 and a relative SE <30% according to National Center for Health Statistics standards) could be generated for the 3 main racial/ethnic groups—non-Hispanic whites, non-Hispanic blacks, and Hispanics. A series of multivariate logistic regression analyses were performed using the RLOGIST procedure in SAS-Callable SUDAAN (RTI, Research Triangle Park, NC) to test racial/ethnic disparities in indicator performance after adjusting for patient age, sex, and practice setting.
For each of the 23 quality measures, Table 3 provides the mean performance rates with 99% confidence intervals in 2002 and 1992 and differences in performance by race/ethnicity and practice setting.
In 2002, mean performance rates were greater than 50% of applicable visits for 12 indicators, including 4 of the 10 quality measures of medicinal management of common chronic diseases, all 3 measures of appropriate antibiotic use, all 4 measures of screening tests, and the 1 measure of inappropriate prescribing in elderly patients. The 4 chronic disease management measures were avoiding benzodiazepine use for depression uncomplicated by anxiety (86%), pharmacotherapy and psychotherapy for depression (83%), use of antithrombotic agents for atrial fibrillation (60%), and use of thiazides or β-blockers among treated patients with uncomplicated hypertension (58%). The mean performance rate was 76% for avoiding antibiotic use for uncomplicated upper respiratory tract infection, 67% for recommended antibiotic use for uncomplicated urinary tract infection, and 58% for recommended antibiotic use for acute otitis media. Finally, the mean performance rate was 94% for avoiding routine electrocardiography during annual general medical examinations (GMEs), 86% for avoiding routine hemoglobin/hematocrit measurements during GMEs, 73% for avoiding routine urinalysis during GMEs, 70% for hypertension screening during GMEs (70%), and 95% for avoiding inappropriate medications in elderly patients.
The remaining 6 quality indicators of chronic disease management and the measure of smoking cessation counseling for smokers during their GMEs were performed at a rate between 30% and 42%. The 6 chronic disease management indicators included inhaled corticosteroid use for asthma in adults and children separately, angiotensin-converting enzyme (ACE) inhibitor use for chronic heart failure (CHF), aspirin use for coronary artery disease, statin use for hyperlipidemia, and β-blocker use for coronary artery disease. The mean rates of diet and exercise counseling separately during GME visits were only 15% to 21% for adults and 23% to 27% for adolescents.
Quality of outpatient care between 1992 and 2002 improved for 17 quality indicators; however, the improvements reached statistical significance for only 6 quality indicators: pharmacotherapy or psychotherapy for depression (83% vs 47%), statin use for hyperlipidemia (37% vs 10%), inhaled corticosteroid use for asthma in adults (42% vs 25%) and children (36% vs 11%), avoiding routine urinalysis during GMEs (73% vs 63%), and avoiding inappropriate medications in elderly patients (95% vs 92%) (Table 3).
After adjusting for patient age, sex, and source of data, racial/ethnic differences in performance reached statistical significance for only 2 quality indicators (Table 3). Relative to non-Hispanic whites, non-Hispanic blacks with CHF were more likely to receive ACE inhibitors for CHF (45% vs 32%). Yet, non-Hispanic whites were less likely to receive antibiotics unnecessarily for uncomplicated upper respiratory tract infection than blacks (74% vs 53%). Comparisons between the NAMCS and the NHAMCS suggested some differences by site of care. Compared with patient visits to hospital outpatient departments, visits to private physician offices were more likely to provide exercise counseling for adults (19.0% vs 9.3%; P<.001) and adolescents (25.9% vs 7.5%; P<.001) and diet counseling for adolescents (31.3% vs 15.3%; P=.003). Conversely, patient visits to hospital outpatient departments had high rates of unnecesary electrocardiography (96.0% vs 93.7%; P=.007) and urinalysis (81.4% vs 73.7%; P<.001) as routine screening tests.
We constructed and examined 23 quality measures by using 2 national ambulatory care surveys. Our results suggest that quality of outpatient care falls short of evidence-based recommendations in many clinical areas and that improvements across time are generally modest. On an encouraging note, similar, although less than optimal, care is being provided to members of different racial and ethnic groups.
The 23 measures conform well to the Institute of Medicine’s conceptual framework, which depicts quality measurement based on health care and patient needs.2 Our quality indicators measure the effectiveness of health care delivery in meeting patient needs to stay healthy (eg, preventive counseling and screening testing), get better (eg, appropriate antibiotic use), and live with illness (eg, medicinal management of common chronic diseases). Also, equity of quality by race and ethnicity was compared.
In 2002, 6 of the 10 chronic disease management indicators and all 5 preventive counseling indicators had a mean performance rate of less than 50% of applicable patient visits. Improvements from 1992 were modest for all but 6 quality indicators: pharmacotherapy or psychotherapy for depression, statin use for hyperlipidemia, inhaled corticosteroid use for asthma in adults and children separately, avoiding routine urinalysis during GMEs, and avoiding inappropriate medications in elderly patients. Treatment of depression in primary care has improved considerably during the past decade. In 2002, pharmacotherapy or psychotherapy was provided during 83% of all visits for depression. Even at a significantly improved level in 2002, statins were prescribed at only 37% of all visits by patients with hyperlipidemia, indicating substantial underuse. Likewise, several other indicators suggest continued underuse in 2002 of proven drug therapies in the treatment of cardiovascular disease, including ACE inhibitor use in CHF (39%) and the use of aspirin (38%) and β-blockers (31%) in coronary artery disease.
Despite the difficulty in directly comparing performance across indicators, variations are evident in different clinical areas. Overall, room for improvement is substantial in many clinical areas. This conclusion echoes that from the first NHQR and a recent study26 that evaluated the performance of 439 indicators in American adults, representing 30 acute and chronic conditions and related preventive care. In the present study, the performance in appropriate antibiotic use (3 measures) and screening tests (4 measures) generally conforms more closely to evidence-based recommendations compared with the performance in pharmacotherapy for chronic diseases (10 measures) and preventive counseling (5 measures). This pattern is consistent with that summarized by Schuster et al27 after reviewing a body of literature between 1987 and 1997 that covers a broad range of conditions and settings. Using combined data from medical records and patient self-reports, McGlynn et al26 also identified substantial variability in performance across medical conditions and by delivery mode of care, with counseling services manifesting the lowest rates of adherence to recommendations. Our data also suggest that preventive counseling is less likely to occur in hospital outpatient departments than in private physician practices. The first NHQR and the first National Healthcare Disparities Report noted frequently missed opportunities for preventive care, and the NHQR stated that “management of chronic diseases presents unique quality challenges.”6(p1)
The literature abounds with studies of suboptimal care in ethnic minorities. After controlling for a variety of potential confounders (including practice setting), race/ethnicity did not seem to significantly differentiate the performance of the quality indicators that we examined, except for greater ACE inhibitor use in CHF among non-Hispanic blacks and less unnecessary antibiotic use in uncomplicated upper respiratory tract infection among non-Hispanic whites. In an attempt to find explanations for the observed racial differences, we further controlled for diagnoses of hypertension and chronic renal insufficiency for ACE inhibitor use in CHF and tobacco use for antibiotic use in upper respiratory tract infection in respective multivariate logistic regressions. The results remain unchanged. The per–patient visit basis of our data complicates the interpretation of the quality results by race and ethnicity because of the inability to account for disparities in health care access and utilization, which is widely noted in the literature.7,28 Also, physician-reported racial/ethnic assignments are possibly associated with errors and inaccuracies. Nonetheless, the lack of racial/ethnic differences in per-visit quality may suggest that quality disparities arise more from access to care and site of care than from direct disparities once patients enter a particular care system. This may be particularly true for chronic medical conditions, which are prominent among the measures we assessed. Disparities in health care should be examined and interpreted with great care because of their public health importance and political sensitivity.29 On the one hand, significant racial and ethnic disparities, when observed, are not necessarily causal because of the probable complex interplay of race/ethnicity with other factors that could affect health care.30 On the other hand, the absence of statistical significance does not automatically mean that disparities do not exist or are not meaningful from a clinical or political perspective. For example, the tendencies toward lower use of aspirin and statins and less exercise counseling among non-Hispanic blacks and Hispanics with elevated risk of coronary artery disease are noteworthy given the disproportionately higher burden of the disease borne by these groups at the population level.31
Measure performance depends not only on quality of care but also on quality of data. Our findings reflect the quality of care in the general US outpatient population only to the extent to which information was accurately reported on encounter forms, with both underreporting and overreporting being possibilities. The magnitude of underreporting and overreporting is difficult to predict and is likely to vary by type of care. In general, the current visit-based physician-reported data are likely to be more accurate for assessing medication prescribing and laboratory tests than for counseling services. Yet, the per–patient visit basis may result in overrepresentation of sicker, more frequent users of outpatient care and, consequently, overestimation of some quality measures, particularly those of chronic disease management. Also, a maximum limit of 6 medications per visit may cause the undercounting of medications, particularly nonprescription medications (eg, aspirin). In addition, measurement inaccuracy may result from the inability to exclude otherwise ineligible visits or to account for case-mix variation due to the lack of detailed, patient-level clinical data. The validity of the examined quality measures also is affected by the strength of scientific evidence and the intensity of relevant guidelines, which may vary across measures. These many sources of variation by indicator (eg, data accuracy, measurement accuracy, and strength of supporting evidence) make it difficult to specify an overall or an indicator-specific optimal performance rate.
Despite the acknowledged limitations, the NAMCS and the NHAMCS offer several unique advantages. Physician-reported data have the specific advantage of measuring physician behavior more accurately than patient-reported data. Also, the NAMCS and the NHAMCS cover a longer consecutive time span and provide more complete information about disease-specific physician activities than many other national databases. Nonetheless, future research is warranted to enhance existing databases and to exploit other, complementary data on outpatient care.
This study contributes to the ongoing efforts to develop a national system for measuring and reporting the quality of outpatient health care in the United States. The present findings suggest that large gaps exist between actual clinical practices and evidence-based recommendations in many areas of outpatient care. We found limited evidence that these performance gaps are closing as a result of proliferating evidence-based practice guidelines. Reasons for the underperformance and the lack of improvement are likely multifaceted.32 For example, rapidly advancing medical science and knowledge and dramatically changing health care needs during the past half-century have engendered exceptionally complex health care. These challenges in the nation’s health care are met with a largely acute care–oriented delivery system that often fails to provide timely and appropriate evidence-based care and that lacks uniform and patient-centered services. Unless fundamental changes are made to the health care system and to the way that people working in it practice, existing performance gaps will persist. Possible changes may include facilitating provider adherence to evidence-based care through feedback on practice patterns, better incentives for attaining recommended practices, and innovative use of information technology to assist with longitudinal management. The active use of risk stratification may be the key to making cost-effective use of limited health care resources. Alternative delivery strategies also may need to be considered, including case management, use of paraprofessionals, outreach programs, and group visits. Such fundamental changes can take place only if committed partnerships are formed among all the parties with a stake in high-quality care, including policy makers, health plan administrators, medical researchers, hospital directors, physicians, and patients. Improving quality and reducing disparities warrants strong leadership, targeted public health policies, and creative initiatives at the national, state, and local levels.
Correspondence: Randall S. Stafford, MD, PhD, Stanford Prevention Research Center, Stanford University School of Medicine, Hoover Pavilion, Room N229, Stanford, CA 94305-5705 (firstname.lastname@example.org).
Accepted for Publication: January 20, 2005.
Financial Disclosure: None.
Funding/Support: This study was supported by research grant R01 HS11313-01 from the Agency for Healthcare Research and Quality, Rockville, Md.