A and B, Distributions of annual rates of low-value services (predicted modes) adjusted for beneficiary sociodemographic and clinical characteristics, local area economic and educational characteristics, year, and hospital referral region, as well as health care provider organization (B only). These graphs represent adjusted rates of low-value services for physicians practicing within the same region (A) and within the same health care provider organization (B). C and D, Corresponding predicted distributions of annual rates of low-value services, adjusted for the same factors, based on physician characteristics.
eMethods. Definitions, Weighting, Low-Value Service Measures, and Proportion of Variation
eTable 1. Codes for Measures of Low-Value Care
eTable 2. Association Between Physician Characteristics and Low-Value Services (Narrower Set)
eTable 3. Association Between Physician Characteristics and Low-Value Services (Narrowest Set)
eFigure 1. Variation in Low-Value Services (Narrower Set) Across Physicians
eFigure 2. Variation in Low-Value Services (Narrowest Set) Across Physicians
Customize your JAMA Network experience by selecting one or more topics from the list below.
Schwartz AL, Jena AB, Zaslavsky AM, McWilliams JM. Analysis of Physician Variation in Provision of Low-Value Services. JAMA Intern Med. 2019;179(1):16–25. doi:10.1001/jamainternmed.2018.5086
Does extensive variation exist among physicians in the provision of low-value health care services, and can levels of provision be predicted by observed physician characteristics?
In this observational study using Medicare claims data of 3 159 834 beneficiaries served by 41 773 generalist physicians, the rates of low-value services were 60% higher for primary care physicians at the 90th percentile of their provider organization than for physicians at the 10th percentile. Only 1.4% of physician variation within organizations could be explained by observable physician characteristics.
Although differences in physician practice patterns may substantially contribute to low-value service use, it is difficult to predict which physicians are more wasteful without measuring their behavior.
Facing new financial incentives to reduce unnecessary spending, health care organizations may attempt to reduce wasteful care by influencing physician practices or selecting more cost-effective physicians. However, physicians’ role in determining the use of low-value services has not been well described.
To quantify variation in provision of low-value health care services among primary care physicians and to estimate the proportion of variation attributable to physician characteristics that may be used to predict performance.
Design, Setting, and Participants
This retrospective analysis included national Medicare fee-for-service claims of 3 159 834 beneficiaries served by 41 773 generalist physicians from January 1, 2008, through December 31, 2013 (data were analyzed in 2016 through 2018). Multilevel modeling was used to estimate the extent of variation in service use across physicians within their region and provider organization, adjusted for patient clinical and sociodemographic characteristics and sampling variation. The proportion of variation attributable to physician characteristics that may be used to predict performance (age, sex, academic degree, professorship, publication record, trial investigation, grant receipt, pharmaceutical or device manufacturer payment, and panel size) was estimated via additional regression analysis.
Main Outcomes and Measures
Annual count per beneficiary of 17 primary care–associated services that provide minimal clinical benefit.
Among the 3 159 834 beneficiaries (58.3% women; mean [SD] age, 73.2 [11.0] years) served by 41 773 physicians (74.9% men; mean [SD] age, 48.0 [10.1] years), the mean annual rate of low-value services was 33.1 services per 100 beneficiaries. Considerable variation across physicians within the same region was found (SD, 8.8 [95% CI, 8.7-8.9]; 90th:10th percentile ratio, 2.03 [95% CI, 2.01-2.06]) and across physicians within the same organization (SD, 6.1 [95% CI, 6.0-6.2]; 90th:10th percentile ratio, 1.61 [95% CI, 1.60-1.63]). The corresponding rates at the 10th percentile of physicians within region and within organization respectively were 21.8 and 25.3 services per 100 beneficiaries. Observable physician characteristics accounted for only 4.4% of physician variation within region and 1.4% of physician variation within organization.
Conclusions and Relevance
Physician practices may substantially contribute to low-value service use, which is prevalent even among the least wasteful physicians. Because little variation is predicted by measured physician characteristics, direct measures of low-value care provision may aid organizational efforts to encourage high-value practices.
New payment models reward health care provider organizations for reducing wasteful care. These models include accountable care organization programs that place spending for a provider organization’s patient population under a global budget and provide incentives to bring spending below the budget while achieving high performance on quality measures.1 Understanding physicians’ propensity to provide unnecessary medical services independent of their patients’ needs could promote success under these models. For example, knowing physicians’ practice patterns, accountable care organizations could target more wasteful physicians for retraining. They could also include less wasteful physicians in their risk contracts with payers or in their referral networks. The premise of such physician-focused strategies is that physician-specific factors (eg, education, preferences) may influence their providing unnecessary care. However, efforts to describe the role physicians play in generating unnecessary care have been limited.
Recent studies have demonstrated the feasibility of directly measuring low-value services that provide minimal benefits for patients.2-5 Such direct measurement offers distinct advantages over alternative approaches, such as less specific measures of the use of health care services (eg, total risk-adjusted spending per patient) or profiling based on physician characteristics (eg, age, sex, or training).6-15 Because they do not rely on comparisons, direct measures of low-value service use provide an assessment of overuse for all providers, even the most efficient providers,5 and thus may reveal the full potential for strategies to eliminate wasteful care. Direct measures are also likely to discriminate between more and less efficient physicians better than indirect measures. In particular, although characteristics such as physicians’ age, sex, or training may be correlated with practice patterns,6-30 such characteristics may account for only a small proportion of physician-level variation. Also, nonspecific measures of use of health care services do not distinguish between appropriate and inappropriate variation in service use. Thus, direct measurement could better support efforts to identify physicians with cost-effective practices.
Prior studies have explored variation in low-value service use among regions and organizations to understand the potential role of markets and organizations in shaping practice patterns. However, to our knowledge, no study has examined variation in a broad set of low-value services at the physician level.2,3,5,31,32 Evidence of marked physician-level variation in wasteful practices not attributable to patient, organizational, or regional factors would support a role of physician-level determinants in generating overuse. Using Medicare claims and measures of 17 low-value services commonly provided in the primary care setting, we assessed physician-level variation in low-value service use within regions and within provider organizations. We also assessed the extent to which this variation could be explained by observable physician characteristics that may be used to predict physician performance.
We assessed patient characteristics and health care use from Medicare enrollment and claims files from January 1, 2008, through December 31, 2013, for a random 20% sample of Medicare fee-for-service beneficiaries. We assessed physician characteristics from a database assembled by Doximity, an online social network for US physicians (Doximity, Inc). This database includes not only members of the networking service but also physicians identified via the National Provider Identifier Registry, National Plan and Provider Enumeration System, state licensing boards, specialty societies, and collaborating hospitals and medical schools. The database includes physician characteristics drawn from sources including the National Plan and Provider Enumeration System, the Association of American Medical Colleges faculty roster database, PubMed, ClinicalTrials.gov, the National Institutes of Health Research Portfolio Online Reporting Tools database, and the Centers of Medicare & Medicaid Services Open Payments database. Details of this database and its validation have been published elsewhere.19,33-36 Institutional review board approval was granted by the Harvard University Committee on the Use of Human Subjects. Informed consent was not required for the use of publicly available data, and data were analyzed in 2016 through 2018.
For each measurement year in the 2008-2013 study period, we included beneficiaries who were continuously enrolled in Medicare Parts A and B in that year (while alive for decedents) and in the prior year, were residing in one of the 50 US states or Washington, DC, and had at least 1 of the evaluation and management services that were used to attribute beneficiaries to physicians. We excluded patients attributed to physicians with missing data on key characteristics in the Doximity database (16.4% of patient-year observations) or attributed to physicians without at least 1 attributed patient qualifying for the denominator of each measure of low-value service use (16.7% of remaining patient-year observations).
In each year, each patient was attributed to the primary care physician (physicians with specialty codes indicating general practice, family practice, internal medicine, or geriatric medicine) who accounted for the plurality of the patient’s office visits (Healthcare Common Procedure Coding System codes 92201-99205, 99211-99215, 99241-99245, G0402, G0438, and G0439). Ties were broken based on the plurality of allowed charges for office visits. As in prior work, provider organizations were defined by taxpayer identification numbers.5 Physicians were attributed to the taxpayer identification number accounting for the plurality of the physician’s office visit counts during the study period, with ties broken as above.
The primary outcome of this study was the rate of use of 17 low-value services, reported as the number of services per 100 patients per year. As detailed in prior studies using these measures,2,37 we selected services that produce minimal average clinical benefit in specific clinical situations that could be distinguished from potentially high-value indications with reasonable accuracy using claims and enrollment data. Sources for measure development included the American Board of Internal Medicine Foundation’s Choosing Wisely initiative,38 the US Preventive Services Task Force D recommendations,39 the Canadian Agency for Drugs and Technologies in Health technology assessments,40 and the peer-reviewed medical literature.41 The evidence base for these sources has been detailed previously.2,37 Of note, our study period largely precedes the 2012 expansion of the Choosing Wisely campaign.
As in prior studies,5 when there was discretion in defining measures, we used more specific definitions, which reduce the probability of detecting some low-value services, but also reduce the probability of misclassifying high-value services as low-value. Because this study’s focus was primary care physicians, we restricted our measure set to services whose use could be influenced substantially by a primary care physician’s treatment or referral decisions. Our approach attributed all low-value services received by a patient to the patient’s primary care physician, including some services for which other physicians were responsible. In sensitivity analyses, we therefore repeated analyses with smaller sets of services even more likely to reflect primary care physicians’ decisions.
Table 1 describes each low-value service measure, its operational definition, and the denominator population to which it applies. Operational definitions relied on information from claims, including the presence or timing of procedures, diagnoses, emergency department visits, or hospitalizations, and on information from the annual Master Beneficiary Summary File, such as age and diagnoses recorded in the Chronic Condition Data Warehouse, which draws from Medicare claims since 1999 to describe beneficiaries’ accumulated burden of chronic disease.42 Defining denominator populations for each measure allowed our analysis to account for differences across physicians in the number of patients who could potentially receive certain low-value services. For example, the analysis of low-value preoperative testing only included patients who underwent surgical procedures so that comparisons across physicians would not be confounded by differences in patients’ need for surgery. Specific diagnosis and procedure codes used for each measure are described in eTable 1 in the Supplement.
From the Medicare Master Beneficiary Summary File, we assessed age, sex, race/ethnicity, disability as the initial reason for Medicare enrollment, Medicare-Medicaid dual eligibility, and end-stage renal disease. From the Chronic Conditions Data Warehouse, we assessed the presence of 27 chronic conditions. Finally, from the 2007-2011 American Community Survey Summary File, we assessed the following characteristics of the elderly population in each beneficiary’s zip code tabulation area43: median income, proportion of the population below the federal poverty level, proportion with a high school degree, and proportion with a college degree.
From the Doximity database, we assessed physicians’ age, sex, training, academic engagement, and payment from a pharmaceutical or device manufacturer. Age was recorded in years as of July 2015. Training variables included academic degree (doctor of medicine vs doctor of osteopathy), graduation from a foreign medical school, and graduation from a medical school ranked among the top 20 research schools in US News and World Report in 2013.44 Academic engagement variables included rank of academic appointment (full professorship, associate professorship, assistant professorship, or no professorship), number of authored scientific publications, an indicator for authoring any publications, status as principal or subinvestigator of at least 1 registered clinical trial, and status as principal investigator with at least 1 National Institutes of Health grant. In addition to physicians’ amount of payment from pharmaceutical or medical device companies, we included an indicator for any nonzero payment. More information on these variables and their sources has been detailed elsewhere.19,33-36 Finally, using Medicare claims, we estimated Medicare fee-for-service panel size via the attribution methods detailed above (before exclusion of any patients or physicians from the sample), adjusting panel sizes to account for the 20% random sampling.
Data were analyzed from November 2016 through July 2018. Because of sampling error and nonrandom sorting of patients to physicians,45 unadjusted variation in low-value service use across physicians will tend to overstate the amount of variation that is due to differences in physician practice (ie, the amount of variation that would be observed if physicians treated an infinite number of patients and if patients were randomized to physicians). Therefore, we used multilevel models to estimate physician-level variation in rates of low-value service use net of sampling error, with techniques to adjust for patient, regional, and organizational factors.
As in prior work,5 we used a multilevel modeling approach developed by Fay and Herriot.46 This established method has been used to analyze composite measures of health care quality composed of multiple quality elements.47 First, for each low-value service, we calculated scores reflecting each physician’s covariate-adjusted rate of use. The service score was calculated via linear regression modeling of the number of times a beneficiary received the service as a function of beneficiary covariates, indicators for year, hospital referral region fixed effects, and physician fixed effects. The physician fixed effects yielded estimates of each physician’s score for that service. To better account for patient factors that might affect a patient’s likelihood of receiving a low-value service, we limited the samples for these regressions to beneficiaries satisfying the measure’s denominator criteria.
Second, a physician’s composite score was calculated as a weighted sum of the physician’s service-specific scores (eMethods in the Supplement). Third, we fit multilevel models to those composite scores to estimate variation across physicians, adjusted for sampling error. These models produced final estimates of between-physician variation (ie, SD) in rates of low-value services. To facilitate interpretation, we present SDs and their corresponding ratios of 90th to 10th percentiles, calculated based on the properties of normal distributions and physicians’ mean rate of low-value service provision. For example, a 90th:10th percentile ratio of 2.0 would suggest twice the rate of low-value services per patient per year for physicians at the 90th percentile of low-value service use than for physicians at the 10th percentile.
Examining within-region and within-organization variation allowed us to estimate whether substantial variation in physician practices existed even when holding constant all market-level or organization-level factors. The inclusion of hospital referral region fixed effects in this analysis yielded estimates of variation across physicians within the same geographic service area. To estimate within-organization variation, we repeated the analysis with physicians nested within an additional level, provider organizations, in our multilevel model. This model also allowed us to estimate the extent of between-organization variation in low-value services. To gauge the extent to which our estimates of physician-level variation in low-value service provision reflected variation in patients’ clinical characteristics, we conducted a sensitivity analysis without adjustment for chronic conditions.
To calculate the proportion of variation in low-value service provision predicted by physician characteristics, we compared the magnitude of variation observed in the preceding analyses to the magnitude of variation that would be predicted by differences in physician characteristics alone. This process entailed estimating associations between physician characteristics and low-value service use, predicting each physician’s rate of low-value service use based on the physician’s characteristics, and measuring the variation across physicians in those predicted rates of low-value services (eMethods in the Supplement). Again, we predicted variation within their region and the provider organization. The proportion of low-value service variation attributable to physician characteristics was calculated as the ratio of the variance of these predictions to the variance in observed rates of low-value services from the prior analysis.
Overall, 3 159 834 beneficiaries (58.3% women and 41.7% men; mean [SD] age, 73.2 [11.0] years) served by 41 773 physicians (25.1% women and 74.9% men; mean [SD] age, 48.0 [10.1] years) in 6771 provider organizations distributed across 306 regions were included in this study, with 10 199 293 beneficiary-year observations (Table 2). Physicians’ mean rate of low-value service provision was 33.1 services per 100 beneficiaries per year. Without adjustment for patient, regional, or organizational factors, the SD was 14.4 services per 100 beneficiaries per year, corresponding to a 90th:10th percentile ratio of 3.54.
The Figure displays the distribution of low-value service rates across physicians within the same geographic service (Figure, A) and within the same provider organization (Figure, B), both adjusted for patient characteristics. Within region, the estimated SD was 8.8 services per 100 beneficiaries per year (95% CI, 8.7-8.9), which corresponds to a 90th:10th percentile ratio of 2.03 (95% CI, 2.01-2.06) and a 10th percentile of 21.8 services per 100 beneficiaries per year. Within the same provider organizations, the estimated SD was 6.1 services per 100 beneficiaries per year (95% CI, 6.0-6.2), corresponding to a 90th:10th percentile ratio of 1.61 (95% CI, 1.60-1.63) and a 10th percentile of 25.3 services per 100 beneficiaries per year. The estimated SD in low-value service rates across organizations was 7.7 services per 100 beneficiaries per year. Estimates of variation were not substantially affected by excluding adjustment for patients’ comorbidities (within-region adjusted 90th:10th percentile ratio, 2.13 [10th percentile, 21.1]; within-organization adjusted 90th:10th percentile ratio, 1.66 [10th percentile, 24.9]). Table 3 presents this comparison in tabular form.
The Figure also presents the smaller magnitude of variation in low-value service rates across physicians that would be predicted by differences in physician characteristics. Table 4 presents the estimated associations between physician characteristics and low-value service rates used to calculate the predicted distributions. Within regions, 10 of 15 physician characteristics were predictive of low-value service use. Within organizations, 7 of 15 physician characteristics were predictive of low-value service use. Together, all physician characteristics accounted for 4.4% of within-region physician variation in low-value service provision (Figure, C) and 1.4% of within-organization variation (Figure, D).
Sensitivity analyses using narrower sets of low-value services that were more likely ordered by primary care physicians demonstrated similar results (eFigures 1-2 and eTables 2-3 in the Supplement). For example, for a narrower set of 12 low-value service measures (mean rate, 25.5 services per 100 beneficiaries per year), the adjusted 90th:10th percentile ratio of low-value service rates within region was 2.10 services per 100 beneficiaries per year, with a 10th percentile of 16.4 services per 100 beneficiaries per year; the adjusted 90th:10th percentile ratio of low-value service rates within organization was 1.67 services per 100 beneficiaries per year, with a 10th percentile of 19.1 services per 100 beneficiaries per year (eFigure 1 in the Supplement). Physician characteristics accounted for 4.6% of within-region variance and 1.8% of within-organization variance.
In this study of low-value services received by Medicare beneficiaries, we observed substantial variation among primary care physicians, of which a small fraction was attributable to observable physician characteristics. Even within the same provider organization, low-value service use varied substantially among physicians. Adjusted rates of low-value services for patients served by physicians at an organization’s 90th percentile were 61% greater than those for patients served by physicians at the organization’s 10th percentile. Adjustment for patient’s comorbidities had minimal effect on this variation, suggesting that differences in low-value service use between patients served by different physicians were not primarily driven by differences in clinical needs. These findings of extensive physician-level variation not attributable to patient, organizational, or regional factors suggest that physician practice patterns may contribute substantially to overuse. Accordingly, strategies to identify and remediate wasteful practices at the physician level may be effective for provider organizations participating in alternative payment models.
We also observed substantial rates of low-value services even among physicians whose patients received the fewest low-value services. In a typical region, patients served by physicians at the 10th percentile of low-value service use still received more than 20 low-value services per 100 patients annually. Thus, strategies to reduce levels of overuse among all physicians may achieve greater gains than strategies relying on existing variation to identify and focus on physicians exhibiting the most wasteful practice patterns.
Extending prior work,5 we demonstrated substantial variation in rates of low-value services across different provider organizations even after accounting for variation across physicians. Despite the considerable variation we observed across physicians within the same organization, the variation across organizations in the use of low-value services was much greater than would be expected due to chance. This finding is consistent with the hypothesis that provider organizations shape and/or select for the practice patterns of affiliated physicians, which is a premise of organization-focused reforms such as accountable care organizations. Possible mechanisms include differences across organizations in recruitment, training, or compensation of physicians or in practice guidelines, environment, and management of low-value care specifically.
Although we observed associations between several physician characteristics and rates of low-value services, the associations were weak and accounted for a minimal fraction of the physician variation we observed. This finding suggests potential advantages of directly assessing physician performance on measures of low-value care rather than on their personal characteristics. Indeed, only 1 physician characteristic, greater size of patient panel, predicted substantially higher rates of low-value services. This association must be interpreted with caution, in part because our estimates of panel size, based on visits, may have been inflated for physicians more eager to schedule follow-up appointments. Furthermore, because physician characteristics are not randomly distributed among physicians, associations between characteristics and low-value services should not be interpreted causally.
Our study had several additional limitations. First, the observational design did not support firm causal conclusions about the amount of low-value service use owing to physicians. If patients were randomized to physicians and physicians were randomized to provider organizations, we may have found different magnitudes of variation across physicians in rates of low-value service use. For example, some proportion of the variation we observed could be attributable to differences in patient characteristics such as preferences regarding the receipt of low-value services. However, prior research has indicated that patient preferences are unlikely to be a strong driver of overuse,5,48,49 and many of the services we studied are not likely to be familiar to patients. Although unmeasured patient clinical characteristics could explain some proportion of the variation we observed, our results were similar with and without adjustment for patient chronic conditions. Furthermore, we isolated physician variation within regions or organizations, accounting for systematic differences in patient populations that we could not observe. Although unmeasured differences in patients across physicians could lead to overestimation of physician variation, the tendency of physicians with similar practice styles to work in similar organizations or regions, (ie, homophily) could lead to underestimation of physician variation.
Second, our study examined only one dimension of quality of care: overuse. The overall value of services delivered by a physician depends on additional dimensions such as diagnostic accuracy and provision of necessary treatments. Furthermore, physicians with high use of the measured services may or may not be consistently high users of other low-value services. Our estimates of variation do not support conclusions regarding the strength of association between physicians’ use of different low-value services.
Third, some high-value services may have been misclassified as low-value in our study. This error would not have induced bias in our estimates if the misclassification were random across physicians. However, bias would have resulted if the misclassification frequency was greater among certain physicians or among physicians with certain characteristics. Fourth, low-value services may have been misattributed to a primary care physician if they were ordered or requested by physicians (eg, specialists) other than the primary care physician. However, sensitivity analyses focusing on tests and studies more likely to be ordered by primary care physicians produced estimates of variation that were similar to our main results. Moreover, even if physicians other than the patients’ attributed primary care physicians were responsible for some services, one could still reasonably interpret the variation we measured as driven by physician-level factors.
New financial incentives for provider organizations to reduce spending have spurred efforts to discourage low-value service use and to identify more and less wasteful organizations. Our study suggests that physician practices may drive much of low-value care and that low-value practices are prevalent even among the least wasteful physicians. Because physician characteristics explain little variation in low-value care, profiling physicians using direct measures of low-value care may be useful in supporting efforts to incentivize more cost-effective practices through targeted retraining of physicians or selective inclusion of physicians in networks and risk contracts.
Accepted for Publication: August 9, 2018.
Corresponding Author: Aaron L. Schwartz, MD, PhD, Department of Medicine, Brigham and Women’s Hospital, 75 Francis St, Boston, MA 02115 (firstname.lastname@example.org).
Published Online: December 3, 2018. doi:10.1001/jamainternmed.2018.5086
Author Contributions: Dr Schwartz had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Schwartz, McWilliams.
Drafting of the manuscript: Schwartz, Jena.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: All authors.
Obtained funding: Schwartz.
Administrative, technical, or material support: Schwartz.
Supervision: Jena, McWilliams.
Conflict of Interest Disclosures: Drs Schwartz and McWilliams reported being compensated as consultants by the Medicare Payment Advisory Commission in their efforts to measure low-value care in Medicare for research purposes. Dr Jena reports receiving consulting fees unrelated to this work from Pfizer, Hill Rom Services, Bristol-Myers Squibb, Novartis, Amgen, Eli Lilly, Vertex Pharmaceuticals, AstraZeneca, Celgene, Tesaro, Sanofi-Aventis, Biogen, Precision Health Economics, and Analysis Group. No other disclosures were reported.
Funding/Support: This study was supported by grants P01 AG032952 and F30 AG044106-01A1 from the National Institute on Aging, National Institutes of Health.
Role of the Funder/Sponsor: The funder/sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.