Association of Low-Value Care Exposure With Health Care Experience Ratings Among Patient Panels | Health Care Quality | JAMA Internal Medicine | JAMA Network
[Skip to Navigation]
Figure 1.  Mean Percentage of Beneficiaries Receiving Specific Low-Value Services by Quintile of Low-Value Service Exposure
Mean Percentage of Beneficiaries Receiving Specific Low-Value Services by Quintile of Low-Value Service Exposure

Mean percentage of low-value service receipt is calculated by first computing, for each primary care professional (PCP) patient panel, the percentage of beneficiaries who received the service among those in the denominator population for that service, and then averaging those percentages across PCP patient panels. The first quintile represents the PCP patient panels with the least low-value care exposure and the fifth quintile represents the PCP patient panels with the most low-value care exposure. PSA indicates prostate-specific antigen; PTH, parathyroid hormone; and T3, triiodothyronine.

Figure 2.  Differences Between Consumer Assessment of Healthcare Providers and Systems (CAHPS) Scores at Levels of Low-Value Service Exposure and Overall Mean CAHPS Score
Differences Between Consumer Assessment of Healthcare Providers and Systems (CAHPS) Scores at Levels of Low-Value Service Exposure and Overall Mean CAHPS Score

Each CAHPS outcome was separately modeled with a linear regression that adjusted for age, Medicaid-Medicare dual status, highest level of education completed, overall physical health rating, and overall mental or emotional health rating and included physician-clustered SEs. Trend lines are horizontally offset by small amounts (0.8 deciles) for readability. Vertical bars indicate 95% CIs.

Table 1.  Definitions and Frequencies of Low-Value Services in the Medicare Fee-for-Service Population
Definitions and Frequencies of Low-Value Services in the Medicare Fee-for-Service Population
Table 2.  Medicare Fee-for-Service CAHPS Survey Items of Patients' Experiences With Care
Medicare Fee-for-Service CAHPS Survey Items of Patients' Experiences With Care
Table 3.  Mean Adjusted CAHPS Scores by Low-Value Service Exposurea
Mean Adjusted CAHPS Scores by Low-Value Service Exposurea
Supplement.

eAppendix 1. Identifying Primary Care Providers

eAppendix 2. Missingness of CAHPS Items

eTable 1. Percent of Missing Responses by CAHPS Item and Year

eAppendix 3. Low-Value Service Exposure Analytical Sample Flowchart

eFigure. Low-Value Service Exposure Analytical Sample Flowchart

eAppendix 4. Comparison of Denominator Descriptions Used by Sanghavi et al vs Schwartz et al

eTable 2. Comparison of Denominator Descriptions Used by Sanghavi et al vs Schwartz et al

eAppendix 5. Low-Value Service Exposure Modeling Details

eAppendix 6. Average Characteristics of PCP Patient Panels in First and Fifth Low-Value Service Exposure Composite Quintiles

eTable 3. Average Characteristics of PCP Patient Panels in First and Fifth Low-Value Service Exposure Composite Quintiles, Unweighted

eTable 4. Average Characteristics of PCP Patient Panels in First and Fifth Low-Value Service Exposure Composite Quintiles, Weighted by Panel Size

eAppendix 7. Full Regression Tables

eTable 5. Coefficients and P Values for “Fixed Effects” Sub-Step Regression by Low-Value Service

eTable 6. Coefficients and P Values for Low-Value Service Exposure Composite Model

eTable 7. Coefficients, Standard Errors, and P Values for CAHPS Linear Regressions for Non-Composite Items

eTable 8. Coefficients, Standard Errors, and P Values for CAHPS Linear Regressions for Interactions With Personal Doctor Composite

eAppendix 8. Additional Analysis: Estimating Low-Value Service Exposure Using Only More Physician-Driven Services

eTable 9. Average Adjusted CAHPS Scores by Low-Value Service Exposure, Based on Only More Physician-Driven Services

eAppendix 9. Additional Analysis: No Covariate Adjustment in CAHPS Model

eTable 10. Average Unadjusted CAHPS Scores by Low-Value Service Exposure

eAppendix 10. Description of all Code Files, With Website Address for GitLab Download

eTable 11. Files Related With Identifying Low-Value Services

eTable 12. Files Related With Identifying Primary Care Providers

eTable 13. Files Related With Applying Further Denominator Exclusions

eTable 14. Files Related With Creating Sample Summary Statistics and Adding Additional Covariates

eTable 15. Files Related With Modeling and Analysis

1.
Centers for Medicare & Medicaid Services. Medicare Advantage and prescription drug plan CAHPS (MA and PDP CAHPS). Accessed August 13, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/MCAHPS
2.
Centers for Medicare & Medicaid Services. CAHPS for ACOs. Accessed August 13, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/ACO
3.
Anhang Price  R, Elliott  MN, Zaslavsky  AM,  et al.  Examining the role of patient experience surveys in measuring health care quality.   Med Care Res Rev. 2014;71(5):522-554. doi:10.1177/1077558714541480 PubMedGoogle ScholarCrossref
4.
Manary  MP, Boulding  W, Staelin  R, Glickman  SW.  The patient experience and health outcomes.   N Engl J Med. 2013;368(3):201-203. doi:10.1056/NEJMp1211775 PubMedGoogle ScholarCrossref
5.
Lehrman  WG, Friedberg  MW.  CAHPS surveys: valid and valuable measures of patient experience.   Hastings Cent Rep. 2015;45(6):3-4. doi:10.1002/hast.507 PubMedGoogle ScholarCrossref
6.
Sirovich  BE.  How to feed and grow your health care system: comment on “The cost of satisfaction”.   Arch Intern Med. 2012;172(5):411-413. doi:10.1001/archinternmed.2012.62 PubMedGoogle ScholarCrossref
7.
Jerant  A, Fenton  JJ, Kravitz  RL,  et al.  Association of clinician denial of patient requests with patient satisfaction.   JAMA Intern Med. 2018;178(1):85-91. doi:10.1001/jamainternmed.2017.6611 PubMedGoogle ScholarCrossref
8.
Ross  JS.  Physician denial of inappropriate patient requests—what if I say no??   JAMA Intern Med. 2018;178(1):92-92. doi:10.1001/jamainternmed.2017.7367 PubMedGoogle ScholarCrossref
9.
Junewicz  A, Youngner  SJ.  Patient-satisfaction surveys on a scale of 0 to 10: improving health care, or leading it astray?   Hastings Cent Rep. 2015;45(3):43-51. doi:10.1002/hast.453 PubMedGoogle ScholarCrossref
10.
Schwartz  AL, Chernew  ME, Landon  BE, McWilliams  JM.  Changes in low-value services in year 1 of the Medicare Pioneer Accountable Care Organization Program.   JAMA Intern Med. 2015;175(11):1815-1825. doi:10.1001/jamainternmed.2015.4525 PubMedGoogle ScholarCrossref
11.
Schlesinger  M, Grob  R.  Treating, fast and slow: Americans’ understanding of and responses to low-value care.   Milbank Q. 2017;95(1):70-116. doi:10.1111/1468-0009.12246 PubMedGoogle ScholarCrossref
12.
Paterniti  DA, Fancher  TL, Cipri  CS, Timmermans  S, Heritage  J, Kravitz  RL.  Getting to “no”: strategies primary care physicians use to deny patient requests.   Arch Intern Med. 2010;170(4):381-388. doi:10.1001/archinternmed.2009.533 PubMedGoogle ScholarCrossref
13.
Anhang Price  R, Elliott  MN, Cleary  PD, Zaslavsky  AM, Hays  RD.  Should health care providers be accountable for patients’ care experiences?   J Gen Intern Med. 2015;30(2):253-256. doi:10.1007/s11606-014-3111-7 PubMedGoogle ScholarCrossref
14.
Gogineni  K, Shuman  KL, Chinn  D, Gabler  NB, Emanuel  EJ.  Patient demands and requests for cancer tests and treatments.   JAMA Oncol. 2015;1(1):33-39. doi:10.1001/jamaoncol.2014.197 PubMedGoogle ScholarCrossref
15.
Cutler  D, Skinner  JS, Stern  AD, Wennberg  D.  Physician beliefs and patient preferences: a new look at regional variation in health care spending.   Am Econ J Econ Policy. 2019;11(1):192-221. doi:10.1257/pol.20150421 PubMedGoogle ScholarCrossref
16.
Schwartz  AL, Zaslavsky  AM, Landon  BE, Chernew  ME, McWilliams  JM.  Low-value service use in provider organizations.   Health Serv Res. 2018;53(1):87-119. doi:10.1111/1475-6773.12597 PubMedGoogle ScholarCrossref
17.
Mittler  JN, Landon  BE, Fisher  ES, Cleary  PD, Zaslavsky  AM.  Market variations in intensity of Medicare service use and beneficiary experiences with care.   Health Serv Res. 2010;45(3):647-669. doi:10.1111/j.1475-6773.2010.01108.x PubMedGoogle ScholarCrossref
18.
Fenton  JJ, Jerant  AF, Bertakis  KD, Franks  P.  The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality.   Arch Intern Med. 2012;172(5):405-411. doi:10.1001/archinternmed.2011.1662 PubMedGoogle ScholarCrossref
19.
Xu  X, Buta  E, Anhang Price  R, Elliott  MN, Hays  RD, Cleary  PD.  Methodological considerations when studying the association between patient-reported care experiences and mortality.   Health Serv Res. 2015;50(4):1146-1161. doi:10.1111/1475-6773.12264 PubMedGoogle ScholarCrossref
20.
Schwartz  AL, Jena  AB, Zaslavsky  AM, McWilliams  JM.  Analysis of physician variation in provision of low-value services.   JAMA Intern Med. 2019;179(1):16-25. doi:10.1001/jamainternmed.2018.5086 PubMedGoogle ScholarCrossref
21.
Centers for Medicare & Medicaid Services. Chronic Conditions Data Warehouse. Accessed September 9, 2020. https://www.ccwdata.org/
22.
Dartmouth Institute for Health Policy & Clinical Practice. Dartmouth Atlas data. Published 2014. Accessed March 21, 2020. https://atlasdata.dartmouth.edu/static/supp_research_data#hospital-research-data
23.
Schwartz  AL, Landon  BE, Elshaug  AG, Chernew  ME, McWilliams  JM.  Measuring low-value care in Medicare.   JAMA Intern Med. 2014;174(7):1067-1076. doi:10.1001/jamainternmed.2014.1541 PubMedGoogle ScholarCrossref
24.
McWilliams  JM, Landon  BE, Chernew  ME, Zaslavsky  AM.  Changes in patients’ experiences in Medicare accountable care organizations.   N Engl J Med. 2014;371(18):1715-1724. doi:10.1056/NEJMsa1406552 PubMedGoogle ScholarCrossref
25.
Hays  RD, Mallett  JS, Haas  A,  et al.  Associations of CAHPS composites with global ratings of the doctor vary by Medicare beneficiaries’ health status.   Med Care. 2018;56(8):736-739. doi:10.1097/MLR.0000000000000942 PubMedGoogle ScholarCrossref
26.
Keenan  PS, Elliott  MN, Cleary  PD, Zaslavsky  AM, Landon  BE.  Quality assessments by sick and healthy beneficiaries in traditional Medicare and Medicare managed care.   Med Care. 2009;47(8):882-888. doi:10.1097/MLR.0b013e3181a39415 PubMedGoogle ScholarCrossref
27.
Landon  BE, Zaslavsky  AM, Bernard  SL, Cioffi  MJ, Cleary  PD.  Comparison of performance of traditional Medicare vs Medicare managed care.   JAMA. 2004;291(14):1744-1752. doi:10.1001/jama.291.14.1744 PubMedGoogle ScholarCrossref
28.
Neprash  HT, Barnett  ML.  Association of primary care clinic appointment time with opioid prescribing.   JAMA Netw Open. 2019;2(8):e1910373. doi:10.1001/jamanetworkopen.2019.10373 PubMedGoogle Scholar
29.
Neprash HT. Three Essays in Physician Behavior. Doctoral dissertation. Harvard University, Graduate School of Arts & Sciences. Published online 2017. Accessed September 8, 2020. https://dash.harvard.edu/handle/1/41140239
30.
Landon  BE, Onnela  J-P, Keating  NL,  et al.  Using administrative data to identify naturally occurring networks of physicians.   Med Care. 2013;51(8):715-721. doi:10.1097/MLR.0b013e3182977991 PubMedGoogle ScholarCrossref
31.
Barnett  ML, Landon  BE, O’Malley  AJ, Keating  NL, Christakis  NA.  Mapping physician networks with self-reported and administrative data.   Health Serv Res. 2011;46(5):1592-1609. doi:10.1111/j.1475-6773.2011.01262.x PubMedGoogle ScholarCrossref
32.
Agha  L, Ericson  KM, Geissler  K, Rebitzer  J.  Team Formation and Performance: Evidence from Healthcare Referral Networks. National Bureau of Economic Research; 2018. doi:10.3386/w24338
33.
Legal Information Institute. Number of ACO professionals and beneficiaries: title 42 part §425.110. Accessed April 12, 2021. https://www.law.cornell.edu/cfr/text/42/425.110
34.
Landon  BE, Keating  NL, Barnett  ML,  et al.  Variation in patient-sharing networks of physicians across the United States.   JAMA. 2012;308(3):265-273. doi:10.1001/jama.2012.7615 PubMedGoogle ScholarCrossref
35.
Pham  HH, Schrag  D, O’Malley  AS, Wu  B, Bach  PB.  Care patterns in Medicare and their implications for pay for performance.   N Engl J Med. 2007;356(11):1130-1139. doi:10.1056/NEJMsa063979PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 7,397
    Citations 0
    Original Investigation
    May 28, 2021

    Association of Low-Value Care Exposure With Health Care Experience Ratings Among Patient Panels

    Author Affiliations
    • 1Biological Sciences Division, Department of Public Health Sciences, The University of Chicago, Chicago, Illinois
    • 2Department of Health Care Policy, Harvard Medical School, Brigham and Women’s Hospital, Boston, Massachusetts
    • 3Division of General Internal Medicine, Department of Medical Ethics and Health Policy, Perelman School of Medicine, Philadelphia, Pennsylvania
    • 4Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
    JAMA Intern Med. 2021;181(7):941-948. doi:10.1001/jamainternmed.2021.1974
    Key Points

    Question  Do primary care professional (PCP) patient panels who receive more low-value care rate their health care experiences more favorably?

    Findings  This quality improvement study of 100 743 PCPs with a mean of 258 patients each constructed a composite score of low-value care exposure for each PCP patient panel and estimated its association with patient ratings of health care. With 1 exception (waiting room time), all observed associations between low-value care exposure and health care experience ratings (overall health care, timely access to nonurgent care, timely access to urgent care, personal physician, and interactions with personal physician) were small and/or lacked statistical significance.

    Meaning  This study did not find evidence that more low-value care for a PCP patient panel was associated with more favorable patient ratings of their health care experiences.

    Abstract

    Importance  Patient reviews of health care experiences are increasingly used for public reporting and alternative payment models. Critics have argued that this incentivizes physicians to provide more care, including low-value care, undermining efforts to reduce wasteful practices.

    Objective  To assess associations between rates of low-value service provision to a primary care professional (PCP) patient panel and patients’ ratings of their health care experiences.

    Design, Setting, and Participants  This quality improvement study used Medicare fee-for-service claims from January 1, 2007, to December 31, 2014, for a random 20% sample of beneficiaries to identify beneficiaries for whom each of 8 low-value services could be ordered but would be considered unnecessary. The study also used health care experience reports from independently sampled beneficiaries who responded to the 2010-2015 Consumer Assessment of Healthcare Providers and Systems (CAHPS) Medicare fee-for-service survey. Statistical analysis was performed from January 1, 2019, to December 9, 2020.

    Main Outcomes and Measures  The main outcomes were health care experience ratings from Medicare beneficiaries who responded to the CAHPS survey from 2 domains, namely “Your Health Care in the Last 6 Months” (overall health care, office wait time, timely access to nonurgent care, and timely access to urgent care) and “Your Personal Doctor” (overall personal physician and a composite score for interactions with personal physician). Beneficiaries in both samples were attributed to the PCP with whom they had the most spending. For each PCP, a composite score of low-value service exposure was constructed using the 20% sample; this score represented the adjusted relative propensity of the PCP patient panel to receive low-value care. The association between low-value service exposure and health care experience ratings reported by the CAHPS respondents in the PCP patient panel was estimated using regression analysis.

    Results  The final sample had 100 743 PCPs, with a mean of approximately 258 patients per PCP. Only 1 notable association was found; more low-value care exposure was associated with more frequent reports of having to wait more than 15 minutes after the scheduled time of an appointment (a mean of 0.448 points lower CAHPS score on a 10-point scale for PCP patient panels who received the most low-value care vs the least low-value care). Although some other associations were statistically significant, their magnitudes were substantially smaller than those typically considered meaningful in other CAHPS literature and were inconsistent in direction across levels of low-value service exposure.

    Conclusions and Relevance  This quality improvement study found that more low-value care exposure for a PCP patient panel was not associated with more favorable patient ratings of their health care experiences.

    Introduction

    Patient-reported health care experiences are widely used to incentivize quality improvement through public reporting and performance-based payments. For example, the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey measures of patient experiences are included in Medicare Advantage plan star ratings1 and the Medicare Shared Savings Program accountable care organization contracts.2 These ratings measure dimensions of care that are best reported by patients, such as physician communication and timeliness. They are empirically associated with other measures of clinical process, outcomes, efficiency, and safety.3-5 A concern about this practice is that it may encourage physicians to provide more low-value services (care that is not associated with a clinical benefit), out of a belief that responding to patient demand or the perception that more care is better will improve their ratings.6-9 This may lead to the wasteful use of health care resources and spending, possible iatrogenic injury, and limited success of alternate payment models such as accountable care organizations.10

    In this study, we directly address the question: Do patients of physicians who provide more low-value care rate their health care experiences more favorably? (We use the terms receipt of, exposure to, or provision of low-value care interchangeably, regardless of how or by whom it was initiated.) More low-value care may be interpreted by patients as a signal of better or worse care depending on patient trust and preferences, as well as physician communication of rationales.3,11-13 It remains unclear whether patients are advocating for low-value service provision.10,11,14-17 However, as long as physicians may be acting out of a concern about patient dissatisfaction, it is imperative to address this question to gain physician buy-in to policies for curbing waste.

    One study that garnered national attention found that patients who reported most favorably on their care and physician communication had a higher mortality risk, implying that catering to patient satisfaction may lead to worse outcomes.18 However, the additional attention that patients with severe illness receive may lead them to rate their access and communication with physicians more favorably than would healthier patients with fewer care needs. This scenario may induce a positive correlation between patient experiences and mortality that is actually a reflection of patient factors, not the associations of physician practices with patient experiences. The limitation of the study design that made it vulnerable to this potentially flawed inference is its reliance on data from the same unique patient for both the patient report and the outcome.3,4,6,19

    In the context of our research question, similar confounding due to patient characteristics can arise if such factors are associated with both receipt of low-value care and health care experiences. For example, some patients may have more opportunities to receive low-value care than other patients in similar states of health because they value health care more and thus seek care more frequently. Likewise, patients who comply with physician orders more often are likely to receive more services. Patients with such stronger preferences or adherence may also appreciate their care and physicians more. Consequently, receipt of low-value care may be associated with more favorable reports on care experiences from the same patients even when physicians’ low-value practice patterns are not associated with their patients’ care experiences, and even when patients do not differ between physicians.

    We designed a study that eliminates this source of patient-level confounding. We used a 20% random sample of the full fee-for-service Medicare population to assess low-value care exposure for a primary care professional (PCP) patient panel and a much smaller, independent sample from the CAHPS survey to measure patient experiences. Because the 2 samples were independent and overlapped minimally, our analysis did not rely on observations about the same patients to assess associations. Although our study remains subject to unmeasured confounding arising from unobserved systematic sorting of patients to different PCPs, our methods permit stronger conclusions about the associations between physician provision of wasteful care and care experiences than approaches taken in prior studies.

    Methods
    Data Sources

    To assess provision of low-value services, we analyzed claims and enrollment data from January 1, 2007, to December 31, 2014, for a 20% simple random sample of fee-for-service Medicare beneficiaries, with the first year serving as a look-back period for 2008 claims. We used hospital inpatient data, outpatient claims, and noninstitutional claims for services such as physician visits. For a given year, we required beneficiaries to be continuously enrolled in Medicare Parts A and B in that year (while alive) and the previous year. Eligible beneficiaries also had at least 1 claim for primary care services (Healthcare Common Procedure Coding System codes 99201-99215, G0402, G0438, and G0439) provided by a PCP, defined by specialty codes for general practice, family practice, internal medicine, or geriatric medicine (eAppendix 1 in the Supplement); we attributed beneficiaries to the PCP with whom they had the most spending during the year.20 The research protocol was approved by institutional review boards at Harvard Medical School, the University of Chicago, and the National Bureau of Economic Research with a waiver of informed consent according to CFR §46.116 (e)(3)(ii) because the research could not practicably be carried out without the waiver.

    We assessed patient experiences with data from the 2010-2015 fee-for-service Medicare CAHPS surveys. We assigned each CAHPS respondent to a PCP using the approach described but linked to a PCP in the prior year because the CAHPS survey is administered near the beginning of the year and asks about the prior 6 months. The overall CAHPS survey response rate was 41.9% during these years. The nonresponse rates to items in the CAHPS ranged from 3% to 67%, almost entirely owing to inapplicability of the item to a respondent who did not have relevant experience (eAppendix 2 and eTable 1 in the Supplement). Because the CAHPS sample was only 1.5% the size of the claims sample used to assess low-value service provision, the overlap between the 2 samples was minimal.

    We excluded PCPs with fewer than 11 patients in the 20% sample (in compliance with our data use agreement with the Centers for Medicare & Medicaid Services) or without at least 1 attributed CAHPS respondent (eAppendix 3 and eFigure in the Supplement). Clinical covariates were extracted from the Chronic Conditions Data Warehouse,21 which tracks diagnoses of 27 conditions from 1999 or a beneficiary’s first year of Medicare enrollment onward, zip code–level sociodemographic data from the American Community Survey in the same years as our claims data, and data on hospital referral regions from the Dartmouth Atlas.22

    Study Variables
    Low-Value Service Exposure for PCP Patient Panels

    To create measures of exposure to low-value care for each PCP-attributed patient panel in the 20% samples, we started by identifying episodes of low-value care at the patient level. We adapted methods developed by Schwartz et al16,23 for identifying low-value care in Medicare claims data and selected the 8 services that were most frequently used (Table 1). In addition, for each service, we defined and identified a denominator population among whom the service would be considered unnecessary (eAppendix 4 and eTable 2 in the Supplement). Although the services are often ordered by PCPs, our approach did not require that the PCP provided or ordered the low-value service; rather, we assessed low-value service exposure from all physicians of a PCP patient panel. The contribution of a PCP’s network of specialists to the provision of low-value care was compatible with several dimensions of care experiences examined in our analysis. For example, overall care ratings are not PCP specific, and overall PCP ratings may reflect patients’ valuation of specialty referrals made by their PCP.

    We aggregated the beneficiary-level indicators of low-value service receipt into composite measures of exposure at the level of a PCP-attributed patient panel using a multivariate, multilevel model with the patients nested within the PCPs: logit Pr (LVS_Receiptijst = 1) = αj+Xijst′βs and αj ~ N(0, σ2), where LVS_Receiptijst is the binary indicator of whether patient i of PCP j received low-value service s in year t, and Xijst is a vector of patient- and geography-related covariates (zip code–level covariates and hospital referral region) and year indicators. Exposure to low-value care for the patient panel of PCP j is represented by the random intercept αj, which represents the relative propensity to receipt of low-value services among the patients in the PCP patient panel, after controlling for the vector of covariates in Xijst, and accounts for sampling variation in the number of patients and potential services per PCP (eAppendix 5 in the Supplement).

    CAHPS Measures

    We focused on 9 items in 2 sections of the CAHPS Medicare fee-for-service survey: “Your Health Care in the Last 6 Months” and “Your Personal Doctor,” defined as the “one you would see if you need a checkup, want advice about a health problem, or get sick or hurt” (Table 2). In the former section, we studied responses to questions about overall health care, waiting room times, and timely access to nonurgent and urgent care. In the latter, we assessed ratings of the personal physician overall and interactions with the personal physician. All of these items are commonly studied in the CAHPS literature24,25 and could conceivably be associated with provision of low-value care.

    All responses were linearly rescaled to a 0 to 10 scale for consistency in presentation of results (Table 2). We created patient-level composite scores for respondents’ interactions with their personal physician by averaging across standardized responses (ie, by first subtracting the grand item mean from individual responses), averaging across de-meaned responses, and adding back the mean of all grand item means. We did not expect bias from different nonresponse among these items because 98.8% of patients who responded to at least 1 of these items responded to all of the items.

    Covariates

    Patient covariates in the model estimating low-value service exposure included age, sex, race/ethnicity, Medicaid-Medicare dual status in at least 1 month, and indicators for 27 chronic conditions. To further adjust for health status, we added an indicator of having 6 or more chronic conditions and also used the claims data to calculate Hierarchical Condition Category risk scores, which indicate higher future Medicare spending with higher scores. Zip code–level sociodemographic variables included median household income, percentage of patients in poverty, and percentage of patients with a college education.

    For the main analysis of an association between CAHPS scores and low-value service exposure, we included the following patient variables from the CAHPS survey: age as a categorical variable (<65, 66-69, 70-74, 75-79, 80-84, and ≥85 years), Medicaid-Medicare dual eligibility status, highest level of education completed (less than high school, some high school, some college, college graduate, and above college), and self-reported overall physical health and overall mental or emotional health (poor, fair, good, very good, or excellent).

    Statistical Analysis

    Statistical analysis was performed from January 1, 2019, to December 9, 2020. We categorized PCPs into deciles of the low-value service composite for their attributed patients, from least (decile 1) to most (decile 10) low-value care exposure. We first regressed CAHPS scores on decile indicators and patient covariates, with clustered standard errors to account for patient grouping within physicians, for a 9-df omnibus test of any differences across deciles. Our second analysis regressed CAHPS scores on the numerical decile index and covariates to test for a linear trend across deciles. To further describe trends, for each CAHPS model, we conducted statistical tests of the difference between the mean adjusted CAHPS score in each decile of the low-value service exposure composite and the overall adjusted mean CAHPS score. All statistical tests were 2-sided and conducted at an α level of .05.

    Results

    Approximately 26 million beneficiary-years were used to create the low-value service composites for 100 743 PCP patient panels, with a mean of approximately 258 patients per PCP. Depending on the CAHPS item, there were between 135 657 and 330 600 respondents, ranging from 2 to 3.4 per PCP.

    In outpatient facility claims for low-value services, the PCP was listed on 41.8% of services, as the attending physician (31.4%), operating physician (2.9%), or other physician (7.5%). Among professional claims for low-value services, the PCP was listed on 45.6% of services, either as the performing physician (10.0%) or referring physician (35.6%).

    We observed substantial variation in low-value care exposure across patient panels, consistent with prior studies.20 Figure 1 shows that the low-value service exposure composites are positively associated with use in each of the 8 services. We did not find meaningful differences between patient panels in the first and fifth quintiles of low-value service exposure (eAppendix 6 and eTables 3 and 4 in the Supplement).

    For each CAHPS outcome, Figure 2 plots differences between the mean adjusted CAHPS score for each decile of the low-value service exposure measure and the overall mean adjusted CAHPS score across deciles. With the exception of waiting room time, trend coefficients were bounded to be very close to zero. The waiting room time item had a clear downward trend, with lower CAHPS scores (reporting more frequently late start of appointments) in the deciles of patients receiving the most low-value care. In particular, PCP patient panels who received the most low-value care responded with a CAHPS score that was a mean of 0.448 points lower than those who received the least low-value care, a difference 7.7 times larger than the maximum difference observed between any 2 deciles in any other outcome.

    Table 3 summarizes the mean adjusted CAHPS scores across deciles of the low-value service exposure measure. With the exception of waiting room time, mean adjusted scores in each decile were above 8.5, with a maximum between-decile difference of 0.058, on a 10-point scale. A test of linear trend across the deciles found significant trends of CAHPS scores for 5 of 6 measures (compared with only 3 for the categorical specification). However, excluding waiting room time, the largest-magnitude coefficient indicated that a 1 higher decile of low-value service exposure would increase the CAHPS outcome by a mean of only 0.005 points, adjusting for other covariates. We also provide full regression tables (eAppendix 7 and eTables 5-8 in the Supplement), sensitivity analyses (eAppendices 8 and 9 and eTables 9 and 10 in the Supplement), and description of all code files (eAppendix 10 and eTables 11-15 in the Supplement).

    Discussion

    We assessed the associations between PCP patient panels’ levels of exposure to low-value care and health care experience ratings. Across multiple items from CAHPS sections on “Your Health Care in the Last 6 Months” and “Your Personal Doctor,” we did not find that exposure to more low-value care was associated with better health care experience ratings. Although some associations were statistically significant, their magnitudes were substantially smaller than those typically considered meaningful in other CAHPS literature26,27 and inconsistent in direction across levels of low-value service exposure. These findings challenge the claim that use of patient-reported experiences to measure quality of care may lead to unnecessary care and possibly iatrogenic injury and death.6-9

    The only notable association was that patient panels with higher exposure to low-value care reported more frequently having behind-schedule appointments. Specifically, patient panels with the highest low-value care exposure rated this item 0.448 points lower on a 10-point scale than those with the lowest low-value care exposure, a difference substantially larger than typically observed in comparisons of Medicare plans.26 One possible interpretation is that poorly organized or overwhelmed practices substitute wasteful services for higher-value services that require more cognitive effort and clinician time.28,29

    Our study design, however, was correlational and did not support conclusions about causal mechanisms underlying any associations between low-value service provision and patient experience reports. It improves on the prior literature, which has been characterized by patient-level endogeneity from using the same individuals to measure both patient experiences and health outcomes or health care use.7,18 For the research question our study addressed, such an analysis may potentially misattribute to receipt of low-value care what is actually an association of unobserved patient characteristics, such as preferences for amount and type of care; socioeconomic factors, such as educational level, income, and leisure time; and trust in their physician and the health care system more broadly. Our study design reduced this type of patient-level confounding by using independent samples, instead of the same sample, to assess low-value care provision and health care experiences. Furthermore, because the CAHPS sample was much smaller than the claims sample, there was little overlap between the 2 samples. In other words, patient-level confounding was necessarily limited as the characteristics of a CAHPS respondent were at most weakly associated with the mean characteristics of the PCP patient panel. The large claims sample also made possible a more precise estimation of low-value care exposure, and the variation in the estimated exposure supported informative comparisons of health care experiences between higher and lower levels of low-value care provision.

    Limitations

    This study has limitations. First, we relied on claims data to identify diagnoses and procedures, which may not be as complete or accurate as medical records. Second, our analysis does not address systematic, unobserved differences in patient panels between physicians. For example, patients may sort to physicians based on the consonance between their preferences and a physician’s practice style. Third, although the CAHPS response rate was 41.9%, our interest was in the reports of patients who respond to such surveys. This response rate is typical of surveys of this type, and the respondents are likely representative of those who would respond to other patient surveys.

    Fourth, to measure low-value service exposure, we counted all low-value services received by patients in a PCP patient panel, thereby attributing the provision of low-value care to the network of physicians who provided care to the PCP patient panel. Such PCP-centric models for identifying and establishing care networks are commonly used by researchers and policy makers.30-35 Because only a share of low-value services were provided by a patient’s PCP, associations between low-value care provision and PCP-specific patient ratings may have been attenuated toward the null. However, we found similarly weak associations for CAHPS survey items associated with health care more broadly, including patients’ overall care rating, which may reflect care provided by all physicians involved in the patient’s care.

    Conclusions

    We did not find evidence that patients who are exposed to more low-value care rate their health care experiences better. They also generally do not rate their health care worse, which may be because they are unable to assess the value of care or because their physicians effectively correct information asymmetries through shared decision-making when denying low-value services. These findings are inconsistent with some highly cited works,7,18 likely in part because our methodological approach is not subject to the same levels of patient-level confounding. Knowing this may help reduce the use of low-value care that is provided to appease patients who would be equally satisfied with less wasteful care and help alleviate concerns that patient dissatisfaction should inhibit waste reduction under alternative payment models.

    Back to top
    Article Information

    Accepted for Publication: March 26, 2021.

    Published Online: May 28, 2021. doi:10.1001/jamainternmed.2021.1974

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Sanghavi P et al. JAMA Internal Medicine.

    Corresponding Author: Prachi Sanghavi, PhD, Biological Sciences Division, Department of Public Health Sciences, The University of Chicago, 5841 S Maryland Ave, MC2000, Chicago, IL 60637 (psanghavi@uchicago.edu).

    Author Contributions: Dr Sanghavi had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Sanghavi, McWilliams, Zaslavsky.

    Acquisition, analysis, or interpretation of data: Sanghavi, McWilliams, Schwartz.

    Drafting of the manuscript: Sanghavi, Zaslavsky.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Sanghavi, McWilliams, Zaslavsky.

    Obtained funding: Sanghavi, McWilliams, Zaslavsky.

    Administrative, technical, or material support: Sanghavi, Schwartz.

    Supervision: Sanghavi, McWilliams.

    Conflict of Interest Disclosures: Dr McWilliams reported serving as an unpaid member of the board of directors for the Institute for Accountable Care and receiving personal fees from the Medicare Payment Advisory Commission for serving as a consultant on the application of low-value care measures, outside the submitted work. Dr Schwartz reported receiving personal fees from Lown Institute, CVS Health, and the Medicare Payment Advisory Commission outside the submitted work. No other disclosures were reported.

    Funding/Support: This work was supported by a grant from the National Institute for Health Care Management Foundation and grant P01AG032952 from the National Institute on Aging.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: Lin Ding, PhD, Harvard Medical School, provided programming support for this study. She was not explicitly compensated for this contribution.

    References
    1.
    Centers for Medicare & Medicaid Services. Medicare Advantage and prescription drug plan CAHPS (MA and PDP CAHPS). Accessed August 13, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/MCAHPS
    2.
    Centers for Medicare & Medicaid Services. CAHPS for ACOs. Accessed August 13, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/ACO
    3.
    Anhang Price  R, Elliott  MN, Zaslavsky  AM,  et al.  Examining the role of patient experience surveys in measuring health care quality.   Med Care Res Rev. 2014;71(5):522-554. doi:10.1177/1077558714541480 PubMedGoogle ScholarCrossref
    4.
    Manary  MP, Boulding  W, Staelin  R, Glickman  SW.  The patient experience and health outcomes.   N Engl J Med. 2013;368(3):201-203. doi:10.1056/NEJMp1211775 PubMedGoogle ScholarCrossref
    5.
    Lehrman  WG, Friedberg  MW.  CAHPS surveys: valid and valuable measures of patient experience.   Hastings Cent Rep. 2015;45(6):3-4. doi:10.1002/hast.507 PubMedGoogle ScholarCrossref
    6.
    Sirovich  BE.  How to feed and grow your health care system: comment on “The cost of satisfaction”.   Arch Intern Med. 2012;172(5):411-413. doi:10.1001/archinternmed.2012.62 PubMedGoogle ScholarCrossref
    7.
    Jerant  A, Fenton  JJ, Kravitz  RL,  et al.  Association of clinician denial of patient requests with patient satisfaction.   JAMA Intern Med. 2018;178(1):85-91. doi:10.1001/jamainternmed.2017.6611 PubMedGoogle ScholarCrossref
    8.
    Ross  JS.  Physician denial of inappropriate patient requests—what if I say no??   JAMA Intern Med. 2018;178(1):92-92. doi:10.1001/jamainternmed.2017.7367 PubMedGoogle ScholarCrossref
    9.
    Junewicz  A, Youngner  SJ.  Patient-satisfaction surveys on a scale of 0 to 10: improving health care, or leading it astray?   Hastings Cent Rep. 2015;45(3):43-51. doi:10.1002/hast.453 PubMedGoogle ScholarCrossref
    10.
    Schwartz  AL, Chernew  ME, Landon  BE, McWilliams  JM.  Changes in low-value services in year 1 of the Medicare Pioneer Accountable Care Organization Program.   JAMA Intern Med. 2015;175(11):1815-1825. doi:10.1001/jamainternmed.2015.4525 PubMedGoogle ScholarCrossref
    11.
    Schlesinger  M, Grob  R.  Treating, fast and slow: Americans’ understanding of and responses to low-value care.   Milbank Q. 2017;95(1):70-116. doi:10.1111/1468-0009.12246 PubMedGoogle ScholarCrossref
    12.
    Paterniti  DA, Fancher  TL, Cipri  CS, Timmermans  S, Heritage  J, Kravitz  RL.  Getting to “no”: strategies primary care physicians use to deny patient requests.   Arch Intern Med. 2010;170(4):381-388. doi:10.1001/archinternmed.2009.533 PubMedGoogle ScholarCrossref
    13.
    Anhang Price  R, Elliott  MN, Cleary  PD, Zaslavsky  AM, Hays  RD.  Should health care providers be accountable for patients’ care experiences?   J Gen Intern Med. 2015;30(2):253-256. doi:10.1007/s11606-014-3111-7 PubMedGoogle ScholarCrossref
    14.
    Gogineni  K, Shuman  KL, Chinn  D, Gabler  NB, Emanuel  EJ.  Patient demands and requests for cancer tests and treatments.   JAMA Oncol. 2015;1(1):33-39. doi:10.1001/jamaoncol.2014.197 PubMedGoogle ScholarCrossref
    15.
    Cutler  D, Skinner  JS, Stern  AD, Wennberg  D.  Physician beliefs and patient preferences: a new look at regional variation in health care spending.   Am Econ J Econ Policy. 2019;11(1):192-221. doi:10.1257/pol.20150421 PubMedGoogle ScholarCrossref
    16.
    Schwartz  AL, Zaslavsky  AM, Landon  BE, Chernew  ME, McWilliams  JM.  Low-value service use in provider organizations.   Health Serv Res. 2018;53(1):87-119. doi:10.1111/1475-6773.12597 PubMedGoogle ScholarCrossref
    17.
    Mittler  JN, Landon  BE, Fisher  ES, Cleary  PD, Zaslavsky  AM.  Market variations in intensity of Medicare service use and beneficiary experiences with care.   Health Serv Res. 2010;45(3):647-669. doi:10.1111/j.1475-6773.2010.01108.x PubMedGoogle ScholarCrossref
    18.
    Fenton  JJ, Jerant  AF, Bertakis  KD, Franks  P.  The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality.   Arch Intern Med. 2012;172(5):405-411. doi:10.1001/archinternmed.2011.1662 PubMedGoogle ScholarCrossref
    19.
    Xu  X, Buta  E, Anhang Price  R, Elliott  MN, Hays  RD, Cleary  PD.  Methodological considerations when studying the association between patient-reported care experiences and mortality.   Health Serv Res. 2015;50(4):1146-1161. doi:10.1111/1475-6773.12264 PubMedGoogle ScholarCrossref
    20.
    Schwartz  AL, Jena  AB, Zaslavsky  AM, McWilliams  JM.  Analysis of physician variation in provision of low-value services.   JAMA Intern Med. 2019;179(1):16-25. doi:10.1001/jamainternmed.2018.5086 PubMedGoogle ScholarCrossref
    21.
    Centers for Medicare & Medicaid Services. Chronic Conditions Data Warehouse. Accessed September 9, 2020. https://www.ccwdata.org/
    22.
    Dartmouth Institute for Health Policy & Clinical Practice. Dartmouth Atlas data. Published 2014. Accessed March 21, 2020. https://atlasdata.dartmouth.edu/static/supp_research_data#hospital-research-data
    23.
    Schwartz  AL, Landon  BE, Elshaug  AG, Chernew  ME, McWilliams  JM.  Measuring low-value care in Medicare.   JAMA Intern Med. 2014;174(7):1067-1076. doi:10.1001/jamainternmed.2014.1541 PubMedGoogle ScholarCrossref
    24.
    McWilliams  JM, Landon  BE, Chernew  ME, Zaslavsky  AM.  Changes in patients’ experiences in Medicare accountable care organizations.   N Engl J Med. 2014;371(18):1715-1724. doi:10.1056/NEJMsa1406552 PubMedGoogle ScholarCrossref
    25.
    Hays  RD, Mallett  JS, Haas  A,  et al.  Associations of CAHPS composites with global ratings of the doctor vary by Medicare beneficiaries’ health status.   Med Care. 2018;56(8):736-739. doi:10.1097/MLR.0000000000000942 PubMedGoogle ScholarCrossref
    26.
    Keenan  PS, Elliott  MN, Cleary  PD, Zaslavsky  AM, Landon  BE.  Quality assessments by sick and healthy beneficiaries in traditional Medicare and Medicare managed care.   Med Care. 2009;47(8):882-888. doi:10.1097/MLR.0b013e3181a39415 PubMedGoogle ScholarCrossref
    27.
    Landon  BE, Zaslavsky  AM, Bernard  SL, Cioffi  MJ, Cleary  PD.  Comparison of performance of traditional Medicare vs Medicare managed care.   JAMA. 2004;291(14):1744-1752. doi:10.1001/jama.291.14.1744 PubMedGoogle ScholarCrossref
    28.
    Neprash  HT, Barnett  ML.  Association of primary care clinic appointment time with opioid prescribing.   JAMA Netw Open. 2019;2(8):e1910373. doi:10.1001/jamanetworkopen.2019.10373 PubMedGoogle Scholar
    29.
    Neprash HT. Three Essays in Physician Behavior. Doctoral dissertation. Harvard University, Graduate School of Arts & Sciences. Published online 2017. Accessed September 8, 2020. https://dash.harvard.edu/handle/1/41140239
    30.
    Landon  BE, Onnela  J-P, Keating  NL,  et al.  Using administrative data to identify naturally occurring networks of physicians.   Med Care. 2013;51(8):715-721. doi:10.1097/MLR.0b013e3182977991 PubMedGoogle ScholarCrossref
    31.
    Barnett  ML, Landon  BE, O’Malley  AJ, Keating  NL, Christakis  NA.  Mapping physician networks with self-reported and administrative data.   Health Serv Res. 2011;46(5):1592-1609. doi:10.1111/j.1475-6773.2011.01262.x PubMedGoogle ScholarCrossref
    32.
    Agha  L, Ericson  KM, Geissler  K, Rebitzer  J.  Team Formation and Performance: Evidence from Healthcare Referral Networks. National Bureau of Economic Research; 2018. doi:10.3386/w24338
    33.
    Legal Information Institute. Number of ACO professionals and beneficiaries: title 42 part §425.110. Accessed April 12, 2021. https://www.law.cornell.edu/cfr/text/42/425.110
    34.
    Landon  BE, Keating  NL, Barnett  ML,  et al.  Variation in patient-sharing networks of physicians across the United States.   JAMA. 2012;308(3):265-273. doi:10.1001/jama.2012.7615 PubMedGoogle ScholarCrossref
    35.
    Pham  HH, Schrag  D, O’Malley  AS, Wu  B, Bach  PB.  Care patterns in Medicare and their implications for pay for performance.   N Engl J Med. 2007;356(11):1130-1139. doi:10.1056/NEJMsa063979PubMedGoogle ScholarCrossref
    ×