Development of a Metric to Detect and Decrease Low-Value Prescribing in Older Adults

Key Points

Question  Can a scalable and broadly applicable metric be developed using common forms of health care data to detect low-value prescribing among older adults?

Findings  In this qualitative study of 27 low-value prescribing practices, a panel of experts was convened using an online modified-Delphi approach to develop a scientifically valid and clinically useful low-value prescribing metric. This panel identified the 18 most salient low-value prescribing practices, which were included as a component of the final metric.

Meaning  The metric developed in this study may enhance the detection of low-value prescribing practices, reduce polypharmacy, and enable older adults to receive high-value care.

Importance  Metrics that detect low-value care in common forms of health care data, such as administrative claims or electronic health records, primarily focus on tests and procedures but not on medications, representing a major gap in the ability to systematically measure low-value prescribing.

Objective  To develop a scalable and broadly applicable metric that contains a set of quality indicators (EVOLV-Rx) for use in health care data to detect and reduce low-value prescribing among older adults and that is informed by diverse stakeholders’ perspectives.

Design, Setting, and Participants  This qualitative study used an online modified-Delphi method to convene an expert panel of 15 physicians and pharmacists. This panel, comprising clinicians, health system leaders, and researchers, was tasked with rating and discussing candidate low-value prescribing practices that were derived from medication safety criteria; peer-reviewed literature; and qualitative studies of patient, caregiver, and physician perspectives. The RAND ExpertLens online platform was used to conduct the activities of the panel. The panelists were engaged for 3 rounds between January 1 and March 31, 2021.

Main Outcomes and Measures  Panelists used a 9-point Likert scale to rate and then discuss the scientific validity and clinical usefulness of the criteria to detect low-value prescribing practices. Candidate low-value prescribing practices were rated as follows: 1 to 3, indicating low validity or usefulness; 3.5 to 6, uncertain validity or usefulness; and 6.5 to 9, high validity or usefulness. Agreement among panelists and the degree of scientific validity and clinical usefulness were assessed using the RAND/UCLA (University of California, Los Angeles) Appropriateness Method.

Results  Of the 527 low-value prescribing recommendations identified, 27 discrete candidate low-value prescribing practices were considered for inclusion in EVOLV-Rx. After round 1, 18 candidate practices were rated by the panel as having high scientific validity and clinical usefulness (scores of ≥6.5). After round 2 panel deliberations, the criteria to detect 19 candidate practices were revised. After round 3, 18 candidate practices met the inclusion criteria, receiving final median scores of 6.5 or higher for both scientific validity and clinical usefulness. Of those practices that were not included in the final version of EVOLV-Rx, 3 received high scientific validity (scores ≥6.5) but uncertain clinical usefulness (scores <6.5) ratings, whereas 6 received uncertain scientific validity rating (scores <6.5).

Conclusions and Relevance  This study culminated in the development of EVOLV-Rx and involved a panel of experts who identified the 18 most salient low-value prescribing practices in the care of older adults. Applying EVOLV-Rx may enhance the detection of low-value prescribing practices, reduce polypharmacy, and enable older adults to receive high-value care across the full spectrum of health services.

Low-value care, which is defined as the use of health services whose harms or costs exceed their benefits, is a major factor in wasteful health care spending and has been associated with physical, psychological, and financial harms.^1,2 In the US, more than $100 billion per year has been spent on the delivery of low-value care, affecting up to 43% of Medicare beneficiaries.^1,3 However, low-value care metrics primarily consist of tests and procedures and exclude the low-value prescribing of medications. This exclusion represents a major gap in the ability to systematically detect low-value prescribing and low-value care across the full spectrum of health services.^3,4

Characterizing and reducing low-value prescribing are especially important for individuals who are 65 years or older. More than 40% of older adults have been subjected to polypharmacy (prescribed ≥5 medications), and 30% to 50% have been prescribed a potentially inappropriate medication, increasing their risk for adverse drug events and hospitalizations.^5,6 In addition, 35% of Medicare Part D beneficiaries have experienced hardship in paying for their medications.^6,7

There is currently no metric that consolidates and prioritizes the hundreds of recommendations from professional societies and the Choosing Wisely campaign for health systems and third-party payers to detect, quantify, and reduce low-value prescribing.⁸ Many current recommendations, such as avoiding the use of testosterone to treat nonspecific symptoms of aging, provide only general guidance to clinicians and have not been operationalized to measure low-value prescribing using administrative claims or electronic medical record data, which would ensure the scalability and automation of these measures.⁹ Other tools, such as the Beers Criteria,¹⁰ may be applied to health care data but focus primarily on medication safety and do not incorporate other components of health care value, such as cost. The perspectives of patients or their caregivers were not integral to the development of these and other low-value care metrics.⁹

In this qualitative study, we aimed to develop a scalable and broadly applicable metric that contains a set of quality indicators to detect and reduce low-value prescribing among older adults and that is informed by diverse stakeholders’ perspectives. We named this metric EVOLV-Rx (Evaluating Opportunities to Decrease Low-Value Prescribing).

We developed EVOLV-Rx in 2 sequential steps. First, we generated the criteria to detect candidate low-value prescribing practices. For this step, we synthesized the preexisting recommendations from the Choosing Wisely campaign and other medication safety criteria (eg, the Beers Criteria), peer-reviewed literature, and the results of 3 qualitative studies on low-value prescribing.^11-13 Second, using an online modified-Delphi approach, we convened a panel of physicians and pharmacists to codify the final components of EVOLV-Rx (Figure 1). Members of this panel serve as health system leaders, practicing clinicians, and researchers. This study was deemed exempt by the institutional review boards of the University of Pittsburgh and the RAND Corporation.

We compiled a collection of low-value prescribing recommendations according to previously applied approaches to identifying low-value health services.^3,14 We considered prescribing practices whose costs or harms generally outweigh their benefits for older adults (aged ≥65 years). Practices were assessed on the following 4 value domains, which were adapted from the Lown Institute approach to characterizing medication appropriateness¹⁵: (1) lack of effectiveness for a stated indication, either in general or because of inappropriately prolonged use; (2) potential for harm because of the risk of an adverse drug event, drug-drug or drug-disease interaction, or overly intensive treatment; (3) excessive cost because of the unnecessary use of a costly brand-name medication or specific preparation; or (4) use of a medication to treat the adverse effects of another medication as part of a prescribing cascade.

We acquired low-value prescribing recommendations from the Choosing Wisely campaign⁸ and the most up-to-date versions of the following medication safety and appropriateness guidelines: the Beers Criteria,¹⁰ FORTA (Fit for the Aged) list,¹⁶ STOPP/START (Screening Tool of Older Persons’ Prescriptions and Screening Tool to Alert to Right Treatment),¹⁷ and STOPPFrail (Screening Tool of Older Persons’ Prescriptions in Frail Adults With Limited Life Expectancy).¹⁸ We also considered the findings from published studies and medications or medication classes that were most associated with adverse drug events requiring hospital admission.¹⁹ We integrated the findings from 3 qualitative studies (which were conducted by some of us and other members of our research team), which used focus groups with patients and caregivers as well as semistructured interviews with primary care physicians to characterize their perspectives on and specific examples of low-value prescribing (eTable 1 in the Supplement).^11-13

To focus on those candidate low-value prescribing practices that were current and had the potential to affect the greatest number of older adults, we narrowed down our initial collection of low-value prescribing recommendations to those that overlapped with the 100 most frequently prescribed or costly medications among Medicare beneficiaries in fiscal year 2017 (the most recent year of available data at the beginning of this study). We also included medications whose frequency of use was comparable to drugs that were most frequently prescribed in Medicare Part D but may be purchased over the counter, such as aspirin or docusate sodium.²⁰

All of the investigators on the research team met monthly to generate operational definitions that could be applied to health care data for each candidate low-value prescribing practice. An external advisory panel of 6 geriatricians, pharmacoepidemiologists, and medication safety experts also met every 3 months to provide general feedback and ensure that EVOLV-Rx remained focused on the prescribing practices that were most relevant to older adults.

To define each candidate low-value prescribing practice, we developed both sensitive and specific criteria, applying an approach developed by Schwartz et al.^3,21 The goal of the sensitive criteria was to identify patients who could be subjected to potential low-value prescribing, whereas the goal of the specific criteria was to identify the subset of older patients for whom the prescribing practice would most likely be of low value, according to a priori low-value criteria we had established. For example, regarding the prolonged use of proton pump inhibitors (PPIs), the proposed sensitive criteria may identify patients who were prescribed a PPI for more than 2 months. Recognizing that the prolonged use of a PPI may at times be clinically appropriate, the specific criteria would build on the sensitive criteria to recognize situations in which PPI use would most likely be of low value, such as among patients without a guideline-concordant indication, without concurrent long-term use of NSAIDs and corticosteroids, or with a prescription for a brand-name PPI.

To facilitate the deliberations of the online modified-Delphi panel, we generated a 2-page peer-reviewed infographic for each candidate low-value prescribing practice. The infographic contained the proposed sensitive and specific criteria as well as an evidence-based synopsis and related utilization, cost, and qualitative data (3 examples are provided in the eFigure in the Supplement). Before implementing them in the study, we received feedback on the criteria for each candidate low-value prescribing practice as well as on the accuracy and content of the infographic from a multidisciplinary clinical faculty that was affiliated with the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh.

To establish consensus on and refine the criteria to identify the most salient low-value prescribing practices for EVOLV-Rx, we convened a panel of 15 physicians and pharmacists using an online modified-Delphi method. This panel comprised practicing clinicians; health system leaders; and international experts in deprescribing, pharmacoepidemiology, and health care value (Box). Panelists were recruited on the basis of their record of scholarly publications, academic or industry leadership, and referral by members of the external advisory panel or research team. To conduct the activities of the panel, we applied a validated, 3-round approach using ExpertLens software package (RAND).^22-25 Before its use in this study, the data collection protocol using ExpertLens was tested by 3 local clinicians.

Box Section Ref ID

The panel was engaged for 3 rounds between January 1 and March 31, 2021, and each round lasted 1 to 2 weeks. In round 1, the panelists rated the scientific validity and clinical usefulness of the criteria to detect each candidate low-value prescribing practice using a 9-point Likert scale (median score ranges: 1 to 3 indicating low validity or usefulness; 3.5 to 6, uncertain validity or usefulness; and 6.5 to 9, high validity or usefulness²⁶) and provided corresponding comments to explain their ratings (eTable 2 in the Supplement provides a full description of the panel questions and rating interpretation). In round 2, the panelists reviewed each other’s numeric and free-text responses for each candidate low-value prescribing practice, as depicted by descriptive statistics, statements about the existence of agreement among panelists, and bar charts that were automatically generated by ExpertLens. Next, the panelists participated in an anonymous and asynchronous online discussion, which one of us moderated (T.R.R.). Informed by the round 1 findings and round 2 discussion, we revised or clarified the proposed sensitive and specific criteria. In round 3, informed by the scores and deliberations in rounds 1 and 2, the panelists provided their final ratings for each candidate low-value prescribing practice. In each round, individual members of the panel reviewed the practices in random order to ensure that robust ratings were given across all candidate low-value prescribing practices.

We applied the RAND/UCLA Appropriateness Method to panelist scores in rounds 1 and 3 to ascertain whether there was agreement among the panelists and to identify the degree of scientific validity and clinical usefulness for each candidate low-value prescribing practice.²⁶ This validated method has been used to achieve consensus on the content of clinical guidelines and quality metrics in an online modified-Delphi process.²⁶

Using the RAND/UCLA Appropriateness Method, we looked for the presence of agreement by first calculating the interpercentile range (IPR) between the 70th and 30th percentiles of panelist scores. Next, we calculated the IPR adjusted for symmetry with the following equation: IPRAS = 2.35 + (AI × 1.5), where AI represented the asymmetry index, which is defined as the distance between the central point of the IPR and 5, the central point of the 9-point Likert scale used by panelists to rate each candidate low-value prescribing criteria. If the IPR was greater than the IPR adjusted for symmetry, then there was no agreement. If the IPR was less than the IPR adjusted for symmetry, then there was agreement. For the low-value prescribing criteria wherein agreement was found, we characterized their scientific validity and clinical usefulness according to the median scores. The final metric included only candidate low-value prescribing practices that showed panel agreement and received median scores of 6.5 or higher, which indicated both high scientific validity and high clinical usefulness.

A total of 527 low-value prescribing recommendations were compiled from the Choosing Wisely campaign, other medication safety criteria, and peer-reviewed literature, along with an additional 101 medications or medication classes that were cited as being potential low-value prescribing practices by the participants of the qualitative studies we examined (eTable 1 in the Supplement). Of these medications, 63 overlapped with the costliest or most frequently prescribed medications among Medicare beneficiaries in 2017. We further consolidated medications from the same class or category into a single operationalized definition (eg, antibiotics, anticoagulants, or opioids for pain), resulting in 27 discrete candidate low-value prescribing practices that were considered for inclusion in EVOLV-Rx (eTable 3 in the Supplement provides a full list of practices and definitions).

The round 1 ratings of the online modified-Delphi panel of experts for the scientific validity and clinical usefulness of each of the 27 candidate low-value prescribing practices are shown in Table 1. There was agreement among the panelists regarding the candidate practices except for prescribing statins for the primary prevention of atherosclerotic cardiovascular disease (ASCVD). At the conclusion of round 1, 19 candidate practices received a median score of 6.5 or higher, indicating high scientific validity. Eight candidate practices received a median score of 3.5 to 6 (an uncertain degree). Twenty candidate practices received a high clinical usefulness score of 6.5 or higher, and 7 candidate practices were rated as having uncertain clinical usefulness.

After considering the panelist scores, comments, and deliberations from rounds 1 and 2, we revised the sensitive and/or specific criteria that defined 19 candidate low-value prescribing practices (eTable 3 in the Supplement). Round 3 ratings are also shown in Table 1. In round 3, 18 candidate practices received high validity and clinical usefulness scores of 6.5 or higher and were included in EVOLV-Rx (Table 2).

Of those candidate low-value prescribing practices that were not incorporated in the final metric, 3 (aspirin for primary prevention of ASCVD, opioids for treatment of noncancer pain, and potentially unsafe use of sedative or hypnotic sleeping aids) were rated as having high scientific validity but uncertain clinical usefulness (eTable 3 in the Supplement). Six candidate practices received an uncertain scientific validity rating. One candidate low-value prescribing practice had a median validity score of 5, whereas the other 5 candidate practices received a median score of 6 (Table 1). For example, for the practice of prescribing statins for the primary prevention of ASCVD (final median score of 5 in scientific validity), the panelists commented that trials were ongoing to evaluate the effectiveness of statins for the primary prevention of ASCVD in older adults and that data from observational studies were insufficient to rate this candidate practice as being scientifically valid. In another example, the practice of prescribing potentially unsafe use of antihypertension medications received a final median score of 6 in scientific validity. The explanation for this rating was the presence of conflicting evidence of the reasonable blood pressure target in older adults and the anticipated challenges in operationalizing this practice given other indications for common blood pressure medications, such as β-blockers.

We convened an online modified-Delphi panel of experts to codify the components of EVOLV-Rx (Table 2). Of the 27 candidate low-value prescribing practices derived from 527 low-value prescribing recommendations, the panel identified the 18 most scientifically valid and clinically useful prescribing practices and related criteria for detecting low-value prescribing in the care of older adults. In addition, these practices and criteria reflect the perspectives of patients, caregivers, and practicing physicians. EVOLV-Rx contains a set of quality indicators that may be scaled and automated to detect low-value prescribing in large administrative or other clinical data sets for thousands of patients. We believe this metric would enable health systems, third-party payers, and policy makers to reduce low-value prescribing in ways that are both clinically sound and broadly acceptable.

EVOLV-Rx is distinct from other metrics in that it applies a value-based paradigm to consolidate the hundreds of existing low-value prescribing recommendations to define, prioritize, and systematically measure low-value prescribing practices in a way that can be operationalized in payer and provider data. Kerr et al¹⁴ recently applied a similar approach to identify high-priority recommendations for deintensifying care within the Veterans Health Administration. Rather than develop additional broad-based guidelines, we sought to specify operational definitions for the components of a metric that were intended for use in administrative claims and other health data. Thus, because of measurement concerns, we excluded candidate low-value prescribing practices, such as the overtreatment of hypertension, from the final version of EVOLV-Rx. With clearly defined sensitive and specific criteria for each low-value prescribing practice, EVOLV-Rx may serve as a useful adjunct to the Schwartz low-value care metric, which contains 31 distinct low-value tests and procedures but not medications and is currently used by the Medicare Payment Advisory Commission, Veterans Health Administration, and private insurers.^30,31

Furthermore, EVOLV-Rx is unique in the way it integrates the perspectives of patients, caregivers, and practicing physicians about low-value prescribing and health care value. The panelists acknowledged these perspectives in their deliberations. Consequently, the individual components of EVOLV-Rx reflect stakeholder views on specific medications and their adverse effects; medical comorbidities that make older patients susceptible to low-value prescribing; and burdensome and costly medication preparations and methods of administration, such as the unnecessary subcutaneous or intramuscular injection of vitamin B₁₂. By incorporating the views of diverse stakeholders in the development of EVOLV-Rx, we believe we have increased the likelihood that, when applied in clinical practice, EVOLV-Rx will decrease patients’ medication burden in a way that aligns with their values, is less likely to be perceived as rationing, and has face validity for practicing clinicians who are tasked with its use.³²

The application of EVOLV-Rx may advance a variety of research and policy priorities. There is growing interest in deprescribing low-value medications among older adults, as evidenced by the emergence of federally funded Deprescribing Research Networks in the US and in other countries. EVOLV-Rx may help researchers, third-party payers, and health system leaders involved in deprescribing efforts to identify a research focus, given the hundreds of low-value prescribing recommendations, and conduct more rigorous audit and feedback of prescribing practices.^33,34 From a policy perspective, Medicare has been increasingly discouraging the provision of low-value care through value-based payment reforms.³⁵ In addition, third-party payers are implementing arrangements, such as bundled payments and shared savings plans, to promote the delivery of high-value care.³⁶ The application of EVOLV-Rx may enable such efforts to readily incorporate low-value prescribing.

This study and EVOLV-Rx have several limitations. First, the low-value prescribing practices contained in the metric are not exhaustive given that our goal was to create a focused, acceptable, and scalable metric. Therefore, its use does not preclude the application of traditional tools, such as the Beers Criteria. Second, the infographics we provided to each panelist were intended to guide their deliberations but did not depict the findings of a systematic review for each candidate low-value prescribing practice. However, each infographic underwent peer review to ensure its accuracy and each included a summary, links, and references to systematic reviews or guidelines based on such reviews. The panelists were instructed to use these infographics as a reference or supplementary material rather than as a sole resource for their deliberations and when rating each candidate low-value prescribing practice. Third, the results (the criteria in EVOLV-Rx) are sensitive to the composition of the Delphi panel, and it is not known to what degree these results would be different with a different set of experts. Fourth, EVOLV-Rx has not yet been fully operationalized for use in administrative claims or electronic health record data. The research team is currently working to operationalize and establish the specification validity of EVOLV-Rx. Fifth, each component of EVOLV-Rx may not be applicable in all forms of health data (eg, administrative claims vs electronic health records) or in all populations, which was a necessary tradeoff in developing criteria that best characterize each form of low-value prescribing.

In this qualitative study, a panel of experts identified 18 scientifically valid and clinically useful prescribing practices and related criteria for detecting low-value prescribing practices in the care of older adults, resulting in the development of the EVOLV-Rx metric. The application of EVOLV-Rx may also enhance the detection of low-value prescribing alongside other low-value tests and procedures, reduce polypharmacy, and enable older adults to receive high-value care across the full spectrum of health services in a way that aligns with their perspectives and values.

Accepted for Publication: December 20, 2021.

Published: February 15, 2022. doi:10.1001/jamanetworkopen.2021.48599

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Radomski TR et al. JAMA Network Open.

Corresponding Author: Thomas R. Radomski, MD, MS, Division of General Internal Medicine, Department of Medicine, University of Pittsburgh School of Medicine, 3609 Forbes Ave, 2nd Floor, Pittsburgh, PA 15213 (radomskitr@upmc.edu).

Author Contributions: Dr Radomski had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Radomski, Thorpe, Hanlon, Fine, Gellad.

Acquisition, analysis, or interpretation of data: Radomski, Decker, Khodyakov, Thorpe, Roberts, Gellad.

Drafting of the manuscript: Radomski.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Radomski, Khodyakov.

Obtained funding: Radomski.

Administrative, technical, or material support: Radomski, Decker, Gellad.

Supervision: Radomski, Thorpe, Hanlon, Roberts, Fine.

Conflict of Interest Disclosures: Dr Radomski reported receiving grants from the National Institutes of Health (NIH) and the US Department of Veterans Affairs during the conduct of the study. Dr Khodyakov reported receiving grants from the National Institute on Aging (NIA) during the conduct of the study and serving as the ExpertLens team leader. Dr Gellad reported receiving grants from the NIH during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was funded by Career Development Award K23AG061217 from the NIA (PI: Dr Radomski).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: Emily Dao, BS, RAND Corporation, assisted with applying the RAND ExpertLens Program to work with the Delphi panel. Andrew Klein, MD, MS; Jing Luo, MD, MPH; and Walid Gellad, MD, MPH, Division of General Internal Medicine, University of Pittsburgh School of Medicine, assisted with pilot-testing the ExpertLens software package. These individuals received no additional compensation, outside of their usual salary, for their contributions. The Delphi panelists are as follows: Timothy Anderson, MD, MAS; Elizabeth Bayliss, MD, MSPH; Winfred Frazier, MD, MPH; Mark Friedberg, MD, MPP; Chester B. Good, MD, MPH; Holly Holmes, MD, MS; Matthew Joseph, MD, PharmD; Zach Marcum, PharmD, PhD; Christopher Moriates, MD; Natasha Parekh, MD, MS; Emily Reeve, BPharm, PhD; Gordon Schiff, MD; Kenneth Schmader, MD; Michael Steinman, MD; and Justin Turner, BPharm, MClinPharm, PhD.