Characteristics of Cost-effectiveness Studies for Oncology Drugs Approved in the United States From 2015-2020

Key Points

Question  What are the characteristics of cost-effectiveness studies for oncology drugs approved by the US Food and Drug Administration, and what factors are associated with whether a drug is determined to be cost-effective?

Findings  In this cross-sectional study of 116 drug approvals and 228 cost-effectiveness studies and 254 analyses, a drug was 40 times more likely to be deemed cost-effective when the study was funded by pharmaceutical companies compared with no funding.

Meaning  The finding that pharmaceutical funding was associated with whether a drug was found to be cost-effective suggests that simply disclosing potential conflict of interests is inadequate.

Importance  Increasingly, cost-effectiveness analyses are being done to determine the value of rapidly increasing oncology drugs; however, this assumes that these analyses are unbiased.

Objective  To analyze the characteristics of cost-effectiveness studies and to determine characteristics associated with whether an oncology drug is found to be cost-effective.

Design, Setting, and Participants  This retrospective cross-sectional study included 254 cost-effectiveness analyses for 116 oncology drugs that were approved by the US Food and Drug Administration from 2015 to 2020.

Exposures  Each drug was analyzed for the incremental cost-effectiveness ratio per quality-adjusted life year, the funding of the study, the authors’ conflict of interest, the threshold of willingness-to-pay, from what country’s perspective the analysis was done, and whether a National Institute for Health and Care Excellence cost-effectiveness analysis had been done.

Main Outcomes and Measures  The main outcome was the odds of a study concluding that a drug was cost-effective.

Results  There were 116 drug approvals with 254 studies and country perspectives. Of the country perspectives, 132 (52%) were from the US. Forty-seven of 78 drugs with cost-effective studies had been shown to improve overall survival, whereas 15 of 38 of drugs without a cost-effectiveness study had been shown to improve overall survival. Having a study funded by a pharmaceutical company was associated with higher odds of a study concluding that a drug was cost-effective than studies without funding (odds ratio, 41.36; 95% CI, 11.86-262.23).

Conclusions and Relevance  In this cross-sectional study, pharmaceutical funding was associated with greater odds that an oncology drug would be found to be cost-effective. These findings suggest that simply disclosing potential conflict of interest is inadequate. We encourage cost-effectiveness analyses by independent groups.

The cost of oncology drugs in the US has risen dramatically over the last 2 decades, with current launch prices routinely in excess of 100 000 US dollars (USD) per year of treatment and one-time therapies costing over 400 000 USD.^1-4 These prices remain disconnected from measures of therapeutic response (eg, response rate or improvements in progression-free survival) and clinical benefit (eg, improvements in quality of life or overall survival).^1,5

In light of these rising costs, assessment of cost-effectiveness for approved oncology drugs has become more common. For instance, the Institute for Clinical and Economic Review Group systematically reviews clinical evidence to determine the comparative clinical effectiveness of drugs in the US. Their analyses inform decision making by the US Medicaid agencies, the Department of Veteran’s Affairs, and 75% of private insurance agencies.⁶ Notably, nations, such as Canada and the United Kingdom, use value-based calculations to determine which medicines to cover.^7,8

Two decades ago, a seminal study in the British Medical Journal examined the potential role of conflict of interest (COI) in oncology economic studies. Miners et al⁹ found that pharmaceutical funded studies were more likely to report favorable economic findings for drugs than nonprofit sponsored studies. Their findings, in addition to those of others,¹⁰ are consistent with findings on COI in cost-effectiveness studies in the medical literature at large.¹¹ However, little follow-up research has been done to determine how often COI is reported in the scientific literature and its effects on cost-effectiveness study outcomes. Additionally, there has been no investigation solely in the field of medical oncology. Here, we reviewed cost-effectiveness studies for oncology drugs that were approved by the US Food and Drug Administration between 2015 to 2020 and sought to assess the association between cost-effectiveness results and study and drug characteristics.

In accordance with the US Department of Health and Human Services code of federal regulations, 45 CFR §46.102(f), this cross-sectional study was not submitted for institutional review board approval and informed consent was waived because it involved publicly available data and did not involve individual patient data. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.

We searched all oncology drugs that were FDA-approved for advanced, metastatic, or unresectable cancers based on results from randomized controlled trials from 2015 through 2020. We excluded noninferiority studies if they were about drugs that had an approval for a different dosage or mode of administration for the same drug. For each of the included drugs, we noted the date of approval and searched for the published trial reporting the results that were used for the FDA approval. From the reported trial data, we abstracted the name of the trial, tumor type, indication, and drug used as a comparator. From the studies, we also assessed whether the study reported an improvement in overall survival, progression-free survival, or response rate. If the study did not report on overall survival or results were immature, we searched the clinicaltrials.gov website to see if another published study tested whether the drug improved overall survival for the given indication.

For all oncology drugs for a metastatic or unresectable indication that were approved between 2015 and 2020, we then searched Google Scholar for studies that calculated cost-effectiveness for the drug and indication. We used a similar search for each drug, which included the name of the drug, the tumor type, the name of the trial, and cost effectiveness (eg, apalutamide + prostate + spartan + cost effectiveness). We reviewed the first 10 search results to see if the studies could be included in our analysis. Studies needed to be cost-effective studies, report an incremental cost-effectiveness ratio (ICER) per quality-adjusted life year (QALY), and use the clinical trial used for the drug’s FDA approval in the analysis. We also noted whether the drug was considered cost-effective. We specifically included studies using data from the randomized clinical trial used for the drug’s FDA approval. However, we noticed that not all of the cost-effectiveness studies used the same comparator that was used in the randomized clinical trials. In these cases, we noted whether the comparator drug was the same or different than the comparator drug that was used in the randomized clinical trial for its FDA approval. We included abstracts and poster presentations if they contained enough information for our analysis. Our search was conducted from April 15 to May 3, 2021.

From the included studies, we abstracted the ICER per QALY, incremental additional life years, incremental QALY, incremental costs, cost per life-year, the funding of the study, authors’ COI, the threshold of willingness-to-pay, and from what country’s perspective the analysis was done. For studies that reported monetary amounts in anything other than USD, we converted these amounts to USD as of May 4, 2021. We categorized funding and conflict of interest as either industry or nonindustry. We then created a third variable for studies that had both the author and study funded by industry (ie, double-conflict), studies that were funded by a nonindustry organization but authors were funded by industry (ie, author conflict), or studies that were not funded by industry with no evidence of author receiving funding by industry (ie, none). When multiple willingness-to-pay thresholds were presented in a single study, we used the highest value.

We then searched the UK National Institute for Health and Care Excellence (NICE) website to see if they had conducted their own cost-effective analysis and recommended the drugs for the respective indications. For this search, we used the same search terms we used in the Google Scholar search, but without the study name.

We calculated frequencies and percentages for characteristics of cost-effectiveness studies, stratified by whether they were cost-effective or not. We calculated this for all studies and then for studies that were not funded by pharmaceutical companies. We compared characteristics with χ² and Wilcoxon rank-sum tests. We examined factors associated with studies determining whether a drug was cost-effective with logistic regression. We used cost-effective as a binary dependent outcome (ie, yes or no). We adjusted each of the 3 models—overall survival, progression-free survival, or overall response rate—for clinical outcome (ie, yes or no), conflict of interest (ie, none, author conflict, double conflict, or not indicated), country of perspective (US or non-US), QALY threshold, and whether the comparator in the cost-effectiveness analysis used the same comparator as the FDA approval (ie, yes or no). We conducted all analyses using R software, version 3.6.2 (R Project for Statistical Computing). Statistical significance was set at α = .05, and tests were 2-tailed. Statistical analyses were completed on June 1, 2021.

This study included 116 drug approvals from 228 cost-effectiveness studies reporting on 254 cost-effectiveness claims. Of the 116 drug approvals, we found between 1 and 9 cost effectiveness studies (median, 3). Of the 228 studies, 49 (21%) were funded directly by a pharmaceutical company, 53 (23%) did not indicate funding, and 126 (55%) had no funding. Authors received money from a pharmaceutical company directly or indirectly in 96 of 228 studies (42%). No author conflict of interest was reported in 121 of 228 studies (53%), and 11 of 228 studies (5%) did not report author conflict of interest.

Of the 116 total drug approvals, 62 (53%) had been shown to improve overall survival, whereas 47 of the 78 (60%) drug approvals with cost-effective studies had been shown to improve overall survival. Of the 38 drug approvals that did not have a cost-effectiveness analysis, there were 15 studies (39%) that showed improved overall survival in trial data, which was not significantly lower than drugs that did have a cost-effectiveness analysis (P = .06). Regarding drugs without a cost-effectiveness analysis, 20 (53%) improved overall response rate and 29 (76%) improved progression-free survival. These findings were not statistically different than drugs with a cost-effectiveness analysis (ORR: 42 [54%]; PFS: 66 [85%]). The percentages of drugs without a cost-effectiveness analysis that have been shown to improve ORR or PFS were 53% and 76%, which were no different than drugs with a cost-effectiveness analysis (ORR: 42 [54%]; PFS: 66 [85%]). The 254 cost-effectiveness perspectives came primarily from the US (132 [52%]), China (41 [16%]), and the UK (15 [6%]). Of these studies, 121 (48%) were cost-effective.

In all 254 analyses, there was a higher percentage of studies concluding that a drug was cost-effective when funded by a pharmaceutical company than studies not funded by a pharmaceutical company or when funding was not indicated (48 [96%] vs 44 [30%] vs 29 [53%]; P < .001; Table 1). Studies that did not disclose funding and studies funded by a pharmaceutical company compared with studies not funded by a pharmaceutical company were more likely to use a different comparator in the cost-effectiveness analysis than was tested against in the randomized study used for the drug’s FDA approval (13 [24%] vs 6 [12%] vs 6 [4%]; P < .001). Studies that did not disclose funding and studies funded by a pharmaceutical company compared with studies not funded by a pharmaceutical company were less likely to report a threshold of cost-effectiveness (13 [24%] vs 9 [18%] vs 2 [1%]; P < .001).

In adjusted models (Table 2), having a study funded by a pharmaceutical company was associated with higher odds of a study concluding that a drug was cost-effective than when there was no funding (overall survival model: odds ratio [OR], 41.36; 95% CI, 11.86-262.23). Using a comparator drug different than was tested against in the randomized study used for the drug’s FDA approval was also associated with a study concluding that a drug was cost-effective (OS model: OR, 3.38; 95% CI, 1.01-12.50). Neither PFS nor OS was associated with a study concluding a drug to be cost-effective, but when there was a benefit of overall response rate, a study was less likely to find a drug cost-effective (OR, 0.52; 95% CI, 0.26-0.98).

The Figure shows the difference between QALY ICERs and threshold among studies from a US perspective that reported the study’s source of funding. Studies funded by nonindustry often had QALY ICERs well above the funding threshold. The eFigure in the Supplement shows the QALY ICER for each study in relation to commonly used thresholds (100 000, 150 000, and 180 000).

In this analysis of cost-effectiveness studies on oncology drugs, we found that the largest factor associated with whether a drug was found to be cost-effective was if the study was funded by the pharmaceutical industry. This factor was associated with positive cost-effectiveness findings even when considering therapeutic or clinical outcomes (eg, OS or PFS). Further, cost-effectiveness thresholds were not associated with whether a drug was found to be cost-effective, indicating that if a study used a higher threshold for cost-effectiveness, it was not more likely to conclude that the drug was cost-effective.

In 2016, the Second Panel on Cost-Effectiveness in Health and Medicine published updated recommendations on the methodological practices and reporting of cost-effectiveness studies.¹² One of their recommendations was that studies should disclose funding sources and potential COI. However, there are at least 2 considerations to these recommendations that relate to our findings. First, many authors and journals are not adhering to these recommendations. Over 20% of the studies in our analyses, most of which were published after the Second Panel’s recommendations were made, did not disclose funding sources, and studies that did not report funding were more likely to find a drug cost-effective. Second, encouraging authors to report COI may be inadequate to address the bias from COI in cost-effectiveness analyses. In adjusted analyses, the odds of a study reporting favorable cost-effectiveness for a drug is over 40 times greater among studies funded by pharmaceutical companies compared to those that are not funded by pharmaceutical companies. The concern of the inadequacy of simply disclosing COI in cost-effectiveness studies has been expressed previously.¹³

We also found that when researchers used a different comparator in the cost-effectiveness analysis, the study was almost 4 times as likely to conclude that the drug was cost-effective. This finding is likely related to 84% of positive cost-effectiveness analyses that used a different comparator being funded by pharmaceutical companies, compared with 50% of analyses not finding cost-effective benefit. Often, the comparator drug used in these cases were drugs of the same class as the new drug (eg, kinase inhibitors or programmed death-ligand 1 inhibitors). These studies often result in discrepant results, depending on the funder of the study and the employer of the authors. Such is the case of inotuzumab ozogamicin¹⁴ and blinatumomab cost-effectiveness analyses.¹⁵

Even in pharmaceutical-funded studies that did not find a drug to be cost-effective, the role of funding appeared to influence the narrative of the study. For example, in patients with renal cell carcinoma, cabozantinib had a higher ICER than everolimus, the comparator drug that was used for cabozantinib’s US Food and Drug Administration’s approval. However, this finding was overshadowed by the finding that it was more cost-effective than nivolumab, another approved drug for the same indication and that also used everolimus as a comparator for its approval.

The results of our study are consistent with others that have examined the relationship between study sponsorship and whether the study concluded that a drug was cost-effective. While these studies also found a strong relationship between pharmaceutical funding and cost-effectiveness benefit, both examined cost-effectiveness studies published during earlier years than we did.^16,17 Further, Garattini et al¹⁶ examined studies published between 2004 to 2009 and did not limit by drug or indication; Lane et al¹⁷ examined studies published between 1991 to 2021 and focused on drugs approved for breast cancer only. Our study examines more contemporary cost-effectiveness studies and brings to light that while the percentage of studies not disclosing funding was slightly lower in those published 2017 to 2020, compared with 2015 to 2016 (25% vs 21%), the influence of COI in the study results is as pervasive now as it was before the recommendations by the Second Panel on Cost-Effectiveness in Health and Medicine in 2016.

This study had limitations. First, when determining whether a drug improved OS, PFS, or ORR, we only used the study that reported data used for FDA approval or another study that was listed on the clinicaltrials.gov website. Subsequent studies may have reported these values, and by not including them, our results may be different. However, at least 90% of studies had values for these outcomes. Second, we only used the first 10 search results. As a result, there may have been studies that we missed. However, most drugs had fewer than 10 cost-effectiveness studies, and it is unlikely that a relevant cost-effectiveness article would have been found beyond 10 search results. Third, articles may have been missed because we required the cost-effectiveness analysis to include FDA trial data, and we often used the trial name in our search. However, this may have been beneficial because it eliminated irrelevant studies from our search results, allowing us to focus on relevant studies.

In this cross-sectional study, pharmaceutical funding was associated with whether a drug was found to be cost-effective. These findings suggest that simply encouraging authors to disclose potential COI is inadequate. In many instances, funding was unreported, and even when funding was reported, bias in the results is a serious concern. Independent groups should be encouraged to perform cost-effectiveness analyses and journal editors should preferentially choose such analyses for publication.

Accepted for Publication: September 23, 2021.

Published: November 18, 2021. doi:10.1001/jamanetworkopen.2021.35123

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2021 Haslam A et al. JAMA Network Open.

Corresponding Author: Alyson Haslam, PhD, Department of Epidemiology and Biostatistics, UCSF Mission Bay Campus, Mission Hall: Global Health and Clinical Sciences Building, 550 16th St, 2nd Floor, San Francisco, CA 94158 (alyson.haslam@ucsf.edu).

Author Contributions: Dr Haslam had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Haslam, Prasad.

Acquisition, analysis, or interpretation of data: Haslam, Lythgoe, Greenstreet Akman.

Drafting of the manuscript: Haslam, Lythgoe.

Critical revision of the manuscript for important intellectual content: Lythgoe, Greenstreet Akman, Prasad.

Statistical analysis: Haslam.

Administrative, technical, or material support: Lythgoe.

Supervision: Lythgoe, Prasad.

Conflict of Interest Disclosures: Dr Prasad reported receiving fees from Arnold Ventures during the conduct of the study and receiving personal fees from Johns Hopkins Press, Medscape, MedPage, Evicore, New Century Health, UnitedHealthcare Consulting, and Patreon outside of the submitted work. No other disclosures were reported.

Funding/Support: This study received funding from the Laura and John Arnold Foundation.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.