[Skip to Content]
[Skip to Content Landing]
Figure 1.
Flow Diagram of Review Process
Flow Diagram of Review Process

Flow diagram depicting the process used to systematically identify all initial US Food and Drug Administration approvals occurring between 2003 and 2013 for molecules with an active primary indication for cancer. Exclusion criterion #1 identifies all molecules that are neither used in oncology, nor indicated as an active treatment for cancer. Exclusion criterion #2 identifies all molecules that are used in oncology, but are not indicated as an active treatment for cancer. Food and Drug Administration initial approvals identified through Roberts et al22 and Drugs@FDA registry.31 European Medicines Agency initial approvals identified through the European Medicines Agency’s European public assessment reports search engine.32

Figure 2.
Analysis of Improvements in Overall Survival
Analysis of Improvements in Overall Survival

Development of new cancer medicines (2003-2013), mapped according to therapeutic comparator used by health technology appraisal agencies in appraisal documents to assess therapeutic value. “First generation” drugs are the set of comparators not approved between 2003 and 2013, whereas “third generation” drugs are those that were evaluated against medications that were newly licensed in the study period (“second generation”). Pemetrexeda represents use for a nonprimary indication—it is therefore considered independently of the pemetrexed indication that is evaluated in this study. Survival benefits associated with parallel treatment pathways (afatinib-erlotinib/gefitinib; ponatinib-nilotinib/dasatinib) are considered independently of each other, as are those associated with multiple primary indications (sunitinib). The gain in overall survival (OS) relative to initial standards of care, for all drugs where marginal increases in OS could be quantified, is provided with the use of bars that represent the number of months gained (rounded to nearest integer). If a range of values corresponding to OS benefits were available across health technology appraisal agencies, an average was taken. Uncertain increase in OS is represented with a “+”; NE indicates no established increase in OS.

Table.  
Evidence Generally Reported by HTA Agencies to Evaluate Drug-Related Effects on Key Outcome Measures
Evidence Generally Reported by HTA Agencies to Evaluate Drug-Related Effects on Key Outcome Measures
1.
Weaver  KE, Rowland  JH, Bellizzi  KM, Aziz  NM.  Forgoing medical care because of cost: assessing disparities in healthcare access among cancer survivors living in the United States.  Cancer. 2010;116(14):3493-3504. doi:10.1002/cncr.25209PubMedGoogle ScholarCrossref
2.
Experts in Chronic Myeloid Leukemia.  The price of drugs for chronic myeloid leukemia (CML) is a reflection of the unsustainable prices of cancer drugs: from the perspective of a large group of CML experts.  Blood. 2013;121(22):4439-4442. doi:10.1182/blood-2013-03-490003PubMedGoogle ScholarCrossref
3.
Mailankody  S, Prasad  V.  Five years of cancer drug approvals: innovation, efficacy, and costs.  JAMA Oncol. 2015;1(4):539-540. doi:10.1001/jamaoncol.2015.0373PubMedGoogle ScholarCrossref
4.
Apolone  G, Joppi  R, Garattini  S.  Ten years of marketing approvals of anticancer drugs in Europe: regulatory policy and guidance documents need to find a balance between different pressures.  Brit J Cancer. 2005;93(5):504-509. doi:10.1038/sj.bjc.6602750PubMedGoogle ScholarCrossref
5.
Ellis  LM, Bernstein  DS, Voest  EE,  et al.  American Society of Clinical Oncology perspective: Raising the bar for clinical trials by defining clinically meaningful outcomes.  J Clin Oncol. 2014;32(12):1277-1280. doi:10.1200/JCO.2013.53.8009PubMedGoogle ScholarCrossref
6.
Han  K, Ren  M, Wick  W,  et al.  Progression-free survival as a surrogate endpoint for overall survival in glioblastoma: a literature-based meta-analysis from 91 trials.  Neuro Oncol. 2014;16(5):696-706. doi:10.1093/neuonc/not236PubMedGoogle ScholarCrossref
7.
Beauchemin  C, Johnston  JB, Lapierre  MÈ, Aissa  F, Lachaine  J.  Relationship between progression-free survival and overall survival in chronic lymphocytic leukemia: a literature-based analysis.  Curr Oncol. 2015;22(3):e148-e156. doi:10.3747/co.22.2119PubMedGoogle ScholarCrossref
8.
Food and Drug Administration. Guidance for Industry: Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics. http://www.fda.gov/downloads/Drugs/Guidances/ucm071590.pdf. Accessed October 26, 2016.
9.
European Medicines Agency. Guideline on the evaluation of anticancer medicinal products in man. 2012. Report No.: EMA/CHMP/205/95/Rev.4. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/01/WC500137128.pdf. Accessed October 27, 2016.
10.
Johnson  JR, Ning  Y-M, Farrell  A, Justice  R, Keegan  P, Pazdur  R.  Accelerated approval of oncology products: the food and drug administration experience.  J Natl Cancer Inst. 2011;103(8):636-644. doi:10.1093/jnci/djr062PubMedGoogle ScholarCrossref
11.
112th Congress. Food and Drug Administration Safety and Innovation Act. 126 Stat. United States of America; 2012.
12.
McCain  JA.  The ongoing evolution of endpoints in oncology.  Manag Care. 2010;19(5):Suppl 1.Google Scholar
13.
Booth  CM, Eisenhauer  EA.  Progression-free survival: meaningful or simply measurable?  J Clin Oncol. 2012;30(10):1030-1033. doi:10.1200/JCO.2011.38.7571PubMedGoogle ScholarCrossref
14.
Shi  Q, Sargent  DJ.  Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials.  Int J Clin Oncol. 2009;14(2):102-111. doi:10.1007/s10147-009-0885-4PubMedGoogle ScholarCrossref
15.
Davis  S, Tappenden  P, Cantrell  A. A review of studies examining the relationship between progression-free survival and overall survival in advanced or metastatic cancer. http://www.nicedsu.org.uk/PFSOS%20Report.FINAL.06.08.12.pdf. Accessed October 28 2016.
16.
Angelis  A, Kanavos  P.  Critique of the American Society of Clinical Oncology Value Assessment Framework for Cancer Treatments: putting methodologic robustness first.  J Clin Oncol. 2016;34(24):2935–2936PubMedGoogle ScholarCrossref
17.
Schnipper  LE, Davidson  NE, Wollins  DS,  et al; American Society of Clinical Oncology.  American Society of Clinical Oncology Statement: A Conceptual Framework to Assess the Value of Cancer Treatment Options.  J Clin Oncol. 2015;33(23):2563-2577. doi:10.1200/JCO.2015.61.6706PubMedGoogle ScholarCrossref
18.
Johnson  JR, Williams  G, Pazdur  R.  End points and United States Food and Drug Administration approval of oncology drugs.  J Clin Oncol. 2003;21(7):1404-1411.PubMedGoogle ScholarCrossref
19.
Szende  A, Leidy  NK, Revicki  D.  Health-related quality of life and other patient-reported outcomes in the European centralized drug regulatory process: a review of guidance documents and performed authorizations of medicinal products 1995 to 2003.  Value Health. 2005;8(5):534-548. doi:10.1111/j.1524-4733.2005.00051.xPubMedGoogle ScholarCrossref
20.
Temple  R.  A regulator’s view of comparative effectiveness research.  Clin Trials. 2012;9(1):56-65.PubMedGoogle ScholarCrossref
21.
US Food and Drug Administration. Guidance for Industry: Expedited Programs for Serious Conditions—Drugs and Biologics. 2014. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM358301.pdf. Accessed October 28, 2016.
22.
Roberts  SA, Allen  JD, Sigal  EV.  Despite criticism of the FDA review process, new cancer drugs reach patients sooner in the United States than in Europe.  Health Aff (Millwood). 2011;30(7):1375-1381. doi:10.1377/hlthaff.2011.0231PubMedGoogle ScholarCrossref
23.
National Institute for Health and Care Excellence. NICE technology appraisal guidance. 2016. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance.
24.
Autorité de Santé  Haute. Medical Device Assessment in France: Guidebook.; 2009.http://www.has-sante.fr/portail/upload/docs/application/pdf/2010-03/guide_dm_gb_050310.pdf. Accessed October 28, 2016.
25.
Pharmaceutical Benefits Advisory Committee. About the Guidelines. 2014https://pbac.pbs.gov.au/information/about-the-guidelines.html.
26.
National Institute for Health and Care Excellence. Developing review questions and planning the systematic review. In: The guidelines manual. NICE; 2012. https://www.nice.org.uk/article/pmg6/chapter/4-Developing-review-questions-and-planning-the-systematic-review. Accessed October 27, 2016.
27.
National Institute for Health and Care Excellence. Appraising life-extending, end of life treatments. 2009. https://www.nice.org.uk/guidance/gid-tag387/resources/appraising-life-extending-end-of-life-treatments-paper2. Accessed October 27, 2016.
28.
Pharmaceutical Benefits Advisory Committee. Vinflunine, solution concentration for I.V. infusion, 50 mg in 2 mL and 250 mg in 10 mL (as ditartrate), Javlor. 2011 (Public Summary Document). http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2011-11/pbac-psd-vinflunine-nov11. Accessed October 28, 2016.
29.
Krippendorff  K. Computing Krippendorff’s Alpha-Reliability. 2011. (Departmental Papers [ASC]). Report No. 43. http://repository.upenn.edu/asc_papers/43. Accessed October 27, 2016.
30.
Krippendorff  K. Content Analysis: An Introduction to Its Methodology. Second Edition. SAGE Publications; Thousand Oaks, CA. 2004.
31.
US Food and Drug Administration. Drugs@FDA: FDA Approved Drug Products. https://www.accessdata.fda.gov/scripts/cder/drugsatfda/. Published 2015. Accessed December 7, 2015.
32.
European Medicines Agency. European Medicines Agency. European Public Assessment Reports. 2016. http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar_search.jsp&mid=WC0b01ac058001d125. Accessed May 1, 2015.
33.
Sorenson  C, Drummond  M, Kanavos  P.  Ensuring Value for Money in Health Care: The Role of Health Technology Assessment in the European Union. Copenhagen: World Health Organization; 2008.
34.
Howard  DH, Bach  PB, Berndt  ER, Conti  RM.  Pricing in the market for anticancer drugs.  J Econ Perspect. 2015;29(1):139-162. doi:10.1257/jep.29.1.139Google ScholarCrossref
35.
National Institute for Health and Care Excellence. Guide to the Processes of Technology Appraisal. Processes and Methods PMG19. 2014. https://www.nice.org.uk/article/pmg19/chapter/3-the-appraisal-process. Accessed October 26, 2016.
36.
McDermott  U, Downing  JR, Stratton  MR.  Genomics and the continuum of cancer care.  N Engl J Med. 2011;364(4):340-350. doi:10.1056/NEJMra0907178PubMedGoogle ScholarCrossref
37.
National Institute for Health and Care Excellence. Gene expression profiling and expanded immunohistochemistry tests for guiding adjuvant chemotherapy decisions in early breast cancer management: MammaPrint, Oncotype DX, IHC4 and Mammostrat. Diagnostics guidance DG10. 2013. https://www.nice.org.uk/guidance/dg10. Accessed October 27, 2016.
38.
Kim  C, Prasad  V.  Cancer Drugs Approved on the Basis of a Surrogate End Point and Subsequent Overall Survival: An Analysis of 5 Years of US Food and Drug Administration Approvals.  JAMA Intern Med. 2015;175(12):1992-1994. doi:10.1001/jamainternmed.2015.5868PubMedGoogle ScholarCrossref
Original Investigation
March 2017

Assessment of Overall Survival, Quality of Life, and Safety Benefits Associated With New Cancer Medicines

Author Affiliations
  • 1London School of Economics and Political Science, London, England
  • 2Harvard Medical School, Boston, Massachusetts
  • 3Division of Hematology-Oncology, Department of Medicine, Massachusetts General Hospital, Boston
  • 4Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, England
JAMA Oncol. 2017;3(3):382-390. doi:10.1001/jamaoncol.2016.4166
Key Points

Question  What are the overall survival, quality of life, and safety benefits of recently licensed cancer medicines?

Findings  An analysis of health technology assessment reports found that new cancer drugs were associated with increased overall survival by an average of 3.43 months between 2003 and 2013, with 43% increasing overall survival by 3 months or longer, 11% by less than 3 months and 30% were not associated with an increase in overall survival. Most new cancer drugs improved quality of life, and were associated with reduced patient safety.

Meaning  The added benefits of new cancer medicines vary widely across and within therapeutic indications and may be based on modeled data, indirect or nonactive comparisons, or nonvalidated evidence.

Abstract

Importance  There is a dearth of evidence examining the impact of newly licensed cancer medicines on therapy. This information could otherwise support clinical practice, and promote value-based decision-making in the cancer drug market.

Objective  To evaluate the comparative therapeutic value of all new cancer medicines approved by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) between 2003 and 2013.

Design, Setting, and Participants  We used a narrative synthesis approach to systematically synthesize and analyze English, French, and Australian health technology assessments (HTAs) of all new cancer medicines licensed in the United States and Europe between 2003 and 2013.

Interventions  Sixty-two new molecular entities with a primary oncology indication.

Main Outcomes and Measures  Overall survival (OS), quality of life (QoL), and safety.

Results  Of the 62 new active cancer molecules approved by the FDA and EMA between 2003 and 2013, 53 were appraised by English, French, or Australian HTA agencies through May 2015. Of these 53 drugs, 23 (43%) increased OS by 3 months or longer, 6 (11%) by less than 3 months, and 8 (15%) by an unknown magnitude; there was no evidence to suggest that the remaining 16 (30%) increased OS over best alternative treatments. Where overall survival gains could be quantified, all new cancer drugs were associated with a mean (SE) total increase in OS of 3.43 (0.63) months over the treatments that were available in 2003. Drug-related improvements in OS were, however, widely distributed across therapeutic targets—ranging between 0 (thyroid, ascites) and 8.48 months (breast cancers)—and were sometimes based on modeled data, indirect or nonactive comparisons, or nonvalidated evidence. Although 22 (42%) of 53 new medicines were associated with an increase in QoL, 24 (45%) were also associated with reduced patient safety. Of the 53 new cancer drugs, 42 (79%) were associated with at least some improvement in OS, QoL, or safety.

Conclusions and Relevance  Although innovation in the oncology drug market has contributed to improvements in therapy, the magnitude and dimension of clinical benefits vary widely, and there may be reasons to doubt that claims of efficacy reflect real-world effectiveness exactly. These findings raise important questions for clinical decision-making and value-based policy.

Introduction

There are growing questions about the value gained from spending on what seem to be ever more expensive cancer medicines. Rising expenditures may make it difficult for patients to access or remain compliant with life-extending therapies.1,2 Yet, some have argued that high prices may be justified if new and innovative treatments offer significant benefits to patients.2,3 Even as studies point to gains in overall survival (OS) from innovative cancer medicines,4 efforts to examine the value from spending on new cancer drugs remain stymied by a dearth of systematic evidence on their clinical risks and benefits. This lack of evidence makes it difficult for the public to demand more from innovation,5 and, where costs factor into the decision-making process, for clinicians and patients to balance preferences for the expected impact of treatment against rising drug expenditures.

One difficulty in characterizing the clinical impact of new cancer drugs is the multiplicity of outcome measures. Overall survival has traditionally been taken as the gold standard among oncology efficacy endpoints.6,7 The United States Food and Drug Administration (FDA), for its part, designates OS as the only “universally accepted” and the “most reliable” direct measure of benefit in oncology drug trials.8 The European Medicines Agency (EMA) similarly states that convincing favorable effects on OS are, from a clinical and methodological perspective, the “most persuasive outcome” of oncology trials.9 As policymakers adopt new regulations to expedite access to medicines for serious conditions,10,11 interest has grown in measuring effectiveness through surrogate markers.12 These, however, are not yet systematically measured by regulators8,13 and still have inconsistent or uncertain predictive clinical value.14,15 Despite its potential limitations, the American Society for Clinical Oncology’s (ASCO) recently published16 Value Framework in fact recommends that efficacy benefits be measured through progression-free survival or response rates only if OS is not reported.17 In addition to efficacy measures, quality of life (QoL) and safety are also used by regulators and clinicians to fully consider the impact on how patients feel and function owing to treatment.17-19

To shed light on the clinical risks and benefits from new cancer drugs, this study took a narrative synthesis approach to review regulatory assessments of the impact on OS, QoL, and safety from all cancer medicines newly licensed in the US and Europe over the past decade. Since US licensing decisions do not require proof of comparative efficacy and may not consider OS benefits under accelerated licensing procedures,20,21 we extracted and reviewed summary conclusions of drug-related OS, QoL, and safety benefits from English (The National Institute for Health and Care Excellence, NICE), French (Haute Autorité de Santé, HAS), and Australian (Pharmaceutical Benefits Advisory Committee, PBAC) health technology assessment (HTA) agencies. We find that OS benefits vary widely, improvements in OS and QoL often come at the cost of safety, and there are reasons to doubt whether clinical efficacy has been matched by effectiveness in the real world. This study provides additional clarity on the potential risks and benefits of new cancer medicines. It also raises questions about how clinical impact is measured by regulators, how the scientific evidence is used to inform clinical practice, and how much value is generated from cancer drug spending.

Methods
Inclusion and Exclusion Criteria

All new molecular entities (NMEs) approved by the FDA and EMA between 2003 and 2013 with a primary indication for oncology were eligible for inclusion. This study focused exclusively on primary indications, which are likely to reflect main intended use. Drug inclusion criteria were adapted from a previous study by Roberts et al.22 Because this study did not use or access patient-level data the London School of Economics and Political Science determined that ethical approval was not required.

Drug Appraisals

Evaluations of the clinical impact of new cancer drugs were obtained from English, French, and Australian agency HTAs published through May 2015. These agencies regularly publish HTAs in the English language, and are required to evaluate the clinical impact of new medicines in relation to existing standards of treatment that would most likely be replaced by the new intervention.23-25 Health technology assessments were selected for review if they pertained to the same target condition as the first FDA-approved indication. If multiple reports evaluated the same target condition, we selected the latest report that most closely matched the first FDA-approved indication with respect to any treatment restrictions (eg, cancer staging). In ambiguous cases, determinations were made in consultation with a medical expert. Initial EMA-approved indications were used if the drug had not been approved by US regulators through May 2015.

Data Extraction and Synthesis

Two reviewers (S.S.V. and R.L.) adapted the patient, intervention, comparator, outcomes framework26 to independently review assessments of the clinical impact of new cancer medicines. Information on recommended patient populations, novel interventions, and therapeutic comparators were extracted from each drug appraisal, as were evaluations of the impact on OS, QoL, and safety of drug treatment. The Table lists the classes of evidence that were typically evaluated and reported by English, French, and Australian authorities to assess OS, QoL, and safety. This approach captured key outcome measures that are regularly considered during formal drug reviews,8,18,19 and reflected ASCO’s recently published conceptual framework for measuring the value of cancer treatment options.17

A rules-based process was undertaken to evaluate evidence reported by HTA agencies. For this, we considered overall judgments of the available evidence on OS, QoL, and safety from HTA summary sections, their acknowledgement of the significance of clinical trial results, or referral to prior evaluations of the primary evidence. If these were absent, or if an HTA agency concluded that clinical benefits could not be assessed, corresponding extraction parameters were marked as missing. Disagreements on how to interpret HTA agency summaries about the clinical impact of treatment were resolved through consensus. Overall survival benefits were categorized as 3 months or longer, less than 3 months, increase but of unknown magnitude, and no demonstrated increase. A recent ASCO working group has suggested that improvements in median OS of at least 20% may demonstrate a clinically meaningful improvement in survival. However, our approach was designed to reflect the system that is currently used in England to identify OS improvements that are large enough to justify additional expense in end-of-life care,27 and used at times by Australian authorities to assess new health technologies.28 Quality of life and safety benefits were categorized as improvement, reduction, mixed evidence, or no difference relative to the standards of care existing at time of evaluation. A hierarchical process was followed to generate a composite measure of the drug-related effect on OS, QoL, and safety. In instances where assessments were available from multiple HTA agencies, this involved identifying the most positive drug-related survival benefit to represent what may be possible from treatment. For QoL and safety, if 1 HTA agency indicated that the new medicine was associated with an overall improvement in therapy, but another found no change, we classified the drug as producing a net positive gain. If opposing evidence existed, we classified the drug as being associated with mixed evidence. Please see the eMethods section in the Supplement for additional information. Defining features of each drug appraisal were also recorded, and a physician classified all eligible drugs into therapeutic target groups according to their FDA-approved primary indication. A summary of drug-related effects on OS, QoL, and safety is provided in eTable 1 in the Supplement, and an overview of the regulatory evidence used in our analysis is provided in eTable 2 in the Supplement.

Descriptive statistics were used to summarize composite measures of the impact on OS, QoL, and safety of all medicines included in this analysis. Krippendorff’s α coefficient was used to measure the level of agreement among HTA agency assessments of the clinical benefits of treatment, and to inform our interpretation of results.29 Because reliability standards should relate to local requirements,30 our interpretation of this statistic focuses on the difference between coefficients. For convenience, however, we take an α level of .67 or greater to indicate a high level of agreement.30 Finally, to attempt to externally validate our results, we sought informal feedback on our analysis from 2 medical experts from the FDA. After providing their informed consent to participate anonymously, both experts were given a written description of our methods and a copy of all results. They were asked to review all materials and provide feedback as to whether our synthesis was consistent with their own perception of the impact from new cancer medicines on clinical therapy in the United States. Please see the eMethods section in the Supplement for further details on our methodology as well as a discussion of potential limitations.

Results

A total of 62 new active cancer molecules were eligible for this study (Figure 1), 4 of which were approved by the EMA but not the FDA through May, 2015. Molecule descriptors are provided in eTable 2 in the Supplement. Of those 62 drugs, 53 (85%) were assessed for OS by at least 1 of the 3 international HTA agencies considered through May, 2015. The remaining 9 molecules may have been evaluated after our study end-date, not have been reviewed by HTA agencies if considered low-priority,33 or may have been rejected by European (EMA) or national licensing authorities. Of the 53 drugs that were included in this study, 35 drugs were assessed by all 3 agencies, 7 were assessed by 2, and 11 were assessed by 1 agency. In most cases, HTA agency assessments were based on the same set of comparators (eTable 2 in the Supplement).

Overall Survival

Of the 53 drugs that were evaluated, 23 (43%) were confirmed by at least 1 HTA agency to increase OS by at least 3 months, though an exact magnitude of increase could not be estimated by HTA agencies for 6 of these 23 medicines. Six (11%) of the 53 drugs increased OS by less than 3 months, and 8 (15%) produced an increase in OS of unknown magnitude. The remaining 16 (30%) cancer drugs did not demonstrate an increase in OS over alternative treatments, either because no difference was found or because a determination was not or could not be made by HTA agencies on the basis of the available evidence (eTable 1 in the Supplement).

We examined total gains in OS made over the last decade by mapping new interventions against the treatment comparators that would be replaced, as identified in HTAs (Figure 2). In all cases where OS gains could be quantified, new cancer drugs produced a total mean (SE) improvement in OS of 3.43 (0.63) months (or 0.29 [0.05] years) relative to the treatments that were available in 2003. These benefits, however, varied across and within different classes of therapeutics. For instance, drugs indicated for thyroid cancers produced an average (SE) increment of 0 (0) months in OS; ascites, 0 (0) months; lung cancers, 2.09 (0.75) months; hematological cancers, 2.61 (1.69) months; gastrointestinal cancers, 2.90 (1.12) months; prostate cancers, 3.17 (0.69) months; skin cancers, 4.65 (1.05) months; renal cancers, 6.27 (1.92) months; and breast cancers, 8.48 (3.84) months.

For drugs that were assessed by all 3 agencies, English authorities were most likely to attribute significant OS improvements to new medicines, while Australian authorities were least likely to do so (eFigure 1 in the Supplement). Across the entire sample, Krippendorff’s α was 0.38, suggesting a low to moderate level of agreement in assessments of OS benefits among all 3 HTA agencies (eTable 3 in the Supplement). Interagency agreement was, however, higher when English evaluations were excluded and for the set of drugs that produced marginal to no improvement in OS (eTable 3 in the Supplement). This may suggest that regulators become increasingly uncertain about claims of drug-related survival benefits as the magnitude of those claims increases.

Indeed, HTA agency conclusions for 10 of the 23 drugs that were deemed to increase OS by 3 months or more (axitinib, bosutinib, crizotinib, everolimus, panitumumab, pazopanib, pomalidomide, sorafenib, sunitinib, and trabectedin) were based on modeled data, indirect comparisons, or agency opinions. For 5 of the 23 drugs (axitinib, crizotinib, enzalutamide, panitumumab, and pazopanib), significant OS benefits were also found relative to 1 treatment comparator, but were not established in relation to other possible comparators.

Quality of Life

Of the 53 drugs that were evaluated by at least 1 HTA agency, 22 (42%) improved QoL, 2 (4%) reduced QoL, 1 (2%) was associated with mixed evidence, and 28 (53%) did not demonstrate a difference in QoL relative to best alternative treatments (eTable 1 in the Supplement).

As for OS, England’s HTA agency was most likely to find that new cancer drugs improved QoL (eFigure 1 in the Supplement). Across the entire sample, there was a moderate to high level of agreement among HTA agencies in the assessed level of QoL benefit from new cancer drugs (α, 0.61) (eTable 3 in the Supplement). This suggests that HTA agencies tend to similarly interpret the QoL evidence—more so than that of OS—and lends confidence to the notion that new cancer drugs are providing QoL benefits to patients.

Still, not all regulatory opinions were based on robust evidence. Of the 22 drugs that were deemed to improve QoL, evaluations for 17 were based on a review of empirical evidence, including data from validated QoL instruments. The QoL benefits associated with the remaining 5 drugs (pertuzumab, trametinib, ziv-aflibercept, sipuleucel-T, and vemurafenib) were based exclusively on testimony from patient representatives and clinical experts.

Safety

Eight (15%) of the 53 drugs that were evaluated by HTA agencies were found to improve safety. A far larger share (24, or 45%), however, reduced patient safety. Ten (19%) were associated with mixed evidence and 11 (21%) did not demonstrate any difference in safety compared with alternative treatments (eTable 1 in the Supplement).

Mirroring earlier trends for OS, English and Australian authorities were least and most likely to determine that new cancer drugs reduced patient safety, respectively (eFigure 1 in the Supplement). Across the entire sample, there was a low level of agreement between HTA agencies on the impact on safety from new cancer medicines. This was however driven by a lack of consensus with Australia’s HTA agency: interagency agreement was moderate to high when limited to English and French assessments (eTable 3 in the Supplement).

Clinical Benefits of Treatment

Of the 23 drugs that significantly increased OS by at least 3 months, 15 (65%) were also found to improve QoL, while the remaining 8 (35%) produced no measurable change. In contrast, of the 23 drugs that significantly extended OS, 5 (22%) improved safety, 11 (48%) reduced safety, 5 (22%) were associated with mixed evidence, and 2 (9%) produced no difference in safety relative to existing standards of care. Most new cancer medicines that significantly extend life therefore also improve QoL, but reduce patient safety (eTable 1 in the Supplement).

There was a noticeably smaller improvement in QoL in the drugs that produced marginal to no improvement in OS. Of the 30 evaluated drugs that did not increase OS by at least 3 months, 7 (23%) were found to improve QoL, 2 (7%) worsened QoL, 1 (3%) had a mixed effect, and 20 (67%) were not associated with any effect on QoL. Safety nevertheless remained a concern. Of the 30 drugs that did not increase OS by at least 3 months, 3 (10%) were classified as improving safety, 13 (43%) reduced patient safety, 5 (17%) were associated with mixed evidence; the remaining 9 (30%) did not demonstrate any difference in safety over alternative treatments.

Across the entire sample, 42 of the 53 new cancer medicines (79%) licensed in the United States and the European Union between 2003 and 2013, and evaluated by English, French, and Australian HTA agencies, demonstrated at least some evidence of an OS, QoL, or safety benefit. These results were supported by the feedback that we received from 2 medical experts from the FDA, both of whom generally agreed with the results that were obtained. One—an oncologist—stated that the results summarized in eTable 1 in the Supplement were “in line with [his personal] perceptions” of the added clinical benefits of the new cancer medicines.

Discussion

All new cancer drugs licensed between 2003 and 2013 by the FDA and EMA extended OS by an mean (SE) of 3.43 (0.63) months (0.29 [0.05] years) over the treatments that were available in 2003. This figure is based on regulatory assessments and is consistent with those reported by similar studies.34

While perhaps modest, this OS benefit represents an important step forward for patients and society, as even minor improvements in survival can have an effect on reducing mortality at the population level. It is encouraging to therefore find that most new cancer drugs were associated with some known (55%) or at least unknown (70%) OS benefit, with the largest share (43%) extending life by an amount that English and Australian regulators consider to be clinically meaningful (≥3 months).

To our knowledge, this analysis is the first to take a systematic approach to evaluate the OS, QoL, and safety benefits associated with new cancer drugs. Our findings indicate that most newly approved cancer medicines (79%) increased OS by some known or unknown magnitude, or demonstrated at least some evidence of improved QoL or safety over alternative treatments. In general, innovation in the oncology drug market therefore appears to be bringing real value to patients and society.

There was evidence to suggest that these benefits are also concentrated in particular classes of therapeutics. Ten immunologic drugs were present in our sample, most of which function by antigenic targeting of cancer cells. Ipilimumab was the only drug of a novel class of immunomodulating agents, the immune checkpoint modulators. With the exception of bevacizumab—which elicits an antiangiogenic response—immunologic drugs were, on average, better at extending OS compared to nonimmunologic drugs (5.02 vs 2.30 months). However, this was not true of all immunologic drugs. Perhaps owing to a limited sample size, statistical testing also showed that this group difference was nonsignificant, and that there was no greater effect on quality of life or safety (data not shown). Ipilimumab was itself associated with a marginally larger OS benefit (5.7 months). Future studies may adapt our methodology to examine the efficacy of the newer immune checkpoint modulators, such as nivolumab and pembrolizumab.

Though perhaps promising, findings from this study should be interpreted with caution. To validly draw inference on the impact from new immunologic drugs and other cancer therapeutics, this analysis should be repeated as the number of available molecules grows. Across the entire sample, regulatory evidence is sometimes based on modeled data, nonvalidated inputs, or comparisons against nontargeted or older active treatments (eg, BSC, chlorambucil), though this may reflect the state of clinical practice. Even if these issues are ignored, interagency agreement on drug-related OS benefits decreases as the level of benefit increases, indicating that there may be greater uncertainty about the value from new cancer drugs that claim to bring the greatest health benefit. And, as shown with frequently contrasting English and Australian assessments, the regulatory milieu seems to shape the interpretation of evidence on the clinical impact from new cancer medicines. These findings raise important questions about how clinical benefits are measured and used to inform evidence-based policy, and they give reason to adapt treatment guidelines to the unique circumstances and preferences of the patient.

Regulators nevertheless often have the authority to require submission of all applicable clinical data that is "necessary to address the remit and scope of the technology appraisal."35 To estimate the clinical value of new medicines in the absence of real-world observational data, the approach used in this study may therefore be preferable to secondary reviews of the published scientific literature.

Still, technological assessments may not always accurately reflect the full extent of clinical risks and benefits that are observed in practice. For instance, as is the case for KRAS expression in colon cancer, particular genomic profiles are now known to predict OS benefits. In part for this reason, gene expression profiling is increasingly recommended as a tool to guide chemotherapy decisions.36,37 Since many new anticancer drugs target proteins that are downstream of genes with driver somatic mutations,36 any misapprehension about the genetic mediators of disease may prevent regulators from fully appreciating their clinical value. Validated biomarkers in fact often do not exist to guide the selection of patients in clinical trials who would most likely benefit from treatment.5 Clinical practice may instead incorporate new evidence on the genetic predictors of response as and when it develops,36 enabling personalized and cost-efficient care that optimizes patient outcomes. To better reveal the real-world benefits from new cancer medicines, future studies should therefore periodically repeat this analysis with postmarketing,38 observational or pragmatic clinical trial evidence. The National Cancer Institute’s upcoming National Cancer Knowledge System may provide crucial insights in this regard.

As it stands, 1 in 3 (30%) of all newly approved cancer medicines are not associated with any OS benefit, while 1 in 5 (20%) neither extend life nor improve QoL or safety. While perhaps reflective of nonactive comparisons, the approval of new medicines for orphan indications with no alternative treatment, or the growing use of surrogate efficacy endpoints during regulatory evaluations,12 these findings suggest that expenditures for up to 1 out of every 5 new cancer drugs may be spent without any OS, QoL, or safety benefit to the patient.

In the short term, these findings help to inform clinical decision-making by patients and clinicians who, in personalizing treatment, may have to consider the economic implications of drug prescriptions alongside individual preferences for treatment-related risks and benefits. This may be true for US cancer patients, who typically shoulder high amounts of cost-sharing, but also if public health systems (eg, England’s NHS) do not publicly reimburse for new cancer medicines. Over the longer term, efforts should be made to develop evidence on mechanisms to weight clinical outcome measures according to their value to patients, and to align these initiatives with the regulatory review process.

These findings raise a number of important questions about value-for-money in oncology. We find that there is in fact a wide distribution in the therapeutic benefits associated with recent cancer drug innovations, suggesting a similarly wide variation in the value that they bring to society. Some medications (eg, pertuzumab) have significantly extended life, perhaps giving reason for large and growing expenditures. Others, however, appear to bring little to no tangible benefit to health, raising questions about the justification for additional expense over alternative treatments. Though further research is needed, our analysis may indicate that spending on new cancer drugs is not always commensurate with their clinical benefits. This may be reason for patients and clinicians to take pause when considering new treatments, particularly if related expenditures are of concern.

Conclusions

Cancer drug innovation over the past decade is, on the whole, expected to have contributed to improvements in patient OS and QoL. These gains, however, are unevenly distributed across all newly licensed medicines, often come at the cost of safety, and may not always translate to real-world practice. As calls for value-based health care grow, this analysis raises questions about how clinical benefits are measured by regulators, how regulatory guidance is used to inform clinical decision-making, and how much value is generated from spending in the oncology drug market.

Back to top
Article Information

Corresponding Author: Elias Mossialos, MD, PhD, London School of Economics and Political Science, Houghton Street, London WC2A 2A, England (e.a.mossialos@lse.ac.uk).

Accepted for Publication: August 3, 2016.

Published Online: December 29, 2016. doi:10.1001/jamaoncol.2016.4166

Author Contributions: Mr Salas-Vega and Dr Mossialos had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Salas-Vega, Mossialos.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Salas-Vega.

Administrative, technical, or material support: Salas-Vega, Mossialos.

Study supervision: All authors.

Funding/Support: Funding was provided by LSE Health and the Institute of Global Health Innovation, Imperial College London.

Role of the Funder/Support: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Conflict of Interest Disclosures: None reported.

Additional Contributions: We thank Rayden Llano, MSc, London School of Economics and Political Science, for acting as second reviewer in the evaluation of cancer drug appraisals and for his contributions to data collection and analysis. We also thank Dr Jia Hu, MD, University of Toronto, for his medical advice, Aris Angelis, MSc, London School of Economics and Political Science, for his helpful comments as well as Emily Shearer, MSc, London School of Economics and Political Science, for her feedback on our manuscript. None of the additional contributors received any compensation.

References
1.
Weaver  KE, Rowland  JH, Bellizzi  KM, Aziz  NM.  Forgoing medical care because of cost: assessing disparities in healthcare access among cancer survivors living in the United States.  Cancer. 2010;116(14):3493-3504. doi:10.1002/cncr.25209PubMedGoogle ScholarCrossref
2.
Experts in Chronic Myeloid Leukemia.  The price of drugs for chronic myeloid leukemia (CML) is a reflection of the unsustainable prices of cancer drugs: from the perspective of a large group of CML experts.  Blood. 2013;121(22):4439-4442. doi:10.1182/blood-2013-03-490003PubMedGoogle ScholarCrossref
3.
Mailankody  S, Prasad  V.  Five years of cancer drug approvals: innovation, efficacy, and costs.  JAMA Oncol. 2015;1(4):539-540. doi:10.1001/jamaoncol.2015.0373PubMedGoogle ScholarCrossref
4.
Apolone  G, Joppi  R, Garattini  S.  Ten years of marketing approvals of anticancer drugs in Europe: regulatory policy and guidance documents need to find a balance between different pressures.  Brit J Cancer. 2005;93(5):504-509. doi:10.1038/sj.bjc.6602750PubMedGoogle ScholarCrossref
5.
Ellis  LM, Bernstein  DS, Voest  EE,  et al.  American Society of Clinical Oncology perspective: Raising the bar for clinical trials by defining clinically meaningful outcomes.  J Clin Oncol. 2014;32(12):1277-1280. doi:10.1200/JCO.2013.53.8009PubMedGoogle ScholarCrossref
6.
Han  K, Ren  M, Wick  W,  et al.  Progression-free survival as a surrogate endpoint for overall survival in glioblastoma: a literature-based meta-analysis from 91 trials.  Neuro Oncol. 2014;16(5):696-706. doi:10.1093/neuonc/not236PubMedGoogle ScholarCrossref
7.
Beauchemin  C, Johnston  JB, Lapierre  MÈ, Aissa  F, Lachaine  J.  Relationship between progression-free survival and overall survival in chronic lymphocytic leukemia: a literature-based analysis.  Curr Oncol. 2015;22(3):e148-e156. doi:10.3747/co.22.2119PubMedGoogle ScholarCrossref
8.
Food and Drug Administration. Guidance for Industry: Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics. http://www.fda.gov/downloads/Drugs/Guidances/ucm071590.pdf. Accessed October 26, 2016.
9.
European Medicines Agency. Guideline on the evaluation of anticancer medicinal products in man. 2012. Report No.: EMA/CHMP/205/95/Rev.4. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/01/WC500137128.pdf. Accessed October 27, 2016.
10.
Johnson  JR, Ning  Y-M, Farrell  A, Justice  R, Keegan  P, Pazdur  R.  Accelerated approval of oncology products: the food and drug administration experience.  J Natl Cancer Inst. 2011;103(8):636-644. doi:10.1093/jnci/djr062PubMedGoogle ScholarCrossref
11.
112th Congress. Food and Drug Administration Safety and Innovation Act. 126 Stat. United States of America; 2012.
12.
McCain  JA.  The ongoing evolution of endpoints in oncology.  Manag Care. 2010;19(5):Suppl 1.Google Scholar
13.
Booth  CM, Eisenhauer  EA.  Progression-free survival: meaningful or simply measurable?  J Clin Oncol. 2012;30(10):1030-1033. doi:10.1200/JCO.2011.38.7571PubMedGoogle ScholarCrossref
14.
Shi  Q, Sargent  DJ.  Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials.  Int J Clin Oncol. 2009;14(2):102-111. doi:10.1007/s10147-009-0885-4PubMedGoogle ScholarCrossref
15.
Davis  S, Tappenden  P, Cantrell  A. A review of studies examining the relationship between progression-free survival and overall survival in advanced or metastatic cancer. http://www.nicedsu.org.uk/PFSOS%20Report.FINAL.06.08.12.pdf. Accessed October 28 2016.
16.
Angelis  A, Kanavos  P.  Critique of the American Society of Clinical Oncology Value Assessment Framework for Cancer Treatments: putting methodologic robustness first.  J Clin Oncol. 2016;34(24):2935–2936PubMedGoogle ScholarCrossref
17.
Schnipper  LE, Davidson  NE, Wollins  DS,  et al; American Society of Clinical Oncology.  American Society of Clinical Oncology Statement: A Conceptual Framework to Assess the Value of Cancer Treatment Options.  J Clin Oncol. 2015;33(23):2563-2577. doi:10.1200/JCO.2015.61.6706PubMedGoogle ScholarCrossref
18.
Johnson  JR, Williams  G, Pazdur  R.  End points and United States Food and Drug Administration approval of oncology drugs.  J Clin Oncol. 2003;21(7):1404-1411.PubMedGoogle ScholarCrossref
19.
Szende  A, Leidy  NK, Revicki  D.  Health-related quality of life and other patient-reported outcomes in the European centralized drug regulatory process: a review of guidance documents and performed authorizations of medicinal products 1995 to 2003.  Value Health. 2005;8(5):534-548. doi:10.1111/j.1524-4733.2005.00051.xPubMedGoogle ScholarCrossref
20.
Temple  R.  A regulator’s view of comparative effectiveness research.  Clin Trials. 2012;9(1):56-65.PubMedGoogle ScholarCrossref
21.
US Food and Drug Administration. Guidance for Industry: Expedited Programs for Serious Conditions—Drugs and Biologics. 2014. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM358301.pdf. Accessed October 28, 2016.
22.
Roberts  SA, Allen  JD, Sigal  EV.  Despite criticism of the FDA review process, new cancer drugs reach patients sooner in the United States than in Europe.  Health Aff (Millwood). 2011;30(7):1375-1381. doi:10.1377/hlthaff.2011.0231PubMedGoogle ScholarCrossref
23.
National Institute for Health and Care Excellence. NICE technology appraisal guidance. 2016. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance.
24.
Autorité de Santé  Haute. Medical Device Assessment in France: Guidebook.; 2009.http://www.has-sante.fr/portail/upload/docs/application/pdf/2010-03/guide_dm_gb_050310.pdf. Accessed October 28, 2016.
25.
Pharmaceutical Benefits Advisory Committee. About the Guidelines. 2014https://pbac.pbs.gov.au/information/about-the-guidelines.html.
26.
National Institute for Health and Care Excellence. Developing review questions and planning the systematic review. In: The guidelines manual. NICE; 2012. https://www.nice.org.uk/article/pmg6/chapter/4-Developing-review-questions-and-planning-the-systematic-review. Accessed October 27, 2016.
27.
National Institute for Health and Care Excellence. Appraising life-extending, end of life treatments. 2009. https://www.nice.org.uk/guidance/gid-tag387/resources/appraising-life-extending-end-of-life-treatments-paper2. Accessed October 27, 2016.
28.
Pharmaceutical Benefits Advisory Committee. Vinflunine, solution concentration for I.V. infusion, 50 mg in 2 mL and 250 mg in 10 mL (as ditartrate), Javlor. 2011 (Public Summary Document). http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2011-11/pbac-psd-vinflunine-nov11. Accessed October 28, 2016.
29.
Krippendorff  K. Computing Krippendorff’s Alpha-Reliability. 2011. (Departmental Papers [ASC]). Report No. 43. http://repository.upenn.edu/asc_papers/43. Accessed October 27, 2016.
30.
Krippendorff  K. Content Analysis: An Introduction to Its Methodology. Second Edition. SAGE Publications; Thousand Oaks, CA. 2004.
31.
US Food and Drug Administration. Drugs@FDA: FDA Approved Drug Products. https://www.accessdata.fda.gov/scripts/cder/drugsatfda/. Published 2015. Accessed December 7, 2015.
32.
European Medicines Agency. European Medicines Agency. European Public Assessment Reports. 2016. http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar_search.jsp&mid=WC0b01ac058001d125. Accessed May 1, 2015.
33.
Sorenson  C, Drummond  M, Kanavos  P.  Ensuring Value for Money in Health Care: The Role of Health Technology Assessment in the European Union. Copenhagen: World Health Organization; 2008.
34.
Howard  DH, Bach  PB, Berndt  ER, Conti  RM.  Pricing in the market for anticancer drugs.  J Econ Perspect. 2015;29(1):139-162. doi:10.1257/jep.29.1.139Google ScholarCrossref
35.
National Institute for Health and Care Excellence. Guide to the Processes of Technology Appraisal. Processes and Methods PMG19. 2014. https://www.nice.org.uk/article/pmg19/chapter/3-the-appraisal-process. Accessed October 26, 2016.
36.
McDermott  U, Downing  JR, Stratton  MR.  Genomics and the continuum of cancer care.  N Engl J Med. 2011;364(4):340-350. doi:10.1056/NEJMra0907178PubMedGoogle ScholarCrossref
37.
National Institute for Health and Care Excellence. Gene expression profiling and expanded immunohistochemistry tests for guiding adjuvant chemotherapy decisions in early breast cancer management: MammaPrint, Oncotype DX, IHC4 and Mammostrat. Diagnostics guidance DG10. 2013. https://www.nice.org.uk/guidance/dg10. Accessed October 27, 2016.
38.
Kim  C, Prasad  V.  Cancer Drugs Approved on the Basis of a Surrogate End Point and Subsequent Overall Survival: An Analysis of 5 Years of US Food and Drug Administration Approvals.  JAMA Intern Med. 2015;175(12):1992-1994. doi:10.1001/jamainternmed.2015.5868PubMedGoogle ScholarCrossref
×