To be awarded ASCO bonus points, a drug must meet 2 thresholds: achieving a 50% or greater improvement (blue dotted line) in proportion of patients alive with the test regimen at twice the median overall survival or progression-free survival time point (9 drugs met this requirement) (A) and 20% of patients surviving (blue dotted line) with the standard regimen (3 drugs met this requirement) (B). Only 3 drugs reached both thresholds and met ASCO criteria for bonus points. Drugs using objective response rate as the primary end point are not eligible to gain the bonus points related to the survival tail and are therefore not presented here. HNSCC indicates head and neck squamous cell carcinoma; NSCLC, non–small cell lung cancer.
eTable 1. Tail of the curve bonus points detailed calculation
eTable 2. ASCO NHB detailed calculation
eTable 3. ASCO toxicity score detailed calculation
eTable 4. Median survival index sensitivity analysis
eTable 5. Difference in survival at median OS—test regimen versus standard care—applying an alternative threshold of 7.5%
Customize your JAMA Network experience by selecting one or more topics from the list below.
Ben-Aharon O, Magnezi R, Leshno M, Goldstein DA. Association of Immunotherapy With Durable Survival as Defined by Value Frameworks for Cancer Care. JAMA Oncol. 2018;4(3):326–332. doi:10.1001/jamaoncol.2017.4445
Which modern immuno-oncology agents reach the threshold for bonus points for durable survival in the American Society of Clinical Oncology (ASCO) value framework for cancer care?
In this analysis of 23 indications for 6 immuno-oncology agents approved by the US Food and Drug Administration for metastatic solid tumors between March 2011 and August 2017, only 3 drug indications gained durable survival bonus points under the ASCO framework.
Durable survival and response rates for modern immuno-oncology agents are rarely recognized as significant by the ASCO value framework.
Modern immuno-oncology agents have generated great excitement because of their potential to provide durable survival for some patients. However, there is concern regarding the cost of cancer care, and multiple frameworks have been developed to assess value. The American Society of Clinical Oncology (ASCO) framework awards bonus points if substantial durable survival is demonstrated.
To assess whether modern immuno-oncology agents reach defined efficacy thresholds in value frameworks.
Design, Setting, and Participants
In this analysis, all US Food and Drug Administration (FDA) approvals for immuno-oncology agents between March 2011 and August 2017 were reviewed. Data required for the ASCO framework were collected, specifically improvement in proportion of patients alive with the test regimen and survival rate with standard treatment.
Main Outcomes and Measures
Awarding of bonus points for durable survival based on the ASCO criteria.
Twenty-three metastatic indications for 6 immuno-oncology agents (ipilimumab, pembrolizumab, nivolumab, atezolizumab, avelumab, and durvalumab) were approved by the FDA from March 2011 to August 2017. Ten (43%) of the approvals were based on survival end points, while 13 (57%) were based on response rates. Only 3 drug indications fulfilled the threshold defined for the survival rate of patients receiving standard care (minimum 20%). Nine indications achieved the required level of improvement in proportion to patients alive in the test regimen compared with the standard (above 50%). There was overlap between these 2 criteria for 3 drug indications, allowing them to gain the durable survival bonus points awarded by the ASCO framework.
Conclusions and Relevance
Durable survival and response rates of modern immuno-oncology agents are rarely recognized as significant by current oncology value frameworks. This may be due to insufficient demonstration of efficacy of such agents or inappropriately calibrated value frameworks.
The field of immuno-oncology is redefining patterns of cancer care. In some cancer types, modern immuno-oncology agents have demonstrated significant benefits in durable survival. In metastatic cancer care, traditional cytotoxic chemotherapies have demonstrated increases in survival but usually without durability. These agents successfully prolonged survival, but with inevitable biological resistance. With the arrival of modern immuno-oncology agents, this pattern has changed. While most patients have responses that are not durable, a small proportion demonstrate more durable responses. In that context, durable survival reflects the prevalence of long-term survivors. For example, the pivotal clinical trial1 analyzing the safety and efficacy of ipilimumab indicated for metastatic melanoma demonstrated that after 5 years of follow-up, 16% of patients were alive, compared with 8% of patients who received cytotoxic chemotherapy.
Concerns related to the cost and value of cancer drugs have accompanied these advances in immuno-oncology. Efforts to evaluate the relationship between efficacy and cost of medications have led to the creation of various value frameworks. Since 2015 several oncology-specific frameworks have been introduced, and while organizations define value conceptually as a measure of treatment benefits relative to cost, each organization uses a different method to measure value.2 The American Society of Clinical Oncology (ASCO) and the European Society for Medical Oncology (ESMO) published proposed frameworks for assessing the value of various cancer treatments.3,4 The goal was to evaluate selected treatment regimens on the basis of their clinical benefit and toxic effects. ASCO chose to define value in cancer care by emphasizing 3 critical elements: clinical benefit (efficacy), toxic effects (safety), and cost (efficiency). The ESMO Magnitude of Clinical Benefit Scale also considers the impact of a drug’s efficacy and adverse events but without analyzing the cost. The framework developed by the National Comprehensive Cancer Network (NCCN)5 uses a building block method to visually demonstrate levels of 5 key value measures: efficacy, safety, quality of evidence, consistency of evidence, and affordability. The framework used by the Institute for Clinical and Economic Review (ICER)6 incorporates efficacy, safety, and cost in an economic model analogous to the health technology assessments performed by the National Institute for Health and Care Excellence (NICE) in the United Kingdom. The Memorial Sloan Kettering Cancer Center designed the DrugAbacus,7 a tool that may help determine an appropriate price for a specific agent based on possible components of a drug’s value.
These frameworks use various methods to assess value. With the arrival of modern immuno-oncology agents, it is important to evaluate each framework’s ability to account for durable survival as demonstrated by plateaus in the tails of survival curves. The ESMO guidelines provide no specific consideration related to the tails of survival curves. The NCCN value framework incorporates novel end points, eg, immune-related response rates for immuno-oncology agents; however, because this ranking uses subjective value assessments, its process is less transparent.2 The ICER framework does not dedicate a specific index to the tail of survival curves, but its economic modeling does provide the potential to incorporate this issue in the future. In a recent report regarding the treatment of lung cancer,8 ICER acknowledged the lack of data to assess whether there is a long tail of responders beyond 2 years, but recognized the importance of these data in understanding the potential benefit of modern immuno-oncology agents. The flexibility of the ICER model will be able to incorporate such plateaus when clinical data become available. A summary of the main elements in each value framework is presented in Table 1.
A recent update of the ASCO framework awards bonus points if durable survival is demonstrated on the tail of survival curves. In the original ASCO framework, a treatment gained a score, the net health benefit (NHB), based on its clinical benefit and toxic effects. The maximum NHB score is 180 for the advanced disease (metastatic) framework. In May 2016, ASCO published a revised framework incorporating additional features such as quality of life and durable survival.9 Traditionally, the benefit of cancer therapies has been described in terms of the median survival benefit or the hazard ratio. However, the main reason for excitement regarding immuno-oncology is the potential for long-term durable survival in a small proportion of patients. This appears as a plateau on the tail of a survival curve.10
The revised ASCO framework emphasizes the importance of achieving durable overall survival (OS) or progression-free survival (PFS) by awarding bonus points if certain criteria are met (Box). The revisions to the ASCO framework are intended to assess whether a new regimen offers improved survival that is statistically significant. Although the absolute time in months or years will vary for diseases that differ in natural history and sensitivity to therapy, the ASCO Value in Cancer Care Task Force awards points for an outcome of long-term disease control.9
The revised ASCO framework emphasizes the importance of achieving improved survival or long-term disease control (progression-free survival) by awarding bonus points if certain criteria are met. As a study matures, the tail of the survival curve can reflect a noteworthy change for a significant minority of patients receiving a new regimen, assuming the difference between comparator and test regimen is statistically significant.
Bonus points are awarded if the test regimen results in at least a 50% relative improvement in percentage of patients alive at a time point that is at twice the median overall or progression-free survival point for the control regimen and if at least 20% of patients receiving the control regimen are alive at this time.
The framework awards 20 points if the improvement is in overall survival, and 16 points (0.8 × 20) if the improvement is in progression-free survival.
Bonus points are awarded if the following targets are met: (1) the clinical study demonstrates results of a common survival index (OS or PFS); (2) the time frame is at twice the median survival index; (3) at least 20% of patients receiving the standard care (control arm) survived; and (4) there is a minimum 50% improvement in proportion of patients alive with the test regimen. This algorithm can be illustrated by analyzing ipilimumab, the first modern immuno-oncology agent to be approved by the US Food and Drug Administration (FDA), in 2011. In the clinical registration study,11 it was found that median OS was 10.0 months among patients receiving ipilimumab plus the glycoprotein 100 cancer vaccine, as compared with 6.4 months among patients receiving the vaccine alone. At 12.8 months, 22.2% of the patients in the control regimen survived (approximately 30 of 136 patients, slightly higher than the defined 20% threshold, and 38.1% of the patients in the test arm survived (approximately 153 of 403 patients), which implies improvement of survival in the test regimen of 71.4% (above the 50% threshold). In this example, the indication met both requirements (20% survival in the control group and 50% improvement in survival in the treatment group) and gained ASCO framework bonus points.
The ASCO framework is the only framework that has attempted to develop a structured weighting system on the benefit side,12 although the DrugAbacus does have a weighting system for additional attributes. Still, a critical challenge of the ASCO framework is ensuring that the tail of the curve is appropriately weighted in the efficacy calculation. However, the justification for the specific thresholds adopted is not clear.
The objective of this study was to examine which of the modern immuno-oncology agents approved by the FDA fulfill the durable survival threshold defined in the updated ASCO value framework.
In this analysis, FDA approvals for modern immuno-oncology agents (ipilimumab, pembrolizumab, nivolumab, atezolizumab, avelumab, and durvalumab) from March 25, 2011, to August 1, 2017, were reviewed indication by indication. The information was extracted from the FDA hematology/oncology approvals notifications.13 We included drugs used in the metastatic setting of treatment of solid tumors. No institutional review board approval was required for this study, as the material used was publicly available and was not direct patient data.
Data regarding study end points were collected, specifically the parameters defined in the ASCO 2016 value framework for awarding bonus points for the survival curve tail: median OS, PFS, or recurrence-free survival (RFS) (in months); rate of patients surviving with standard treatment at twice the median survival index; and improvement in proportion of patients alive with the test regimen.
These parameters were used to establish whether each drug reaches the required threshold of 50% or greater improvement in proportion of patients alive with the test regimen at twice the median OS or PFS and whether a minimum of 20% of the patients survived with standard care (the control group).
Clinical outcomes in FDA approvals were cross-checked with the results reported in the respective registration studies. The reported outcome in each published study was then compared with end points used for approval of the drug by the FDA.
Only 3 modern immuno-oncology agents reached the required threshold to gain bonus points for durable survival in the ASCO value framework, 2 of which received the maximum potential score (20 points).
Twenty-three indications for 6 immuno-oncology agents were approved by the FDA for metastatic solid tumors from March 2011 to August 2017 (Table 2). Any approvals that were for adjuvant therapy or for hematologic malignant neoplasms were excluded from analysis. Ten approvals (43%) were based on survival end points (OS, PFS, or RFS), while the remaining 13 approvals (57%) were based on objective response rate. The approvals were for melanoma (6 approvals [26%]),11,14-18 non–small cell lung cancer (NSCLC) (6 approvals [26%]),19-23 urothelial carcinoma (5 approvals [22%]),24-26 head and neck cancer (2 approvals [9%]),27,28 microsatellite instability–high cancer (2 approvals [9%]), renal carcinoma (1 approval [4%]),29 and Merkel cell carcinoma (1 approval [4%]).30
Three drug indications fulfilled the criteria of a minimum 20% survival rate with the standard care (control group): ipilimumab indicated for second-line treatment of melanoma (22%), nivolumab for first-line treatment of melanoma (25%), and nivolumab for second-line treatment of squamous NSCLC (22%). Only a small number of patients (≤10%) were alive at twice the median OS time point with all indications except for pembrolizumab indicated for first-line treatment of melanoma, in which 18% of the patients survived at twice the median OS time point.
Nine drugs achieved the required 50% improvement in patients alive in the test regimen compared with the standard. The most significant improvement was found for pembrolizumab indicated for first-line treatment of NSCLC. At 12.0 months (twice the time point of the comparator median PFS [6.0 months]), 22 of 154 patients (14.3%) in the test group (pembrolizumab, 200 mg every 3 weeks) had no progression, compared with 9 of 151 (6.0%) in the control group (platinum-based chemotherapy)—an improvement of 140%. Table 2 shows that an additional 8 drug indications demonstrated an improvement above the 50% threshold (range, 71%-132%). The ninth indication showed no improvement in durable survival because of the relatively short follow-up period in the clinical study.
As there was an overlap between the 2 requirements of the ASCO framework (Figure)—the improvement in the proportion of patients alive at twice the median OS or PFS time point assuming at least 20% of the patients surviving with standard treatment—3 drug indications gained the long tail bonus points in the ASCO framework. According to the ASCO definition, only studies using OS as a primary end point may gain the maximum score (20 points). Of the aforementioned 3 cases, nivolumab indicated for first-line treatment of melanoma received only 16 points as the examination was based on PFS data (median survival data for the test regimen were not reached in the registration study).
To further investigate the data distribution, a sensitivity analysis of the median survival measure was conducted. The ASCO thresholds were checked for the long tail survival using a multiplier for the median OS or PFS of 1.25, 1.5, and 1.75 instead of 2. As demonstrated in eTable 1 in the Supplement, we found that at 1.5 and 1.75 times the median time points, only 1 additional agent—pembrolizumab indicated for first-line treatment of melanoma—would reach both thresholds. At 1.25 times the median time point, the improvements were not significant based on the ASCO threshold of 50%.
When analyzing the clinical trials of modern immuno-oncology agents approved by the FDA in light of the ASCO framework, we found that only 3 of 23 drug indications gained the bonus points for durable survival benefits. There are 2 possible reasons for this. The first is that the public excitement concerning durable survival with immuno-oncology may lack sufficient supporting data. The second is that the ASCO value framework may be insufficiently calibrated to reward durable survival benefits.
Various value-based frameworks attempt to address increasing health care costs, specifically the cost of oncology pharmaceuticals. The tremendous interest in immuno-oncology is focused on the tail of the survival curve of these innovative drugs. As a result, the updated ASCO framework appropriately aimed to reward durable survival in its NHB calculation, as detailed in eTables 2 and 3 in the Supplement.
A recent report8 by ICER on treatment options for advanced NSCLC raised questions about the appropriate analysis of immuno-oncology survival curves. The report suggested that the proportional hazards assumption may not be valid, and that there may be a long survival tail among responders to therapy. The ICER analysis indicated that the difficulty in using a proportional hazards model is caused by 2 populations in the immuno-oncology arms of the trials: a majority of patients who do not have sustained responses to therapy and have a high hazard for progression and mortality, and a minority of patients who do have sustained responses and have a much lower hazard. We have relatively little data to assess whether there is a sustained durable survival beyond 2 years in the case of NSCLC, but this is clearly an important issue in understanding the potential benefit of immuno-oncology.
High-quality biomarkers may be useful in immuno-oncology. Testing for programmed death-ligand 1 (PD-L1) has been proven to be useful in first-line treatment of NSCLC with pembrolizumab in order to select the patients most likely to benefit. However PD-L1 testing may not be useful for other cancer types and with other drugs. As the science of biomarkers improves, the value of immuno-oncology may also improve as we are better able to determine which patients will most likely benefit from this therapy.
The trend of trials allowing patients to cross over to the test group of new immuno-oncology agents increases the need to use PFS in the approval process. Therefore, achieving the durable survival threshold may be even more challenging in the future.
In an era of new immuno-oncology agents, understanding efficacy purely by the median survival is hazardous. As some therapies are expected to provide durable responses to a small percentage of patients, research needs to focus on the tail of survival curves. This is possible with modeling techniques in which the survival curves are digitized to incorporate differences in the distribution of outcomes beyond the point estimate of the median OS.31
Ipilimumab was the first modern immuno-oncology agent approved by the FDA for second-line treatment of metastatic melanoma, in 2011. A 5-year follow-up study1 was published in 2015 and demonstrated sustained durable survival. The Kaplan-Meier survival curve for OS in the study presents the additional survival benefit of ipilimumab vs dacarbazine and shows twice as many patients alive at 5 years compared with those who initially received dacarbazine. These results suggest a durable survival benefit with ipilimumab in second-line treatment of melanoma. At the 60-month time point, 40 of 250 patients (16.0%) in the test group were alive, a rate more than 100% higher than the 20 of 252 patients (7.9%) surviving in the control group—a rate much greater than the ASCO framework threshold of 50%. However, with 5 years of follow-up, this drug would not gain bonus points in the ASCO framework because less than 20% of the patients in the control group survived.
A major limitation of this study is its dependence on therapeutic outcomes as reported in clinical studies and FDA approvals. The selective population participating in these trials might lead to survival results that differ from outcomes in the general population. However, we are dealing with very new treatments, and the relatively short follow-up period limits the available data.
A possible refinement of the ASCO framework might remove or relax the required threshold for survival with standard care and focus on the difference between the 2 treatments. If we use a survival threshold of 7.5%, for instance, at 2 times the median end point, then 7 indications would receive durable survival bonus points (eTable 4 in the Supplement) as opposed to the 3 indications that currently meet the required threshold of 20% survival in the control group. These indications include ipilimumab for second-line treatment of melanoma, which demonstrated a durable survival of 16% in the 5-year follow-up study. However, this approach risks rewarding treatments in which only a small number of patients survived in the control group. There is a delicate balance between the need for statistical significance and the need to identify treatments with the potential for durable survival. If, for instance, a minimum 15% survival rate in the control group is required, only 1 additional medication (pembrolizumab for first-line treatment of melanoma) would be awarded points for durable survival.
Our ability to assess the value of a medical intervention is crucial in a global health system in need of strategies for improving outcomes with the resources available. Modern immuno-oncology agents have generated great excitement because of their potential to provide durable survival for some patients. However, if the frameworks are to be used to make coverage decisions, they will require additional refinement.32 Furthermore, additional long-term follow-up data for immuno-oncology therapies are needed to understand the magnitude of clinical benefit provided.
Corresponding Author: Daniel A. Goldstein, MD, Davidoff Cancer Center, Rabin Medical Center, 39 Jabotinski St, Petah Tikva, Israel (firstname.lastname@example.org).
Accepted for Publication: October 9, 2017.
Published Online: December 28, 2017. doi:10.1001/jamaoncol.2017.4445
Author Contributions: Mr Ben-Aharon and Dr Goldstein had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Ben-Aharon, Magnezi, Goldstein.
Acquisition, analysis, or interpretation of data: Ben-Aharon, Leshno, Goldstein.
Drafting of the manuscript: Ben-Aharon, Magnezi.
Critical revision of the manuscript for important intellectual content: Ben-Aharon, Leshno, Goldstein.
Statistical analysis: Ben-Aharon, Leshno.
Study supervision: Magnezi, Leshno, Goldstein.
Conflict of Interest Disclosures: None reported.
Create a personal account or sign in to: