Evaluation of Information Theoretic Network Meta-analysis to Rank First-Line Anticancer Regimens for Hormone Receptor–Positive, ERBB2-Negative Metastatic Breast Cancer

Key Points

Question  Can information theoretic network meta-analysis (IT-NMA) rank the estimated efficacy of regimens for hormone receptor–positive, ERBB2-negative metastatic breast cancer (HR-positive, ERBB2-negative MBC)?

Findings  In this network meta-analysis study, a combination of targeted and endocrine therapy, ie, letrozole and palbociclib, had the highest ranking. Rarely used regimens’ rank scores gravitated to indeterminacy, while monotherapies that compared unfavorably with novel agents or combinations in recent trials, such as anastrozole, had low rankings.

Meaning  In this study, combination therapies were ranked more highly than monotherapies for treating HR-positive, ERBB2-negative MBC.

Importance  Hormone receptor–positive, ERBB2 (formerly HER2/neu)-negative metastatic breast cancer (HR-positive, ERBB2-negative MBC) is treated with targeted therapy, endocrine therapy, chemotherapy, or combinations of these modalities; however, evaluating the increasing number of treatment options is challenging because few regimens have been directly compared in randomized clinical trials (RCTs), and evidence has evolved over decades. Information theoretic network meta-analysis (IT-NMA) is a graph theory–based approach for regimen ranking that takes effect sizes and temporality of evidence into account.

Objective  To examine the performance of an IT-NMA approach to rank HR-positive, ERBB2-negative MBC treatment regimens.

Data Sources  HemOnc.org, a freely available medical online resource of interventions, regimens, and general information relevant to the fields of hematology and oncology, was used to identify relevant RCTs.

Study Selection  All primary and subsequent reports of RCTs of first-line systemic treatments for HR-positive, ERBB2-negative MBC that were referenced on HemOnc.org and published between 1974 and 2019 were included. Additional RCTs that were evaluated by a prior traditional network meta-analysis on HR-positive, ERBB2-negative MBC were also included.

Data Extraction and Synthesis  RCTs were independently extracted from HemOnc.org and a traditional NMA by separate observers. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline for NMA with several exceptions: the risk of bias within individual studies and inconsistency in the treatment network were not assessed.

Main Outcomes and Measures  Regimen rankings generated by IT-NMA based on clinical trial variables, including primary end point, enrollment number per trial arm, P value, effect size, years of enrollment, and year of publication.

Results  A total of 203 RCTs with 63 629 patients encompassing 252 distinct regimens were compared by IT-NMA, which resulted in 151 rankings as of 2019. Combinations of targeted and endocrine therapy were highly ranked, especially the combination of endocrine therapy with cyclin-dependent kinase 4 and 6 (CDK4/6) inhibitors. For example, letrozole plus palbociclib was ranked first and letrozole plus ribociclib, third. Older monotherapies that continue to be used in RCTs in comparator groups, such as anastrozole (251 of 252) and letrozole (252), fell to the bottom of the rankings. Many regimens gravitated toward indeterminacy by 2019.

Conclusions and Relevance  In this network meta-analysis study, combination therapies appeared to be associated with better outcomes than monotherapies in the treatment of HR-positive, ERBB2-negative MBC. These findings suggest that IT-NMA is a promising method for longitudinal ranking of anticancer regimens from RCTs with different end points, sparse interconnectivity, and decades-long timeframes.

Cancer treatment has become increasingly sophisticated. This increased complexity of cancer treatment has contributed to a 31% decrease in the overall cancer death rate from 1991 to 2018.¹ However, keeping current with the increasing number of treatment options is challenging for health care practitioners, researchers, policy makers, and guideline developers.

Given the large number of cancer treatment regimens available, prospective head-to-head comparisons of all regimens are infeasible. Expert-driven clinical guidelines, such as those from the National Comprehensive Cancer Network (NCCN), often serve as the reference standard for regimen selection. But a potential limitation is that recommendations may be informed by opinions and clinical experiences of the guideline development group.² Meta-analysis integrates findings from multiple trials and can considerably increase statistical power and resolve disparate findings among studies.³ Network meta-analysis (NMA) is a ranking technique that combines direct and indirect evidence within a network of clinical trials.⁴ Traditional NMA (tNMA) requires that the compared trials have the same clinical end point and does not account for time-varying trends, such as changes in prevalence, underlying biologic factors, obsolescence of certain treatment approaches, and evolving supportive care measures. We previously developed an approach to address some of the shortcomings of tNMA using information theoretic NMA (IT-NMA).^5,6 Information theory is a broad array of techniques that provide quantitative measures of information content.⁷ We adapted concepts of information theory to model networks of randomized clinical trials (RCTs) as a distributed power grid where the “charge” of a regimen corresponds directly to its ranking, and the “impedance” of the “transmission line” between regimens corresponds to both the recency of the evidence and the surrogacy of the comparison. Regimens with comparatively superior efficacy gain charge at the expense of those with comparatively inferior efficacy, and these changes are propagated through the network; this propagation is made more difficult for older regimens to account for information decay over time. As an analogy to power distribution networks, older transmission lines accumulate impurities over time that decrease their conductivity and increase their resistance.⁸

Currently, breast cancer has the highest incidence and the second highest cancer mortality among US women.⁹ Hormone-receptor–positive, ERBB2 (formerly HER2/neu)-negative metastatic breast cancer (HR-positive, ERBB2-negative MBC) is the most common subtype of metastatic breast cancer and is considered to be incurable.¹⁰ HR-positive, ERBB2-negative MBC has been studied through carefully designed RCTs of cytotoxic chemotherapy, endocrine therapy, targeted therapy, and their combinations since the 1970s. Because of a very large number of treatment options, HR-positive, ERBB2-negative MBC is a good candidate for testing IT-NMA–based regimen ranking. We therefore used IT-NMA to conduct the largest analysis of first-line HR-positive/ERBB2-negative MBC trials to date, to our knowledge.

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline for NMA with several exceptions: the risk of bias within individual studies and inconsistency in the treatment network were not assessed. We evaluated all RCTs of first-line systemic treatments for HR-positive, ERBB2-negative MBC that were referenced on HemOnc.org, an expert-curated website that undergoes continuous review for standard-of-care systemic anticancer therapy,¹¹ or included in a prior tNMA.¹² To include older RCTs that did not specify breast cancer subtypes, we included all RCTs that were not specifically for ERBB2-positive or triple-negative MBC. HemOnc.org aims to include all phase 3 RCTs through a standardized approach that has been previously described¹³; it also includes randomized phase 2 RCTs with regimens that are considered to be standard of care and/or have been the basis of regulatory approval by the US Food and Drug Administration (FDA). All breast cancer pages of HemOnc.org were hand searched; the related HemOnc ontology was also used to automatically screen out ineligible studies.¹⁴ Eligible reports were reviewed independently by 2 authors (X.L. and J.L.W.), and disagreements were adjudicated through discussion to consensus.

The IT-NMA algorithm has been previously described and is inspired by power grid network analysis^6,15; briefly, it ranks regimens taking several elements of trials into account, including the P value and effect size for the least surrogate outcome with P ≤ .10, number of patients enrolled in each group, and an aging coefficient to account for the salience of a particular regimen. We used reported P values when available; for studies that reported only confidence intervals, we calculated P values following the method of Altman and Bland.¹⁶ For effect sizes, we used hazard ratios for time-based end points, such as overall survival (OS) and progression-free survival (PFS), and odds ratios (ORs) for fixed end points, such as response rates; if the hazard ratio was not available, as was the case for many older studies, we substituted the ratio of median survival times. The aging coefficient was an exponential decay coefficient with a half-life of 5.5 years, which was set to a value of 1 at the end of trial enrollment; subsequent publications and value propagation events refreshed the coefficient by 1 half-life (to a maximum value of 1). Examples of network value propagation are shown in eFigure 1 in the Supplement. The overall score for a given regimen at any given year was the summation of values from all trials of that regimen up to that date. This unitless score can be interpreted as follows: (1) regimens with positive scores are considered recommendable; (2) regimens with negative scores are considered not recommendable; and (3) scores near zero are considered to be indeterminate. Indeterminacy can be the result of the passage of time since a regimen has been evaluated (obsolescence), the failure of a regimen to demonstrate statistical superiority (a negative trial), and/or superior results for a regimen being counteracted by inferior results for the regimen. In addition to the overall rank score, the aging coefficient is reported for each regimen and is incorporated into network visualizations using the alpha (transparency) channel. For longitudinal regimen ranking, the algorithm generated a network and rank list for each year since the year of publication for the oldest RCTs in the network. The code to generate rankings is freely available on GitHub.¹⁷ The analysis was conducted in R version 4.1.1 (R Project for Statistical Computing) and also used R packages igraph version 1.2.8 and RColorBrewer version 1.1-2.

Beginning with a total of 6591 records (6434 from HemOnc.org and 157 from a prior tNMA), a total of 203 RCTs that enrolled 63 629 patients from 1972 onwards that were published between 1974 and 2019 were identified and included in this IT-NMA (Figure 1). The RCTs included 134 phase 3 trials, 27 randomized phase 2 trials, and 42 studies that did not specify phase; 172 were identified from HemOnc.org and 31 were from the prior tNMA.¹² Overall, 24 trials (77%) not present on HemOnc.org were randomized phase 2 trials. Information about included trials, including reference, study type, population, regimens, years of enrollment and publication, and study end point are provided in eTable 1 in the Supplement. Considering variations in dose, frequency, and number of cycles, there were 252 unique regimens, including 149 cytotoxic chemotherapies, 37 endocrine therapies, 15 combination endocrine and targeted therapies, 31 combination cytotoxic chemotherapy and targeted therapies, 17 combination chemotherapy and endocrine therapies, and 3 other therapies, including observation.

The median (IQR) number of patients per RCT was 236 (148-414), and the median (IQR) period of enrollment was 3 (2-4) years (Figure 2). Intervals between the end of enrollment and first publication varied; although the median (IQR) was 4 (3-5) years, some trials did not publish their results until many years later.

Figure 3 shows the network of regimens for the year 2019. After accounting for ties, there were 151 regimen ranks in 2019. The Table includes the 54 regimens with an aging coefficient of 0.5 or greater; the full list of ranked regimens and their modalities is provided in eTable 2 in the Supplement. Four of the top 5 regimens were combinations of endocrine therapy and cyclin-dependent kinase 4 and 6 inhibitors (CDK4/6i). For example, letrozole plus palbociclib was ranked first and letrozole plus ribociclib, third. Most of the lowest-ranked regimens were monotherapies, such as letrozole (last of 252) and anastrozole (251); the lowest-ranked combination regimens with aging coefficients of 0.5 or greater were vinorelbine plus capecitabine and weekly ixabepilone plus bevacizumab. eFigure 2 in the Supplement illustrates the distribution of rank scores vs aging coefficients. Many regimens gravitated toward indeterminacy by 2019.

Figure 4 shows snapshots of the regimen networks in the years of 1974, 1994, 2007, and 2018. Since the publication of the first RCTs on treatments for MBC in 1974,¹⁸ many RCTs and regimens have been added to the network, and clusters began to emerge. When a novel treatment regimen was tested in escalation trials (ie, higher dose intensity and/or additional drugs in the experimental group), it tended to rise to the top of the rank list. For example, the combination regimen of paclitaxel and bevacizumab came to near the top of the rankings in 2007, which coincides with the publication of the ECOG E2100 trial.¹⁹ This trial was the basis of accelerated approval of bevacizumab for metastatic breast cancer in 2008. We visualized the changing landscape of the regimen network from 1974 to 2019 (a span of 45 years) in videos, links to which are provided in the eAppendix in the Supplement.

We evaluated 252 unique treatment regimens for HR-positive, ERBB2-negative MBC from 203 RCTs by IT-NMA. This is by far the largest such comparison that we are aware of; the aforementioned largest published tNMA evaluated 149 regimens. Results from the current study of RCTs through 2019 showed that combinations of targeted and endocrine therapy ranked highest, and monotherapies, such as letrozole, ranked lowest.

Current NCCN guidelines recommend the combination of an aromatase inhibitor or selective ER down-regulator and a CDK4/6i as first-line therapy for HR-positive, ERBB2-negative MBC as category 1, meaning that based on high-level evidence, there is uniform NCCN consensus that the intervention is appropriate.²⁰ Our results agree with this recommendation. Endocrine monotherapies, such as letrozole, are listed as first-line other recommended regimens by NCCN, although the recommendation is category 2A, meaning that it is based on lower-level evidence, but there is still uniform NCCN consensus that the intervention is appropriate.²⁰ These regimens were at the bottom of our ranked list. A possible explanation is that because the older endocrine monotherapies are still recognized as a treatment option, clinical trials continue to use them in control groups. As an example, single-agent letrozole was used in the control group in 10 RCTs evaluated; all experimental groups in these trials were 2 drug combinations. In 9 of these RCTs, the control group was comparatively inferior. As previously reported, our algorithm can partially address but not completely overcome such “straw man” effects in regimen networks.⁶ Future work will explore the incorporation of scores that account for toxic effects, such as the ESMO Magnitude of Clinical Benefit Scale,²¹ which may offset the tendency to reward escalation trial designs, which usually trade efficacy for toxic effects.

Treatment for HR-positive, ERBB2-negative MBC has evolved. Since the late 1970s, endocrine therapy regimens, such as anastrozole, exemestane, fulvestrant, goserelin, letrozole, tamoxifen, and toremifene, have been investigated for the treatment of HR-positive breast cancers.^22-28 In more recent years, targeted therapies such as palbociclib, ribociclib, and abemaciclib, which target the cyclin-D–CDK4/6–Rb pathway, have been tested. These have been shown to act synergistically with antiestrogens to inhibit the growth of HR-positive breast cancer.²⁹ The combination of targeted and endocrine therapies has been shown to be efficacious and has led to improvements in both response rates and survival.^30-34 On the other hand, chemotherapy has remained a mainstay for the treatment of MBC. But the addition of targeted therapies to cytotoxic chemotherapies has yielded equivocal results. The addition of bevacizumab to paclitaxel led to improvement in PFS but not OS in ECOG E2100, and bevacizumab’s FDA indication for breast cancer was subsequently withdrawn; however, the European Medicines Agency indication remains.^19,35 Despite this, the regimen was highly ranked by our analysis because of positive PFS results in 2 subsequently published phase 3 RCTs: CALGB 40502 and MERiDiAN.^36,37 Notably, similar findings to ours were reported in a traditional NMA by Giuliano et al.¹² These authors theorized that the finding may be in part because of the inclusion of patients with triple-negative breast cancer in some trials³⁸; this possibility cannot be disambiguated from methodologic issues in the absence of individual patient-level analysis. Despite the common use of surrogate end points in oncology RCTs, the bevacizumab finding is discrepant with prevailing clinical opinion. Although this highlights issues with a clinical trial literature that largely relies on surrogate end points, it also suggests that further methodological refinements are needed in the IT-NMA approach.

Conversely, IT-NMA enables ranking of chemotherapy and endocrine therapy regimens together, even though they are rarely directly compared (Figure 3B). The network distance between these distinct treatment approaches can be vast, suggesting that probabilistic methods, such as tNMA, may be less reliable or even impossible to execute if the network is not completely interconnected.

Many regimens have been discarded over time, with rank scores gravitating toward zero due to the aging coefficient in our IT-NMA algorithm.⁶ Primarily because of this obsolescence, more than half of the regimens in the rank list were indeterminate, with a rank score near zero. Given that these regimens have no contemporary comparisons, it cannot be concluded that they are any better or worse than currently studied regimens; in fact, some could even be candidates for repurposing in the future. However, there are some older monotherapy regimens that continue to be used in control groups in more recent clinical trials. Because these older regimens performed unfavorably in trials, they sank to the bottom of our rank list, meaning that they remained salient. For example, tamoxifen at 20 mg/d was ranked 220 of 252. This regimen was first tested in 1977,²³ and the latest trial with this regimen in the control group was published in 2018—a span of more than 4 decades.³³ Our finding provides evidence supporting the concern that suboptimal regimens in control groups may be used in many RCTs. The use of a substandard comparator regimen may result in a trial that is more likely to be positive but actually prevents the trial from addressing the clinically relevant question of whether a new drug is better than the current standard of care.³⁹

Our regimen ranking results are in accordance with findings from Giuliano et al¹²; however, that study included both first-line and second-line studies, whereas the current study was restricted to the first-line setting only, potentially increasing the clinical relevance of our findings. Nevertheless, it must be acknowledged that substantial heterogeneity remains in the characteristics of the patients studied, specifically regarding prior treatment exposure. De novo MBC is rare,^40-42 such that most patients with MBC had early-stage disease at diagnosis and either had a late recurrence or primary disease progression. In many cases, these patients will have received endocrine and/or cytotoxic chemotherapy in the adjuvant or neoadjuvant setting.

IT-NMA has several advantages when compared with other regimen ranking methods, such as those used in Giuliano et al.¹² One of the challenges associated with meta-analysis of oncology clinical trials is heterogenous outcomes. Our IT-NMA approach can construct a network containing RCTs with different primary end points, using empirically derived coefficients for different primary end points, eg, 1 for OS, and 0.8 for PFS. Other regimen ranking methods, such as the American Society of Clinical Oncology Value Framework, use comparable coefficients, although future exploration of varying coefficients for surrogate end points is warranted in light of the bevacizumab findings noted previously.⁴³ More importantly, IT-NMA allows longitudinal regimen ranking. For HR-positive, ERBB2-negative MBC, our IT-NMA produced a rank list of regimens for each year from 1974 to 2019, which enabled us to examine how the field has evolved over time. Given the successful ranking of regimens for HR-positive, ERBB2-negative MBC, we plan to use IT-NMA to rank regimens for other common and extensively studied cancers.

This study has limitations. One of the challenges of this study was visualizing the regimen network. To date, most published NMAs have used circular layouts; however, this layout can become cluttered and less informative with increasing numbers of nodes and edges. To illustrate this, we have reconstructed the circular network as reported by Giuliano et al¹² using a force-directed layout (eFigure 3 in the Supplement). The regimen network in this reanalysis contained 273 nodes and 237 edges and was visualized with the Fruchterman-Rheingold force-based layout.⁴⁴ As this layout shows, chemotherapy- and endocrine therapy–based regimens are clearly segregated, with just a tenuous connection between the 2 dominant components by way of 2 rarely used and relatively understudied regimens (mitoxantrone monotherapy and everolimus plus exemestane). We believe that, compared with a circular layout, a force-based layout can lead to a greater understanding of the complexity of a high-dimensional space of indirect comparisons. Next steps include building interactive approaches for the visualization of regimen networks.

Furthermore, because the chemotherapy and endocrine therapy subnetworks are essentially disconnected, direct comparisons of chemotherapy and endocrine therapy–based regimens should be undertaken with caution. We did not incorporate toxic effects into our IT-NMA algorithm, such that our rankings are primarily driven by efficacy. As mentioned previously, value-based frameworks such as the ESMO Magnitude of Clinical Benefit Scale²¹ can present a more balanced picture between efficacy and toxic effects and will be incorporated into future iterations. Although we attempted to be exhaustive in our search for RCTs, it is likely that some trials were missed, especially non–English language trials and older trials that may be less discoverable using current literature searching techniques. Additionally, we concede that the published literature is biased toward trials with positive results, exacerbating the tendency to overvalue experimental groups and undervalue control groups. Mandatory clinical trial registration may alleviate this tendency, although adherence to registration has become uniform only in the last decade. We also did not evaluate several issues with clinical trial design and implementation that could skew the outcomes, such as the appropriateness of the control regimen or crossover designs.⁴⁵

In this study, combination therapies were more highly ranked than endocrine therapy or chemotherapy alone for treating HR-positive, ERBB2-negative MBC, with combination CDK4/6i and endocrine therapy at the top of our ranked list. Our findings demonstrate that IT-NMA is a promising method for direct and indirect comparisons to rank cancer treatment regimens and their evolution over time. Although these results are best considered preliminary, we anticipate that with continued refinement the IT-NMA approach will eventually demonstrate clinical utility.

Accepted for Publication: February 2, 2022.

Published: April 13, 2022. doi:10.1001/jamanetworkopen.2022.4361

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Li X et al. JAMA Network Open.

Corresponding Author: Jeremy L. Warner, MD, MS, Departments of Medicine (Division of Hematology/Oncology) and Biomedical Informatics, Vanderbilt University, 2220 Pierce Ave, Preston Research Building 777, Nashville, TN 37232 (jeremy.warner@vumc.org).

Author Contributions: Drs Li and Warner had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Li, Tavana, Gyawali, Warner.

Acquisition, analysis, or interpretation of data: Li, Beeghly-Fadiel, Bhavnani, Rubinstein, Gyawali, Riaz, Fernandes, Warner.

Drafting of the manuscript: Li, Beeghly-Fadiel, Rubinstein, Warner.

Critical revision of the manuscript for important intellectual content: Li, Beeghly-Fadiel, Bhavnani, Tavana, Gyawali, Riaz, Fernandes, Warner.

Statistical analysis: Li, Bhavnani, Warner.

Obtained funding: Bhavnani, Tavana, Warner.

Administrative, technical, or material support: Bhavnani, Warner.

Supervision: Beeghly-Fadiel, Bhavnani, Fernandes, Warner.

Conflict of Interest Disclosures: Dr Li reported being an intern for HemOnc.org from January 2020 to June 2021. Dr Beeghly-Fadiel reported receiving grants the National Institutes of Health outside the submitted work. Dr Rubinstein reported receiving personal fees for consulting and advisory roles from Eusa Pharma, Janssen Pharmaceuticals, Roche, and Sanofi outside the submitted work. Dr Gyawali reported serving as associate editor for HemOnc.org and receiving personal fees for consulting from Vivio Healthcare. Dr Fernandes reported being a section editor for HemOnc.org outside the submitted work. Dr Warner reported grants from the National Institutes of Health/National Cancer Institute during the conduct of the study; receiving funding from the American Association for Cancer Research; receiving personal fees from Westat, Roche, IBM Watson Health, Melax Tech, and Flatiron Health outside the submitted work; and being deputy editor and cofounder of HemOnc.org, LLC, outside the submitted work. No other disclosures were reported.

Funding/Support: This study was funded by grants U01 CA231840 (Drs Warner and Bhavnani) and U54 MD010722 (Dr Beeghly-Fadiel) from the National Cancer Institute of the National Institutes of Health, the Ann Melly Summer Scholarship (Dr Li), grant U54 CA163072 (Dr Beeghly-Fadiel) from the National Institute on Minority Health and Health Disparities of the National Institutes of Health, and the Ontario Institute of Cancer Research (Dr Gyawali).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We would like to thank Peter C. Yang, MD, founder and editor-in-chief of HemOnc.org, for his ongoing support of the HemOnc.org website; he did not receive compensation related to this work. We would also like to thank all of the contributors and editors of HemOnc.org, who are uncompensated.

Additional Information: The code to create the networks is freely available for download on GitHub.