Figure 1.  Posterior Distributions and Probabilities Assuming Weakly Informative Priors

Results of the meta-analysis assuming weakly informative priors: posterior distributions and probabilities of each subgroup. An odds ratio (OR) lower than 1 indicates reduced mortality owing to tocilizumab in comparison with control treatment. A, Posterior distributions, in which point estimates (black solid-circle data markers) depict the median and interval bars represent the 95% credible (highest-density) intervals (CrIs). B, Cumulative posterior probabilities, which correspond to the probabilities that the OR is lower than or equal to the effect size on the x-axis (X). C through E, Posterior distributions in the risk difference scale (control minus tocilizumab risk) across plausible ranges of mortality risk under control treatment for each subgroup (solid lines represent the median, and shaded areas represent the 95% CrIs). Vertical dashed lines represent the assumed mean risk under control treatment in each subgroup (eMethods in the Supplement). A risk difference greater than 0 indicates reduced mortality owing to tocilizumab in comparison with control treatment. Underlying weakly informative priors are normal, mean (SD) of 0 (0.82) for coefficients and half-normal of 0.5 for the between-study standard deviation.

Figure 2.  Posterior Predictive Distributions and Probabilities Assuming Weakly Informative Priors

Posterior predictive distributions and probabilities of each subgroup. A, Posterior predictive distributions, in which point estimates (black solid-circle data markers) depict the median, and interval bars represent the 95% credible (highest-density) intervals (CrIs). B, Cumulative posterior probabilities, which correspond to the probabilities that future studies (specific to each subgroup) will find a point estimate lower than or equal to the effect size on the x-axis (X). Underlying weakly informative priors are normal, mean (SD) of 0 (0.82) for coefficients and half-normal of 0.5 for the between-study standard deviation. OR indicates odds ratio.

Figure 3.  Posterior Distributions and Probabilities From Normal Conjugate Analyses on the Invasive Mechanical Ventilation Subgroup

Results for the invasive mechanical ventilation subgroup from meta-analyses using an informative prior based on simulated randomized clinical trials (eTable 5 in the Supplement). “(Current) 717” depicts the results previously shown in Figure 1A and B for this subgroup. A, Posterior distributions for each separate meta-analysis. The y-axis depicts the number of total patients receiving invasive mechanical ventilation included in each respective model (current plus simulated patients). Point estimates (black solid-circle data markers) depict the median, and interval bars represent the 95% credible (highest-density) intervals (CrIs). The vertical dashed line represents a 0.77 odds ratio (OR; World Health Organization Rapid Evidence Appraisal for COVID-19 Therapies Working Group mean result for tocilizumab- and corticosteroid-treated patients).1 This value is the mean OR underlying the generated randomized clinical trials (eTable 5 in the Supplement). B, Posterior probability of benefit for different thresholds (OR <1.0 and <0.9). Underlying weakly informative priors are normal, mean (SD) of 0 (1.5) for coefficients and half-normal of 0.5 for the between-study standard deviation.

Table 1.  Included Studies
Table 2.  Posterior Distributions and Probabilities Assuming Distinct Prior Beliefs
1.
Shankar-Hari  M﻿, Vale  CL﻿, Godolphin  PJ﻿,  et al; WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group.  Association between administration of IL-6 antagonists and mortality among patients hospitalized for COVID-19: a meta-analysis. ﻿  JAMA. 2021;326(6):499-518. doi:10.1001/jama.2021.11330﻿ PubMedGoogle Scholar
2.
Therapeutics and COVID-19: living guideline. World Health Organization. Accessed September 2, 2021. https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2021.2
3.
Wijeysundera  DN﻿, Austin  PC﻿, Hux  JE﻿, Beattie  WS﻿, Laupacis  A﻿.  Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials. ﻿  J Clin Epidemiol. 2009;62(1):13-21.e5. doi:10.1016/j.jclinepi.2008.07.006﻿ PubMedGoogle ScholarCrossref
4.
Goligher  EC﻿, Tomlinson  G﻿, Hajage  D﻿,  et al.  Extracorporeal membrane oxygenation for severe acute respiratory distress syndrome and posterior probability of mortality benefit in a post hoc bayesian analysis of a randomized clinical trial. ﻿  JAMA. 2018;320(21):2251-2259. doi:10.1001/jama.2018.14276﻿ PubMedGoogle ScholarCrossref
5.
Brophy  JM﻿.  Bayesian interpretation of the EXCEL trial and other randomized clinical trials of left main coronary artery revascularization. ﻿  JAMA Intern Med. 2020;180(7):986-992. doi:10.1001/jamainternmed.2020.1647﻿ PubMedGoogle ScholarCrossref
6.
Spiegelhalter  DJ﻿, Abrams  KR﻿, Myles  JP﻿.  Bayesian Approaches to Clinical Trials and Health Care Evaluation. Wiley; 2004.
7.
Greenland  S﻿.  Invited commentary: the need for cognitive science in methodology. ﻿  Am J Epidemiol. 2017;186(6):639-645. doi:10.1093/aje/kwx259﻿ PubMedGoogle ScholarCrossref
8.
Page  MJ﻿, McKenzie  JE﻿, Bossuyt  PM﻿,  et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. ﻿  BMJ. 2021;372(n71):n71. doi:10.1136/bmj.n71﻿ PubMedGoogle Scholar
9.
Hedges  LV﻿, Vevea  JL﻿.  Fixed- and random-effects models in meta-analysis. ﻿  Psychol Methods. 1998;3(4):486-504. doi:10.1037/1082-989X.3.4.486﻿ Google ScholarCrossref
10.
Viechtbauer  W﻿.  Conducting meta-analyses in R with the metafor package. ﻿  J Stat Softw. 2010;36(1):1-48. doi:10.18637/jss.v036.i03PubMedGoogle Scholar
11.
van de Schoot  R﻿, Veen  D﻿, Smeets  L﻿, Winter  SD﻿, Depaoli  S﻿. A tutorial on using the WAMBS checklist to avoid the misuse of bayesian statistics. In: van de Schoot  R﻿, Miočević  M﻿, eds.  Small Sample Size Solutions. 1st ed. Routledge; 2020:30-49. doi:10.4324/9780429273872-4﻿
12.
Röver  C﻿, Bender  R﻿, Dias  S﻿,  et al.  On weakly informative prior distributions for the heterogeneity parameter in bayesian random-effects meta-analysis. ﻿  Res Synth Methods. 2021;12(4):448-474. doi:10.1002/jrsm.1475﻿ PubMedGoogle ScholarCrossref
13.
Turner  RM﻿, Jackson  D﻿, Wei  Y﻿, Thompson  SG﻿, Higgins  JPT﻿.  Predictive distributions for between-study heterogeneity and simple methods for their application in bayesian meta-analysis. ﻿  Stat Med. 2015;34(6):984-998. doi:10.1002/sim.6381﻿ PubMedGoogle ScholarCrossref
14.
McElreath  R﻿.  Statistical Rethinking: A Bayesian Course With Examples in R and Stan. Chapman & Hall/CRC; 2020.
15.
Doi  SA﻿, Furuya-Kanamori  L﻿, Xu  C﻿, Lin  L﻿, Chivese  T﻿, Thalib  L﻿.  Questionable utility of the relative risk in clinical research: a call for change to practice. ﻿  J Clin Epidemiol. Published online November 7, 2020. doi:10.1016/j.jclinepi.2020.08.019PubMedGoogle Scholar
16.
Doi  SA﻿, Furuya-Kanamori  L﻿, Xu  C﻿,  et al.  The odds ratio is “portable” but not the relative risk: time to do away with the log link in binomial regression. ﻿  J Clin Epidemiol. Published online August 8, 2021. doi:10.1016/j.jclinepi.2021.08.003Google Scholar
17.
Welton  NJ﻿, Jones  HE﻿, Dias  S﻿. Bayesian methods for meta-analysis. In: Lesaffre E, Baio G, Boulanger B, eds.  Bayesian Methods in Pharmaceutical Research. Chapman & Hall/CRC; 2020.
18.
IntHout  J﻿, Ioannidis  JP﻿, Rovers  MM﻿, Goeman  JJ﻿.  Plea for routinely presenting prediction intervals in meta-analysis. ﻿  BMJ Open. 2016;6(7):e010247. doi:10.1136/bmjopen-2015-010247﻿ PubMedGoogle Scholar
19.
Higgins  JPT﻿, Thompson  SG﻿, Spiegelhalter  DJ﻿.  A re-evaluation of random-effects meta-analysis. ﻿  J R Stat Soc Ser A Stat Soc. 2009;172(1):137-159. doi:10.1111/j.1467-985X.2008.00552.x﻿ PubMedGoogle ScholarCrossref
20.
Kalil  AC﻿, Sun  J﻿.  How many patients with severe sepsis are needed to confirm the efficacy of drotrecogin alfa activated? a bayesian design. ﻿  Intensive Care Med. 2008;34(10):1804-1811. doi:10.1007/s00134-008-1159-8﻿ PubMedGoogle ScholarCrossref
21.
Carpenter  B﻿, Gelman  A﻿, Hoffman  MD﻿,  et al.  Stan: a probabilistic programming language. ﻿  J Stat Softw. 2017;76(1):1-32. doi:10.18637/jss.v076.i01﻿ Google ScholarCrossref
22.
Bürkner  PC﻿.  brms: an R package for bayesian multilevel models using stan. ﻿  J Stat Softw. 2017;80(1):1-28. doi:10.18637/jss.v080.i01﻿ Google ScholarCrossref
23.
Depaoli  S﻿, van de Schoot  R﻿.  Improving transparency and replication in bayesian statistics: the WAMBS-checklist. ﻿  Psychol Methods. 2017;22(2):240-261. doi:10.1037/met0000065﻿ PubMedGoogle ScholarCrossref
24.
van de Schoot  R﻿, Depaoli  S﻿, King  R﻿,  et al.  Bayesian statistics and modelling. ﻿  Nat Rev Methods Primer. 2021;1(1):1-26. doi:10.1038/s43586-020-00001-2﻿ Google ScholarCrossref
25.
Stone  JH﻿, Frigault  MJ﻿, Serling-Boyd  NJ﻿,  et al; BACC Bay Tocilizumab Trial Investigators.  Efficacy of tocilizumab in patients hospitalized with COVID-19. ﻿  N Engl J Med. 2020;383(24):2333-2344. doi:10.1056/NEJMoa2028836﻿ PubMedGoogle ScholarCrossref
26.
Hermine  O﻿, Mariette  X﻿, Tharaux  PL﻿, Resche-Rigon  M﻿, Porcher  R﻿, Ravaud  P﻿; CORIMUNO-19 Collaborative Group.  Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: a randomized clinical trial. ﻿  JAMA Intern Med. 2021;181(1):32-40. doi:10.1001/jamainternmed.2020.6820﻿ PubMedGoogle ScholarCrossref
27.
Rosas  IO﻿, Bräu  N﻿, Waters  M﻿,  et al.  Tocilizumab in hospitalized patients with severe COVID-19 pneumonia. ﻿  N Engl J Med. 2021;384(16):1503-1516. doi:10.1056/NEJMoa2028700﻿ PubMedGoogle ScholarCrossref
28.
Salama  C﻿, Han  J﻿, Yau  L﻿,  et al.  Tocilizumab in patients hospitalized with COVID-19 pneumonia. ﻿  N Engl J Med. 2021;384(1):20-30. doi:10.1056/NEJMoa2030340﻿ PubMedGoogle ScholarCrossref
29.
Rosas  IO﻿, Diaz  G﻿, Gottlieb  RL﻿,  et al.  Tocilizumab and remdesivir in hospitalized patients with severe COVID-19 pneumonia: a randomized clinical trial. ﻿  Intensive Care Med. 2021;47(11):1258-1270. doi:10.1007/s00134-021-06507-x﻿ PubMedGoogle ScholarCrossref
30.
Veiga  VC﻿, Prats  JAGG﻿, Farias  DLC﻿,  et al; Coalition COVID-19 Brazil VI Investigators.  Effect of tocilizumab on clinical outcomes at 15 days in patients with severe or critical coronavirus disease 2019: randomised controlled trial. ﻿  BMJ. 2021;372(n84):n84. doi:10.1136/bmj.n84﻿ PubMedGoogle Scholar
31.
Abani  O﻿, Abbas  A﻿, Abbas  F﻿,  et al; RECOVERY Collaborative Group.  Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. ﻿  Lancet. 2021;397(10285):1637-1645. doi:10.1016/S0140-6736(21)00676-0﻿ PubMedGoogle ScholarCrossref
32.
Gordon  AC﻿, Mouncey  PR﻿, Al-Beidh  F﻿,  et al; REMAP-CAP Investigators.  Interleukin-6 receptor antagonists in critically ill patients with COVID-19. ﻿  N Engl J Med. 2021;384(16):1491-1502. doi:10.1056/NEJMoa2100433﻿ PubMedGoogle Scholar
33.
Kent  DM﻿, Steyerberg  E﻿, van Klaveren  D﻿.  Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. ﻿  BMJ. 2018;363:k4245. doi:10.1136/bmj.k4245﻿ PubMedGoogle Scholar
34.
Albuquerque  AM﻿, Tramujas  L﻿, Sewanan  LR﻿, Brophy  JM﻿.  Tocilizumab in COVID-19—a bayesian reanalysis of RECOVERY. ﻿  medRxiv. Preprint posted online June 21, 2021. doi:10.1101/2021.06.15.21258966Google Scholar
35.
Kent  DM﻿, Paulus  JK﻿, van Klaveren  D﻿,  et al.  The Predictive Approaches to Treatment effect Heterogeneity (PATH) statement. ﻿  Ann Intern Med. 2020;172(1):35-45. doi:10.7326/M18-3667﻿ PubMedGoogle ScholarCrossref
36.
Aberegg  SK﻿.  Post hoc bayesian analyses. ﻿  JAMA. 2019;321(16):1631-1632. doi:10.1001/jama.2019.1198﻿ PubMedGoogle ScholarCrossref
37.
Fisher  DJ﻿, Carpenter  JR﻿, Morris  TP﻿, Freeman  SC﻿, Tierney  JF﻿.  Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? ﻿  BMJ. 2017;356:j573. doi:10.1136/bmj.j573﻿ PubMedGoogle Scholar
Views 9,510
• CME & MOC
• Cite This

### Citation

Albuquerque AM, Tramujas L, Sewanan LR, Williams DR, Brophy JM. Mortality Rates Among Hospitalized Patients With COVID-19 Infection Treated With Tocilizumab and Corticosteroids: A Bayesian Reanalysis of a Previous Meta-analysis. JAMA Netw Open. 2022;5(2):e220548. doi:10.1001/jamanetworkopen.2022.0548

#### Manage citations:

Original Investigation
Critical Care Medicine
February 28, 2022

# Mortality Rates Among Hospitalized Patients With COVID-19 Infection Treated With Tocilizumab and Corticosteroids: A Bayesian Reanalysis of a Previous Meta-analysis

Author Affiliations
• 1School of Medicine, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
• 2HCor Research Institute, Sao Paulo, Brazil
• 3Department of Internal Medicine, Columbia University, New York, New York
• 4Department of Psychology, University of California, Davis
• 5McGill Health University Center, Montreal, Québec, Canada
JAMA Netw Open. 2022;5(2):e220548. doi:10.1001/jamanetworkopen.2022.0548
Key Points

Question  Can bayesian methods clarify the uncertainty around tocilizumab’s association with mortality benefit in subgroups of hospitalized patients with COVID-19 receiving corticosteroids?

Findings  In this bayesian reanalysis of a previous meta-analysis of 15 randomized clinical trials comprising 5339 hospitalized patients with COVID-19 treated with tocilizumab and corticosteroids, those receiving simple oxygen only or noninvasive ventilation were associated with a clinically meaningful mortality benefit. In contrast, for those receiving invasive mechanical ventilation, an association with benefit was uncertain.

Meaning  This study’s findings indicate that further research is needed to assess the association between mortality benefit or risk in patients with COVID-19 receiving invasive mechanical ventilation and treated with tocilizumab and corticosteroids.

Abstract

Importance  A World Health Organization (WHO) meta-analysis found that tocilizumab was associated with reduced mortality in hospitalized patients with COVID-19. However, uncertainty remains concerning the magnitude of tocilizumab’s benefits and whether its association with mortality benefit is similar across respiratory subgroups.

Objective  To use bayesian methods to assess the magnitude of mortality benefit associated with tocilizumab and the differences between respiratory support subgroups in hospitalized patients with COVID-19.

Design, Setting, and Participants  A bayesian hierarchical reanalysis of the WHO meta-analysis of tocilizumab studies published in 2020 and 2021 was performed. Main results were estimated using weakly informative priors to exert little influence on the observed data. The robustness of these results was evaluated using vague and informative priors. The studies featured in the meta-analysis were randomized clinical tocilizumab trials of hospitalized patients with COVID-19. Only patients receiving corticosteroids were included.

Interventions  Usual care plus tocilizumab in comparison with usual care or placebo.

Main Outcomes and Measures  All-cause mortality at 28 days after randomization.

Results  Among the 5339 patients included in this analysis, most were men, with mean ages between 56 and 66 years. There were 2117 patients receiving simple oxygen only, 2505 receiving noninvasive ventilation (NIV), and 717 receiving invasive mechanical ventilation (IMV) in 15 studies from multiple countries and continents. Assuming weakly informative priors, the overall odds ratios (ORs) for survival were 0.70 (95% credible interval [CrI], 0.50-0.91) for patients receiving simple oxygen only, 0.81 (95% CrI, 0.63-1.03) for patients receiving NIV, and 0.89 (95% CrI, 0.61-1.22) for patients receiving IMV, respectively. The posterior probabilities of any benefit (OR <1) were notably different between patients receiving simple oxygen only (98.9%), NIV (95.5%), and IMV (75.4%). The posterior probabilities of a clinically meaningful association (absolute mortality risk difference >1%) were greater than 95% in patients receiving simple oxygen only and greater than 90% in patients receiving NIV. In contrast, the posterior probability of this clinically meaningful association was only approximately 67% in patients receiving IMV. The probabilities of tocilizumab superiority in the simple oxygen only subgroup compared with the NIV and IMV subgroups were 85% and 90%, respectively. Predictive intervals highlighted that only 72.1% of future tocilizumab IMV studies would show benefit. The conclusions did not change with different prior distributions.

Conclusions and Relevance  In this bayesian reanalysis of a previous meta-analysis of 15 studies of hospitalized patients with COVID-19 treated with tocilizumab and corticosteroids, use of simple oxygen only and NIV was associated with a probability of a clinically meaningful mortality benefit from tocilizumab. Future research should clarify whether patients receiving IMV also benefit from tocilizumab.

Introduction

Randomized clinical trials (RCTs) on tocilizumab, an interleukin 6 antagonist, have shown mixed results compared with control or usual care in hospitalized patients with COVID-19 as analyzed in 2021 by the World Health Organization (WHO) Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group.1 The authors reported that the association with lower 28-day all-cause mortality “…was more marked among patients who did not require invasive mechanical ventilation (IMV) at randomization.”1(p515) However, an exact quantification of tocilizumab’s clinical benefits and associated uncertainties in this or any of the other respiratory subgroups was not provided. Despite this absence, guidelines now suggest the use of tocilizumab in patients with either severe or critical COVID-19 independent of their respiratory status.2

Whether additional insights into this evidence base are available by the application of bayesian analytical methods is unknown.3-5 In brief, bayesian analyses can fully embrace uncertainty and directly answer the clinical question of interest: what is the probability of tocilizumab’s benefit in each subgroup?6 Moreover, one could also estimate the probability that tocilizumab’s association with a mortality benefit is different between subgroups. These features can contribute to a more thorough understanding of tocilizumab’s role, avoiding the “nullism” and “dichotomania” inherent with conventional null hypothesis significance testing.7

Therefore, we reanalyzed the WHO’s tocilizumab data using bayesian methods, with the primary goal of providing a further appreciation of the results and the reliability of specific subgroup conclusions.

Methods

This bayesian analysis of a previous meta-analysis was performed in accordance with the Preferred Items for Systematic Reviews and Meta-analyses (PRISMA) guideline.8 This study was exempt from obtaining formal institutional review board approval from the McGill University Health Center and the requirement to obtain informed patient consent because it is secondary research of publicly available data sets.

Data

We included all RCTs collected by the WHO REACT Working Group regardless of publication status or risk of bias. The primary outcome was 28-day total mortality. We added 0.5 events to treatment arms to include studies with no events in one of the arms. Two authors (A.M.A. and L.T.) independently extracted the number of deaths and total patients directly from the WHO REACT Working Group supplementary figures. Only patients receiving corticosteroids were included. In case of any discrepancy, both authors discussed and reached a consensus.

Meta-analysis

We used a random-effects meta-analysis to perform unconditional inferences beyond the included studies.9 Random-effects meta-analysis assumes a distribution of studies, allowing inferences not only on this collection of studies but also on studies that may not have been included in this work.9 Therefore, our inferences regard the mean effect of tocilizumab from this hypothetical population of studies,10 ie, not exclusively conditioned to the included studies as in a fixed-effect model.

We analyzed tocilizumab’s association with mortality as a function of baseline respiratory disease severity. We included 3 subgroups, defined by the WHO REACT Working Group as supplemental oxygen therapy (oxygen flow rate ≤15 L/min by face mask or nasal cannula) (hereafter, simple oxygen only), noninvasive ventilation (NIV; oxygen flow rate >15 L/min, high-flow nasal cannula, and continuous positive airway pressure), or patients receiving IMV or extracorporeal membrane oxygenation at baseline. We assumed the variability in study outcomes was the same in every subgroup (common between-study heterogeneity).

Bayesian Analysis

We applied the bayesian framework, which updates prior beliefs with the current data to form a posterior distribution.6 In brief, we implemented priors that cover plausible values for all parameters, assigning limited density to impossible values and thereby exerting little influence on the results (hereafter, known as weakly informative priors).11,12 To check whether this choice of priors meaningfully affected our results or our conclusions, we also fitted models using vague or informative priors, defined as approximately uniform or centered at the null value with small dispersion, respectively.13 Full details about our model and priors can be found in the eMethods of the Supplement.

Bayesian random-effects meta-analysis yields marginal posterior distributions for the treatment associations and between-study heterogeneity. We used medians and 95% highest-density intervals (hereafter, credible intervals [CrIs]) to describe these distributions, defined as the narrowest interval containing 95% of the probability density function.14 Our primary estimands are expressed as odds ratios (ORs) and focused on the calculation of posterior probabilities of any benefit (OR <1) and of meaningful clinical association (arbitrarily defined as OR <0.9, ie, at least a 10% odds reduction in outcomes). We also derived the risk difference from the OR across multiple plausible control mortality risks, further explained in the eMethods in the Supplement.15,16 In this case, we defined the association with meaningful clinical benefit as a 1% risk difference.

To compare the marginal posterior distributions of each respiratory support subgroup, we generated the ratio of odds ratios (RORs) of each possible comparison: simple oxygen only vs NIV, simple oxygen only vs IMV, and NIV vs IMV. We then calculated the posterior probability of superiority for each comparison (ROR >1).

Between-Study Heterogeneity and Prediction

By accounting for both within-study and between-study variation, random-effects meta-analysis can better incorporate overall heterogeneity.6,12 In the bayesian framework, one acknowledges the prior uncertainty in the heterogeneity parameter and can estimate the posterior distribution of the between-study heterogeneity. We assessed the posterior distribution of the between-study standard deviation (τ, a proxy for heterogeneity) by calculating the posterior probabilities of low, “reasonable,” and “fairly high” heterogeneity (eMethods in the Supplement).6

Further, the posterior predictive distribution can better explore the effects of between-study heterogeneity. This distribution allows inference about “…what we would expect to see in a new study population that is exchangeable with the studies included in our meta-analysis.”17 The posterior predictive distribution is vital to inform probable values for the true treatment association in future settings, which is valuable for power calculations or prior distribution elicitation in future RCTs.6,18 The posterior predictive distribution for each subgroup incorporates the uncertainty both in the overall effect size and the between-study standard deviation parameters and generates new trial population parameter estimates for each subgroup, independent of sample size and other population characteristics.19

Predictive Analysis to Confirm Tocilizumab’s Association With Mortality Reduction

The posterior predictive distribution is useful to predict future settings. However, it does not allow one to directly estimate how many patients a next future trial would have to include to confirm the association of tocilizumab with mortality benefit. To assess future sample size requirements, we generated 6 RCTs of different sample sizes (from 200 to 4000 total patients) comparing tocilizumab and control in patients receiving IMV,20 assuming an OR of 0.77 (WHO REACT Working Group mean result for tocilizumab- and corticosteroids-treated patients).1 Next, using bayesian normal conjugate analyses, we updated our current knowledge (the meta-analysis results) with these generated RCTs (likelihoods) to form 6 separate posterior distributions. Further explanations about these calculations can be found in the eMethods in the Supplement.

Model Diagnostics and Code

Meta-analyses and posterior predictive results were conducted using Stan through the R package brms (the R Foundation).21,22 Four Markov chains were implemented with an initial warm-up phase of 2000 iterations, followed by 4000 iterations. We followed the When to Worry and How to Avoid the Misuse of Bayesian Statistics Checklist11,23,24 for checking details about our analysis, confirming the convergence and adequate sampling of the models. All analyses were conducted in R, version 4.0.4 (R Environment).

Results
Studies

Among the 5339 patients included in this analysis, most were men, with mean ages between 56 and 66 years. There were a total of 2117 patients receiving simple oxygen only, 2505 receiving NIV, and 717 receiving IMV in 15 studies from multiple countries and continents.1,25-32﻿ Table 1 shows the number of deaths and total patients in each RCT reported in the WHO Working Group publication and included in this reanalysis. As expected, smaller studies showed wider confidence intervals, and these studies were more likely to be unpublished studies. eTable 1 in the Supplement shows general demographic information and a more detailed assessment of sample sizes per trial and subgroup.

Tocilizumab’s Association With Subgroup Mortality Benefits

The separate subgroup marginal posterior distributions, generated with weakly informative priors, are displayed in Figure 1A. In patients receiving simple oxygen only, the median OR was 0.70 (95% CrI, 0.50-0.91), the posterior probability of any benefit association (<1.0 OR) was 98.9% (Figure 1B), and the posterior probability of a meaningful clinical association (OR < 0.9) was 95.5% (Figure 1B). In patients receiving NIV, the median OR was 0.81 (95% CrI, 0.63-1.03), and the posterior probabilities of any and of a meaningful clinical association were 95.5% (Figure 1A) and 82.2% (Figure 1B), respectively. The median OR was 0.89 (95% CrI, 0.61-1.22) among patients receiving IMV, whereas the posterior probabilities of any and of a meaningful clinical association were 75.4% (Figure 1A) and 52.9% (Figure 1B), respectively.

Absolute risk differences (RDs) are more informative for clinical decision-making.33﻿ Figure 1C-E shows the posterior distributions of the RDs for the 3 respiratory subgroups as a function of the baseline risk while assuming weakly informative priors. In patients receiving simple oxygen only (Figure 1C), the RD ranged from 2.8% (95% CrI, 0.7-4.6), with baseline risk of 10%, to an RD equal to 7.0% (95% CrI, 1.6-12.0), with baseline risk of 30%. Risk differences for the noninvasive subgroup are shown in Figure 1D. Among patients receiving IMV (Figure 1E), the RD ranged from 2.6% (95% CrI, −4.9 to 9.0), with baseline risk of 32%, to an RD equal to 3.0% (95% CrI, −5.4 to 11.2), with baseline risk of 52%.

Results in the simple oxygen only and NIV subgroups mostly showed associations with probabilities of any and meaningful clinical mortality benefit in the absolute scale (eFigure 1 in the Supplement). The posterior probabilities of clinically meaningful association (risk difference >1%) were greater than 95% and 90% in patients receiving simple oxygen only and NIV, respectively. In contrast, in the IMV subgroup, probabilities of any benefit (>0%) and clinically important associations were 80% and 67%, respectively. The posterior probabilities were mainly stable in each subgroup regardless of the baseline risk.

To assess whether tocilizumab was associated with reduced mortality to a greater extent in a specific subgroup, we calculated the RORs between subgroups (eFigure 2 and eTable 2 in the Supplement).1 We found a posterior probability of superiority for the simple oxygen subgroup over the NIV and IMV subgroups of 85.3% and 89.7%, respectively. For the NIV subgroup, we found a posterior probability of superiority of 68.9% over the IMV subgroup.

To test the robustness of these results, we repeated the analyses using 2 different sets of prior distributions (vague and informative). There were no substantial changes in the marginal posterior distributions and probabilities (Table 2 and eTable 2 in the Supplement).

Between-Study Heterogeneity

While assuming weakly informative priors, most of the posterior distribution for the between-study standard deviation (eFigure 3 in the Supplement) was concentrated around low or reasonable levels of heterogeneity (eTable 3 in the Supplement).6,12 These probabilities did not notably change on the use of different priors (eTable 3 in the Supplement).

Posterior Predictive Distributions

As shown in previous subsections, tocilizumab’s association with mortality benefit in hospitalized patients with COVID-19 receiving IMV is uncertain, as is the probability of clinically meaningful benefits in other respiratory subgroups (Figure 1). Figure 2A and eTable 4 in the Supplement show the predictive distributions for future trial populations for each subgroup while assuming weakly informative priors. The simple oxygen only subgroup showed large benefit (OR, 0.69; 95% CrI, 0.40-1.09) but now acknowledges that the next trial has a small possibility of showing harm (posterior probability of 5.6% for OR >1). In patients receiving NIV, the median OR was 0.80 (95% CrI, 0.46-1.25) along with a posterior probability of 12.4% for harm. The median predictive OR was 0.88 (95% CrI, 0.47-1.40) in the IMV group, and 27.9% of the area under the curve is to the right of 1, representing the probability of potential harm.

In a bayesian framework, one can also calculate the probability that future study populations will find a mean effect size less than a specific threshold (Figure 2B; eTable 4 in the Supplement). Regarding future study populations that exclusively include patients receiving simple oxygen only, there is a 94.2% probability they will find any benefit (<1 OR) and an 88.9% probability for greater benefit (OR <0.9). In contrast, regarding future studies that exclusively include patients receiving IMV, these probabilities were lower (72.1% and 53.3%, respectively).

Generating RCTs of Patients Receiving IMV

The results depicted in Figure 1A and E and Figure 2A highlight the uncertainty regarding the association of tocilizumab with mortality benefit, especially in patients receiving IMV. One could argue that a possible explanation for this uncertainty is the low total number of deaths (n = 357) and patients (n = 717) in this subgroup (Table 1), yielding a low statistical power to reach definitive conclusions. Therefore, it would be of interest to know the required sample size of a future trial meant to diminish this uncertainty. Bayesian analysis allows one to update posterior distributions with new (real or generated) RCTs of different sample sizes (eTable 5 in the Supplement).

Figure 3 shows the marginal posterior distributions and probabilities of these analyses. On the addition of more patients, the current evidence (717 patients) is increasingly dominated by new data (Figure 3A; eFigure 4 in the Supplement). For example, current evidence shows that there is only 75.4% probability of any benefit (<1 OR) in patients receiving IMV (Figure 1B and Figure 3B). However, on the addition of a potential future RCT (with an OR = 0.77 as in the WHO REACT meta-analysis on all patients treated with corticosteroids) of 1500 patients, the posterior probability of any benefit becomes greater than 99% (if 2217 patients were included; Figure 3B). Moreover, in a future potential RCT with the same effect size, enrolling 2000 patients (if 2717 patients were included; Figure 3B) will provide a posterior probability of an association with meaningful clinical outcome (<0.9 OR) greater than 95%.

On the other hand, sensitivity analyses combining our current invasive subgroup evidence with a potential future RCT enrolling 1500 patients showing a null effect (OR = 1.0) results in a posterior distribution with only a 20.9% probability of a meaningful clinical association (OR<0.9), suggesting that the futility threshold has likely been reached (eTable 6 and eFigure 5 in the Supplement).

Discussion

Herein we report our bayesian reanalysis of tocilizumab in COVID-19 respiratory support subgroups. The posterior probabilities of any benefit (OR <1) were notably different between patients on simple oxygen only and IMV (98.9% vs 75.4%). While the point estimates for benefit with tocilizumab were similar across different subgroups, the uncertainties and the probabilities of clinically meaningful association varied substantially, perhaps in part owing to differences in sample size and statistical power. Patients receiving simple oxygen only showed the largest benefit with tocilizumab and the most certainty of being associated with a clinically meaningful benefit, indicated by a greater than 94% probability that the mortality risk difference is greater than 1%. The probabilities of tocilizumab superiority in the simple oxygen subgroup compared with the noninvasive and invasive mechanical subgroups were 85.3% and 89.7%, respectively. Although we found evidence for low or “reasonable” between-study heterogeneity, predictive intervals highlighted that only 72% of studies for patients receiving IMV will demonstrate any benefit. Through bayesian normal conjugate analyses, we also found that a future tocilizumab IMV RCT (with an effect size compatible with what has been observed to date) would require approximately 2000 patients to find an association with a probability of meaningful clinical outcome in this subgroup.

Comparison With Other Studies

In the WHO REACT Working Group publication,1 the authors state, “These differences between subgroups may have arisen due to sampling variation.”1(p515) Here, we present a more in-depth analysis and interpretation of these results. First, the association with posterior probabilities of any and of clinically important mortality benefits is strikingly different across subgroups (Figure 1). Second, we estimated the probability that the association of tocilizumab with mortality benefit is different between subgroups (eFigure 2 in the Supplement). Third, we estimated the prediction intervals in each subgroup, reflecting both the uncertainty around the association in each subgroup and the heterogeneity between studies. Prediction intervals showed that one could expect a large proportion of future studies (15%) on patients receiving IMV only to find an association with clinically meaningful harm (ie, OR >1.11; eTable 4 in the Supplement). Fourth, bayesian normal conjugate analysis highlighted that we would require a 2000-patient RCT with a substantial benefit to provide an association with a probability that tocilizumab reduces mortality in patients receiving IMV. Ultimately, we revealed a clear contrast between the strength of the mortality associations for the IMV subgroup and patients with less severe COVID-19, indicating the need for more evidence of tocilizumab’s benefit in critically ill patients.

Our research group has previously reanalyzed the association of tocilizumab with clinical outcomes in different COVID-19 subgroups of patients while focusing on the RECOVERY trial.34 However, we did not collect and analyze data limited to patients using corticosteroids in different respiratory subgroups. The lack of this restriction is an important limitation because there is evidence that corticosteroids may synergistically interact with tocilizumab.1 Moreover, guidelines suggest that all hospitalized patients with COVID-19 who are receiving supplemental oxygen should receive corticosteroids.2 Thus, in this work, we conditioned our results and conclusions to the use of corticosteroids, which limit potential biases if tocilizumab’s association with mortality benefit depends on the use of corticosteroids.

Limitations

Notwithstanding these strengths, our study also has limitations. First, this reanalysis was not preplanned and should be interpreted as exploratory. Second, we did not have access to patient-level data for any RCTs included in this article. Subgroup analyses that separate patients by a single baseline characteristic are oversimplified and can bring shortcomings.33 Lack of patient-level data does not allow analyses using more complex statistical models that incorporate multiple characteristics.35 Nevertheless, by limiting these analyses to patients using corticosteroids, we believe concerns about this limitation were mitigated.1 Third is the classical argument of bayesian subjectivity owing to the need for priors.36 Although we agree with the need for transparency and caution in the choice of priors, the use of weakly informative priors is likely to furnish the most accurate and least biased treatment estimates. Moreover, we also applied different priors yielding similar results, confirming the robustness of the main analysis. Fourth, we limited our focus to a single outcome: 28-day all-cause mortality. Fifth, while the WHO REACT Working Group included other drugs in their meta-analysis,1 we limited our reanalysis to tocilizumab because we did not want to introduce further uncertainties about possible drug equivalences. Sixth, we combined within-trial and across-trial interaction estimates to compare subgroups, which could subject our analysis to ecological bias.37

Conclusions

In this bayesian reanalysis of a previous meta-analysis of 15 RCTs, patients with COVID-19 receiving simple oxygen only or noninvasive ventilation and receiving tocilizumab treatment were associated with a probability of a clinically meaningful mortality benefit. In contrast, the beneficial association of this drug was uncertain in patients receiving IMV, and increased harm was not conclusively excluded. Future research should better define whether patients receiving IMV also benefit from tocilizumab.

Article Information

Accepted for Publication: January 11, 2022.

Published: February 28, 2022. doi:10.1001/jamanetworkopen.2022.0548

Correction: This article was corrected on March 22, 2022, to delete “(95% CI)” from the x-axis labels of Figures 1B and 2B.

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Albuquerque AM et al. JAMA Network Open.

Corresponding Author: James M. Brophy, MD, PhD, McGill Health University Center, 5252 Boul de Maisonneuve W, Room 2B.37, Montreal, QC H4A 3S5, Canada (james.brophy@mcgill.ca).

Author Contributions: Mr Albuquerque and Dr Brophy had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Albuquerque, Tramujas, Williams, Brophy.

Acquisition, analysis, or interpretation of data: Albuquerque, Sewanan, Brophy.

Drafting of the manuscript: Albuquerque, Sewanan, Williams.

Critical revision of the manuscript for important intellectual content: Albuquerque, Tramujas, Sewanan, Brophy.

Statistical analysis: Albuquerque, Sewanan, Williams, Brophy.

Supervision: Tramujas, Brophy.

Conflict of Interest Disclosures: Dr Tramujas participated as a subinvestigator in TOCIBRAS, a randomized clinical trial included in this article. No other disclosures were reported.

Funding/Support: Dr Brophy is a research scholar supported by Les Fonds de Recherche Québec Santé.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Information: The complete analysis code is available at https://github.com/arthur-albuquerque/tocilizumab_reanalysis.

References
1.
Shankar-Hari  M﻿, Vale  CL﻿, Godolphin  PJ﻿,  et al; WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group.  Association between administration of IL-6 antagonists and mortality among patients hospitalized for COVID-19: a meta-analysis. ﻿  JAMA. 2021;326(6):499-518. doi:10.1001/jama.2021.11330﻿ PubMedGoogle Scholar
2.
Therapeutics and COVID-19: living guideline. World Health Organization. Accessed September 2, 2021. https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2021.2
3.
Wijeysundera  DN﻿, Austin  PC﻿, Hux  JE﻿, Beattie  WS﻿, Laupacis  A﻿.  Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials. ﻿  J Clin Epidemiol. 2009;62(1):13-21.e5. doi:10.1016/j.jclinepi.2008.07.006﻿ PubMedGoogle ScholarCrossref
4.
Goligher  EC﻿, Tomlinson  G﻿, Hajage  D﻿,  et al.  Extracorporeal membrane oxygenation for severe acute respiratory distress syndrome and posterior probability of mortality benefit in a post hoc bayesian analysis of a randomized clinical trial. ﻿  JAMA. 2018;320(21):2251-2259. doi:10.1001/jama.2018.14276﻿ PubMedGoogle ScholarCrossref
5.
Brophy  JM﻿.  Bayesian interpretation of the EXCEL trial and other randomized clinical trials of left main coronary artery revascularization. ﻿  JAMA Intern Med. 2020;180(7):986-992. doi:10.1001/jamainternmed.2020.1647﻿ PubMedGoogle ScholarCrossref
6.
Spiegelhalter  DJ﻿, Abrams  KR﻿, Myles  JP﻿.  Bayesian Approaches to Clinical Trials and Health Care Evaluation. Wiley; 2004.
7.
Greenland  S﻿.  Invited commentary: the need for cognitive science in methodology. ﻿  Am J Epidemiol. 2017;186(6):639-645. doi:10.1093/aje/kwx259﻿ PubMedGoogle ScholarCrossref
8.
Page  MJ﻿, McKenzie  JE﻿, Bossuyt  PM﻿,  et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. ﻿  BMJ. 2021;372(n71):n71. doi:10.1136/bmj.n71﻿ PubMedGoogle Scholar
9.
Hedges  LV﻿, Vevea  JL﻿.  Fixed- and random-effects models in meta-analysis. ﻿  Psychol Methods. 1998;3(4):486-504. doi:10.1037/1082-989X.3.4.486﻿ Google ScholarCrossref
10.
Viechtbauer  W﻿.  Conducting meta-analyses in R with the metafor package. ﻿  J Stat Softw. 2010;36(1):1-48. doi:10.18637/jss.v036.i03PubMedGoogle Scholar
11.
van de Schoot  R﻿, Veen  D﻿, Smeets  L﻿, Winter  SD﻿, Depaoli  S﻿. A tutorial on using the WAMBS checklist to avoid the misuse of bayesian statistics. In: van de Schoot  R﻿, Miočević  M﻿, eds.  Small Sample Size Solutions. 1st ed. Routledge; 2020:30-49. doi:10.4324/9780429273872-4﻿
12.
Röver  C﻿, Bender  R﻿, Dias  S﻿,  et al.  On weakly informative prior distributions for the heterogeneity parameter in bayesian random-effects meta-analysis. ﻿  Res Synth Methods. 2021;12(4):448-474. doi:10.1002/jrsm.1475﻿ PubMedGoogle ScholarCrossref
13.
Turner  RM﻿, Jackson  D﻿, Wei  Y﻿, Thompson  SG﻿, Higgins  JPT﻿.  Predictive distributions for between-study heterogeneity and simple methods for their application in bayesian meta-analysis. ﻿  Stat Med. 2015;34(6):984-998. doi:10.1002/sim.6381﻿ PubMedGoogle ScholarCrossref
14.
McElreath  R﻿.  Statistical Rethinking: A Bayesian Course With Examples in R and Stan. Chapman & Hall/CRC; 2020.
15.
Doi  SA﻿, Furuya-Kanamori  L﻿, Xu  C﻿, Lin  L﻿, Chivese  T﻿, Thalib  L﻿.  Questionable utility of the relative risk in clinical research: a call for change to practice. ﻿  J Clin Epidemiol. Published online November 7, 2020. doi:10.1016/j.jclinepi.2020.08.019PubMedGoogle Scholar
16.
Doi  SA﻿, Furuya-Kanamori  L﻿, Xu  C﻿,  et al.  The odds ratio is “portable” but not the relative risk: time to do away with the log link in binomial regression. ﻿  J Clin Epidemiol. Published online August 8, 2021. doi:10.1016/j.jclinepi.2021.08.003Google Scholar
17.
Welton  NJ﻿, Jones  HE﻿, Dias  S﻿. Bayesian methods for meta-analysis. In: Lesaffre E, Baio G, Boulanger B, eds.  Bayesian Methods in Pharmaceutical Research. Chapman & Hall/CRC; 2020.
18.
IntHout  J﻿, Ioannidis  JP﻿, Rovers  MM﻿, Goeman  JJ﻿.  Plea for routinely presenting prediction intervals in meta-analysis. ﻿  BMJ Open. 2016;6(7):e010247. doi:10.1136/bmjopen-2015-010247﻿ PubMedGoogle Scholar
19.
Higgins  JPT﻿, Thompson  SG﻿, Spiegelhalter  DJ﻿.  A re-evaluation of random-effects meta-analysis. ﻿  J R Stat Soc Ser A Stat Soc. 2009;172(1):137-159. doi:10.1111/j.1467-985X.2008.00552.x﻿ PubMedGoogle ScholarCrossref
20.
Kalil  AC﻿, Sun  J﻿.  How many patients with severe sepsis are needed to confirm the efficacy of drotrecogin alfa activated? a bayesian design. ﻿  Intensive Care Med. 2008;34(10):1804-1811. doi:10.1007/s00134-008-1159-8﻿ PubMedGoogle ScholarCrossref
21.
Carpenter  B﻿, Gelman  A﻿, Hoffman  MD﻿,  et al.  Stan: a probabilistic programming language. ﻿  J Stat Softw. 2017;76(1):1-32. doi:10.18637/jss.v076.i01﻿ Google ScholarCrossref
22.
Bürkner  PC﻿.  brms: an R package for bayesian multilevel models using stan. ﻿  J Stat Softw. 2017;80(1):1-28. doi:10.18637/jss.v080.i01﻿ Google ScholarCrossref
23.
Depaoli  S﻿, van de Schoot  R﻿.  Improving transparency and replication in bayesian statistics: the WAMBS-checklist. ﻿  Psychol Methods. 2017;22(2):240-261. doi:10.1037/met0000065﻿ PubMedGoogle ScholarCrossref
24.
van de Schoot  R﻿, Depaoli  S﻿, King  R﻿,  et al.  Bayesian statistics and modelling. ﻿  Nat Rev Methods Primer. 2021;1(1):1-26. doi:10.1038/s43586-020-00001-2﻿ Google ScholarCrossref
25.
Stone  JH﻿, Frigault  MJ﻿, Serling-Boyd  NJ﻿,  et al; BACC Bay Tocilizumab Trial Investigators.  Efficacy of tocilizumab in patients hospitalized with COVID-19. ﻿  N Engl J Med. 2020;383(24):2333-2344. doi:10.1056/NEJMoa2028836﻿ PubMedGoogle ScholarCrossref
26.
Hermine  O﻿, Mariette  X﻿, Tharaux  PL﻿, Resche-Rigon  M﻿, Porcher  R﻿, Ravaud  P﻿; CORIMUNO-19 Collaborative Group.  Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: a randomized clinical trial. ﻿  JAMA Intern Med. 2021;181(1):32-40. doi:10.1001/jamainternmed.2020.6820﻿ PubMedGoogle ScholarCrossref
27.
Rosas  IO﻿, Bräu  N﻿, Waters  M﻿,  et al.  Tocilizumab in hospitalized patients with severe COVID-19 pneumonia. ﻿  N Engl J Med. 2021;384(16):1503-1516. doi:10.1056/NEJMoa2028700﻿ PubMedGoogle ScholarCrossref
28.
Salama  C﻿, Han  J﻿, Yau  L﻿,  et al.  Tocilizumab in patients hospitalized with COVID-19 pneumonia. ﻿  N Engl J Med. 2021;384(1):20-30. doi:10.1056/NEJMoa2030340﻿ PubMedGoogle ScholarCrossref
29.
Rosas  IO﻿, Diaz  G﻿, Gottlieb  RL﻿,  et al.  Tocilizumab and remdesivir in hospitalized patients with severe COVID-19 pneumonia: a randomized clinical trial. ﻿  Intensive Care Med. 2021;47(11):1258-1270. doi:10.1007/s00134-021-06507-x﻿ PubMedGoogle ScholarCrossref
30.
Veiga  VC﻿, Prats  JAGG﻿, Farias  DLC﻿,  et al; Coalition COVID-19 Brazil VI Investigators.  Effect of tocilizumab on clinical outcomes at 15 days in patients with severe or critical coronavirus disease 2019: randomised controlled trial. ﻿  BMJ. 2021;372(n84):n84. doi:10.1136/bmj.n84﻿ PubMedGoogle Scholar
31.
Abani  O﻿, Abbas  A﻿, Abbas  F﻿,  et al; RECOVERY Collaborative Group.  Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. ﻿  Lancet. 2021;397(10285):1637-1645. doi:10.1016/S0140-6736(21)00676-0﻿ PubMedGoogle ScholarCrossref
32.
Gordon  AC﻿, Mouncey  PR﻿, Al-Beidh  F﻿,  et al; REMAP-CAP Investigators.  Interleukin-6 receptor antagonists in critically ill patients with COVID-19. ﻿  N Engl J Med. 2021;384(16):1491-1502. doi:10.1056/NEJMoa2100433﻿ PubMedGoogle Scholar
33.
Kent  DM﻿, Steyerberg  E﻿, van Klaveren  D﻿.  Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. ﻿  BMJ. 2018;363:k4245. doi:10.1136/bmj.k4245﻿ PubMedGoogle Scholar
34.
Albuquerque  AM﻿, Tramujas  L﻿, Sewanan  LR﻿, Brophy  JM﻿.  Tocilizumab in COVID-19—a bayesian reanalysis of RECOVERY. ﻿  medRxiv. Preprint posted online June 21, 2021. doi:10.1101/2021.06.15.21258966Google Scholar
35.
Kent  DM﻿, Paulus  JK﻿, van Klaveren  D﻿,  et al.  The Predictive Approaches to Treatment effect Heterogeneity (PATH) statement. ﻿  Ann Intern Med. 2020;172(1):35-45. doi:10.7326/M18-3667﻿ PubMedGoogle ScholarCrossref
36.
Aberegg  SK﻿.  Post hoc bayesian analyses. ﻿  JAMA. 2019;321(16):1631-1632. doi:10.1001/jama.2019.1198﻿ PubMedGoogle ScholarCrossref
37.
Fisher  DJ﻿, Carpenter  JR﻿, Morris  TP﻿, Freeman  SC﻿, Tierney  JF﻿.  Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? ﻿  BMJ. 2017;356:j573. doi:10.1136/bmj.j573﻿ PubMedGoogle Scholar