Use of Prediction Intervals in Network Meta-analysis

This cross-sectional study examines the prevalence of reporting prediction intervals in network meta-analysis articles and provides a worked example.


eAppendix 1. Literature Search for Articles on Network Meta-analysis
We searched for full-length articles with original data of network meta-analyses from 2010 to 2018, and we excluded letters, commentaries, news, and research methodology and reporting. The literature search was conducted via the official websites of JAMA (https://jamanetwork.com/advanced-search), the Lancet (https://www.thelancet.com/search/advanced), and BMJ (https://www.bmj.com/search/advanced). We downloaded articles with the term "meta analysis", "meta analyses", "meta epidemiological", "network meta analysis", "network meta analyses", "mixed treatment comparison(s)", or "multiple treatment(s)" in their titles. Then, we looked at the articles' contents, and excluded articles if they were not network metaanalyses of multiple treatments. Some articles were also excluded when they presented data of multiple treatments but did not formally perform network meta-analyses to simultaneously synthesize their evidence. Of note, although most articles with the term "meta analysis", "meta analyses", or "meta epidemiological" were conventional pairwise meta-analyses, some were actually network meta-analyses; therefore, these terms were retained in our literature search. In addition, we compared the obtained articles with the network meta-analyses collected by Nikolakopoulou et al. 1 (searched before 2014), Bafeta et al. 2,3 (searched before 2013), and Trinquart et al. 4 (searched before 2013 with restrictions on model types). Missed articles were added to our final list (eTable 1). eAppendix 2. Example of Producing Prediction Intervals in Network Meta-analysis eTable 2 presents software programs for NMAs and the corresponding commands for producing prediction intervals. All commands are simple and do not require much additional effort in the analyses. This appendix applies the available software programs to the example dataset of smoking cessation. The following statistical analyses estimated odds ratios on a logarithmic scale with 95% confidence/credible intervals and prediction intervals. The specific commands for producing prediction intervals are highlighted in the following statistical code.

Stata
If the Stata routines for network meta-analysis have not been installed yet, type the following command in Stata . net from http://www.homepages.ucl.ac.uk/~rmjwiww/stata/ Click the link meta and install the packages network and mvmeta. Also, type the following command . net from http://www.stata-journal.com/software/sj15-4/ Install the package st0411. The detailed instructions of these Stata routines have been provided by Chaimani et al., 5 Chaimani and Salanti, 6 and White. 7 The smoking cessation dataset is stored in the file "smokingcessation.dta". By typing Note that the continuity correction of 0.5 is applied to each study that contains zero event in a certain treatment arm. Then, we run the following commands to implement network meta-analysis and derive 95% prediction intervals: . network setup r n, studyvar(study) . network table . network meta consistency . intervalplot, pred The last command returns the following results: The intervalplot command assumes that the saved results from mvmeta or network meta commands have been derived from the current > dataset

R package "netmeta"
If the R package "netmeta" has not been installed yet, type the following command in R to install this package: Load this package:

WinBUGS (implemented via R package "R2WinBUGS")
If the software program WinBUGS has not been downloaded, it is available at https://www.mrcbsu.cam.ac.uk/wp-content/uploads/2018/11/winbugs143_unrestricted.zip. After unzipping the downloaded file, the software program "WinBUGS14.exe" is located in the folder "WinBUGS14". If the R package "R2WinBUGS" has not been installed yet, type the following command in R to install this package: This package implements BUGS models via the R platform. Load the R package "R2WinBUGS": > library("R2WinBUGS") The BUGS model for network meta-analysis can be specified in the following R function:  8 and all remaining parts are the conventional code to perform Bayesian network metaanalysis. 9,10 After specifying the BUGS model, we type the following R code to prepare the smoking cessation dataset in the format of the above BUGS model: (3,2), rep(2, NS -2)) > r <-as.matrix(smokingcessation[,c("event1", "event2", "event3")]) > n <-as.matrix(smokingcessation[,c("n1", "n2", "n3")]) > treat <-smokingcessation[,c("treat1", "treat2", "treat3")] > t <-matrix(NA, NS, 3) > for(i in 1: Based on the above reformatted data, we specify the objects of data and the parameters' initial values (used to initialize the Markov chain Monte Carlo algorithm) in the BUGS model as follows: Consequently, the following code performs Bayesian network meta-analysis by invoking WinBUGS from R: > set.seed(1234) > out.bugs <-bugs(data = data, inits = inits, + parameters.to.save = c("lor", "lor.new", "sd"), + model.file = BayesianNMAModel, + n.chains = 3, n.iter = 50000, n.burnin = 20000, n.thin = 2, + bugs.directory = bugs.dir) > out.bugs.smry <-out.bugs$summary > est.bugs <-CI.lower. Here, the first line set.seed(1234) specifies a seed to generate random numbers for starting the Markov chain Monte Carlo algorithm, so that all results can be exactly reproduced. The argument model.file = BayesianNMAModel in the function bugs() specifies the BUGS model; here, recall that BayesianNMAModel is an R function containing the BUGS model as defined above. If a separate text file is used to specify the BUGS model, the argument model.file should be a character string of the text file's name (and the path to this file if the working directory is not pre-specified). In addition, the argument bugs.dir specifies the path to the location of the WinBUGS software program on the user's PC; usually, it has the form of ".../WinBUGS14".
The results are obtained using three Markov chains (specified by the argument n.chains), each having 30,000 iterations (which is the argument n.iter minus the argument n.burnin) after a 20,000-  15 2015 JAMA Frequentist RE model No Isayama et al. 16 2016 JAMA Bayesian RE model No Khera et al. 17 2016 JAMA Bayesian RE model No Palmer et al. 18 2016 JAMA Frequentist RE model No Tricco et al. 19 2017 JAMA Frequentist RE model No Gregori et al. 20 2018 JAMA Bayesian RE model No Mitra et al. 21 2018 JAMA Bayesian RE model No Zheng et al. 22 2018 JAMA Bayesian FE/RE model No Cipriani et al. 23 2011 Lancet Bayesian RE model No Palmerini et al. 24 2012 Lancet Bayesian RE model No Leucht et al. 25 2013 Lancet Bayesian RE model No Palmer et al. 26 2015 Lancet Frequentist RE model No Palmerini et al. 27 2015 Lancet Bayesian RE model No Singh et al. 28 2015 Lancet Bayesian FE/RE model No Siontis et al. 29 2015 Lancet Frequentist RE model No Cipriani et al. 30 2016 Lancet Bayesian RE model No da Costa et al. 31 2017 Lancet Bayesian RE model No Jinatongthai et al. 32 2017 Lancet Frequentist RE model No Cipriani et al. 33 2018 Lancet Bayesian RE model No Wandel et al. 34 2010 BMJ Bayesian RE model No Baldwin et al. 35 2011 BMJ Bayesian/frequentist RE model No Hartling et al. 36 2011 BMJ Bayesian RE model No Trelle et al. 37 2011 BMJ Bayesian RE model No Bangalore et al. 38 2012 BMJ Bayesian RE model No Daniels et al. 39 2012 BMJ Bayesian FE/RE model No Haas et al. 40 2012 BMJ Bayesian RE model Yes Hutton et al. 41 2012 BMJ Bayesian RE model Yes Bangalore et al. 42 2013 BMJ Bayesian RE model No Castellucci et al. 43 2013 BMJ Bayesian RE model No Chatterjee et al. 44 2013 BMJ Bayesian RE model No a Naci and Ioannidis 45 2013 BMJ Bayesian RE model No Navarese et al. 46 2013 BMJ Bayesian RE model No Stegeman et al. 47 2013 BMJ Frequentist RE model No Uthman et al. 48 2013 BMJ Bayesian RE model No Wu et al. 49 2013 BMJ Bayesian RE model No Bangalore et al. 50 2014 BMJ Frequentist RE model No Loymans et al. 51 2014 BMJ Bayesian RE model No Naci et al. 52 2014 BMJ Bayesian FE/RE model No Price et al. 53 2014 BMJ Bayesian RE model No Tricco et al. 54 2014 BMJ Frequentist RE model Yes Windecker et al. 55 2014 BMJ Bayesian RE model No Alfirevic et al. 56 2015 BMJ Bayesian FE/RE model No Giacoppo et al. 57 2015 BMJ Bayesian RE model No Li et al. 58 2015 BMJ Other software programs that support the Markov chain Monte Carlo algorithm (e.g., JAGS, OpenBUGS, SAS, Stan) can be also used to perform Bayesian network meta-analyses, and thus readily produce prediction intervals.

eFigure. Estimated Overall Odds Ratios of 6 Treatment Comparisons in the Network Meta-analysis of Smoking Cessation Using Stata, R Package netmeta, and WinBUGS
A indicates no intervention; B, self-help; C, individual counseling; and D, group counseling.