[Skip to Navigation]
Sign In
Figure.  PRISMA-IPD Flow Diagram
PRISMA-IPD Flow Diagram

IPD indicates individual patient data; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Reproduced with permission of the PRISMA-IPD Group, which encourages sharing and reuse.

Table.  PRISMA-IPD Checklist of Items to Include When Reporting a Systematic Review and Meta-analysis of Individual Participant Data (IPD)a
PRISMA-IPD Checklist of Items to Include When Reporting a Systematic Review and Meta-analysis of Individual Participant Data (IPD)a
1.
Moher  D, Liberati  A, Tetzlaff  J, Altman  DG; PRISMA Group.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.  PLoS Med. 2009;6(7):e1000097.PubMedGoogle ScholarCrossref
2.
Moher  D, Schulz  KF, Simera  I, Altman  DG.  Guidance for developers of health research reporting guidelines.  PLoS Med. 2010;7(2):e1000217.PubMedGoogle ScholarCrossref
3.
Simmonds  MC, Higgins  JP, Stewart  LA, Tierney  JF, Clarke  MJ, Thompson  SG.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice.  Clin Trials. 2005;2(3):209-217.PubMedGoogle ScholarCrossref
4.
Riley  RD, Lambert  PC, Abo-Zaid  G.  Meta-analysis of individual participant data: rationale, conduct, and reporting.  BMJ. 2010;340:c221.PubMedGoogle ScholarCrossref
5.
Chalmers  I; Cochrane Collaboration.  The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care.  Ann N Y Acad Sci. 1993;703:156-163.PubMedGoogle ScholarCrossref
6.
Stewart  LA, Clarke  MJ; Cochrane Working Group.  Practical methodology of meta-analyses (overviews) using updated individual patient data.  Stat Med. 1995;14(19):2057-2079.PubMedGoogle ScholarCrossref
7.
Stewart  LA, Tierney  JF.  To IPD or not to IPD? advantages and disadvantages of systematic reviews using individual patient data.  Eval Health Prof. 2002;25(1):76-97.PubMedGoogle ScholarCrossref
8.
Stewart  L, Tierney  J, Burdett  S. Do systematic reviews based on individual patient data offer a means of circumventing biases associated with trial publications? In: Rothstein  H, Sutton  A, Borenstein  M, eds.  Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, United Kingdom: John Wiley & Sons; 2005:261-286.
9.
Burgess  S, White  IR, Resche-Rigon  M, Wood  AM.  Combining multiple imputation and meta-analysis with individual participant data.  Stat Med. 2013;32(26):4499-4514.PubMedGoogle ScholarCrossref
10.
Debray  TP, Moons  KG, Ahmed  I, Koffijberg  H, Riley  RD.  A framework for developing, implementing, and evaluating clinical prediction models in an individual patient data meta-analysis.  Stat Med. 2013;32(18):3158-3180.PubMedGoogle ScholarCrossref
11.
Phillips  RS, Sutton  AJ, Riley  RD, Chisholm  JC, Picton  SV, Stewart  LA; PICNICC Collaboration.  Predicting infectious complications in neutropenic children and young people with cancer (IPD protocol).  Syst Rev. 2012;1:8.PubMedGoogle ScholarCrossref
12.
Clarke  M, Stewart  L, Pignon  JP, Bijnens  L.  Individual patient data meta-analysis in cancer.  Br J Cancer. 1998;77(11):2036-2044.PubMedGoogle ScholarCrossref
13.
Selker  HP, Griffith  JL, Beshansky  JR,  et al.  Patient-specific predictions of outcomes in myocardial infarction for real-time emergency use: a thrombolytic predictive instrument.  Ann Intern Med. 1997;127(7):538-556.PubMedGoogle ScholarCrossref
14.
Krumholz  HM.  Why data sharing should be the expected norm.  BMJ. 2015;350:h599.PubMedGoogle ScholarCrossref
15.
Lo  B.  Sharing clinical trial data: maximizing benefits, minimizing risk.  JAMA. 2015;313(8):793-794.PubMedGoogle ScholarCrossref
16.
Ahmed  I, Sutton  AJ, Riley  RD.  Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey.  BMJ. 2012;344:d7762.PubMedGoogle ScholarCrossref
17.
Beller  EM, Glasziou  PP, Altman  DG,  et al; PRISMA for Abstracts Group.  PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts.  PLoS Med. 2013;10(4):e1001419.PubMedGoogle ScholarCrossref
18.
Vale  CL, Rydzewska  LHM, Rovers  MM, Emberson  JR, Gueyffier  F, Stewart  LA; Cochrane IPD Meta-analysis Methods Group.  Uptake of systematic reviews and meta-analyses based on individual participant data in clinical practice guidelines: descriptive study.  BMJ. 2015;350:h1088.PubMedGoogle ScholarCrossref
19.
Schmid  CH, Landa  M, Jafar  TH,  et al; Angiotensin-Converting Enzyme Inhibition in Progressive Renal Disease (AIPRD) Study Group.  Constructing a database of individual clinical trials for longitudinal analysis.  Control Clin Trials. 2003;24(3):324-340.PubMedGoogle ScholarCrossref
20.
Riley  RD, Lambert  PC, Staessen  JA,  et al.  Meta-analysis of continuous outcomes combining individual patient data and aggregate data.  Stat Med. 2008;27(11):1870-1893.PubMedGoogle ScholarCrossref
21.
Abo-Zaid  G, Guo  B, Deeks  JJ,  et al.  Individual participant data meta-analyses should not ignore clustering.  J Clin Epidemiol. 2013;66(8):865-873.PubMedGoogle ScholarCrossref
22.
Non–small Cell Lung Cancer Collaborative Group.  Chemotherapy in non–small cell lung cancer: a meta-analysis using updated data on individual patients from 52 randomised clinical trials.  BMJ. 1995;311(7010):899-909.PubMedGoogle ScholarCrossref
23.
Steyerberg  EW, Moons  KG, van der Windt  DA,  et al; PROGRESS Group.  Prognosis Research Strategy (PROGRESS) 3: prognostic model research.  PLoS Med. 2013;10(2):e1001381.PubMedGoogle ScholarCrossref
24.
Hingorani  AD, Windt  DA, Riley  RD,  et al; PROGRESS Group.  Prognosis research strategy (PROGRESS) 4: stratified medicine research.  BMJ. 2013;346:e5793.PubMedGoogle ScholarCrossref
25.
Early Breast Cancer Trialists' Collaborative Group.  Effects of adjuvant tamoxifen and of cytotoxic therapy on mortality in early breast cancer: an overview of 61 randomized trials among 28,896 women.  N Engl J Med. 1988;319(26):1681-1692.PubMedGoogle ScholarCrossref
26.
Advanced Ovarian Cancer Trialists’ Group.  Chemotherapy in advanced ovarian cancer: an overview of randomised clinical trials.  BMJ. 1991;303(6807):884-893.PubMedGoogle ScholarCrossref
27.
Mant  J, Doust  J, Roalfe  A,  et al.  Systematic review and individual patient data meta-analysis of diagnosis of heart failure, with modelling of implications of different diagnostic strategies in primary care.  Health Technol Assess. 2009;13(32):1-207.PubMedGoogle ScholarCrossref
28.
Abo-Zaid  G, Sauerbrei  W, Riley  RD.  Individual participant data meta-analysis of prognostic factor studies: state of the art?  BMC Med Res Methodol. 2012;12:56.PubMedGoogle ScholarCrossref
29.
Ahmed  I, Debray  TP, Moons  KG, Riley  RD.  Developing and validating risk prediction models in an individual participant data meta-analysis.  BMC Med Res Methodol. 2014;14:3.PubMedGoogle ScholarCrossref
30.
Danesh  J, Lewington  S, Thompson  SG,  et al; Fibrinogen Studies Collaboration.  Plasma fibrinogen level and the risk of major cardiovascular diseases and nonvascular mortality: an individual participant meta-analysis.  JAMA. 2005;294(14):1799-1809.PubMedGoogle Scholar
31.
van der A  DL, Rovers  MM, Grobbee  DE,  et al.  Mutations in the HFE gene and cardiovascular disease risk: an individual patient data meta-analysis of 53 880 subjects.  Circ Cardiovasc Genet. 2008;1(1):43-50.PubMedGoogle ScholarCrossref
32.
Burgess  S, Thompson  SG; Genetics Collaboration CC.  Methods for meta-analysis of individual participant data from mendelian randomisation studies with binary outcomes [published online June 19, 2012].  Stat Methods Med Res. doi:10.1177/0962280212451882.Google Scholar
33.
Bath  PM, Gray  LJ, Bath  AJ, Buchan  A, Miyata  T, Green  AR; NXY-059 Efficacy Meta-analysis in Individual Animals With Stroke Investigators.  Effects of NXY-059 in experimental stroke: an individual animal meta-analysis.  Br J Pharmacol. 2009;157(7):1157-1171.PubMedGoogle ScholarCrossref
34.
Schmidt  AF, Nielen  M, Klungel  OH,  et al; V.S.S.O. Investigators.  Prognostic factors of early metastasis and mortality in dogs with appendicular osteosarcoma after receiving surgery: an individual patient data meta-analysis.  Prev Vet Med. 2013;112(3-4):414-422.PubMedGoogle ScholarCrossref
35.
Berlin  JA, Ghersi  D. Preventing publication bias: registries and prospective meta-analysis. In: Rothstein  HR, Sutton  AJ, Borenstein  M, eds.  Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. Chichester, United Kingdom: John Wiley & Sons; 2005:35-48.
Special Communication
April 28, 2015

Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement

Author Affiliations
  • 1Centre for Reviews and Dissemination, University of York, York, United Kingdom
  • 2All-Ireland Hub for Trials Methodology Research, Queen’s University Belfast, Belfast, United Kingdom
  • 3Radbound Institue of Health Sciences, Radboudumc, Nijmegen, the Netherlands
  • 4Research Institute of Primary Care and Health Sciences, Keele University, Keele, United Kingdom (initial work carried out at School of Health and Population Sciences, University of Birmingham, Birmingham, United Kingdom)
  • 5Centre for Rural Economy, School of Agriculture, Food and Rural Development Newcastle University, Newcastle, United Kingdom (work carried out at the Centre for Reviews and Dissemination, University of York, United Kingdom)
  • 6MRC Clinical Trials Unit at UCL, London, United Kingdom
JAMA. 2015;313(16):1657-1665. doi:10.1001/jama.2015.3656
Abstract

Importance  Systematic reviews and meta-analyses of individual participant data (IPD) aim to collect, check, and reanalyze individual-level data from all studies addressing a particular research question and are therefore considered a gold standard approach to evidence synthesis. They are likely to be used with increasing frequency as current initiatives to share clinical trial data gain momentum and may be particularly important in reviewing controversial therapeutic areas.

Objective  To develop PRISMA-IPD as a stand-alone extension to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, tailored to the specific requirements of reporting systematic reviews and meta-analyses of IPD. Although developed primarily for reviews of randomized trials, many items will apply in other contexts, including reviews of diagnosis and prognosis.

Design  Development of PRISMA-IPD followed the EQUATOR Network framework guidance and used the existing standard PRISMA Statement as a starting point to draft additional relevant material. A web-based survey informed discussion at an international workshop that included researchers, clinicians, methodologists experienced in conducting systematic reviews and meta-analyses of IPD, and journal editors. The statement was drafted and iterative refinements were made by the project, advisory, and development groups. The PRISMA-IPD Development Group reached agreement on the PRISMA-IPD checklist and flow diagram by consensus.

Findings  Compared with standard PRISMA, the PRISMA-IPD checklist includes 3 new items that address (1) methods of checking the integrity of the IPD (such as pattern of randomization, data consistency, baseline imbalance, and missing data), (2) reporting any important issues that emerge, and (3) exploring variation (such as whether certain types of individual benefit more from the intervention than others). A further additional item was created by reorganization of standard PRISMA items relating to interpreting results. Wording was modified in 23 items to reflect the IPD approach.

Conclusions and Relevance  PRISMA-IPD provides guidelines for reporting systematic reviews and meta-analyses of IPD.

Introduction

Systematic reviews and meta-analyses of individual participant data (IPD) aim to identify, appraise, and summarize the evidence from multiple studies addressing the same research question or topic. Unlike most systematic reviews, they do not rely on aggregate data extracted from journal publications. Rather, the original data on each individual participant are sought from each eligible study. These data typically include characteristics such as age or stage of disease, the intervention or exposure being investigated, and follow-up data on outcomes and events.

“Participant” is used to describe the unit of analysis because most commonly this is an individual person. However, it may apply equally to other units of analysis, such as a school, primary care practice, or hospital in a cluster randomized trial, or an individual body part. What is important is the availability of raw unit-level data rather than aggregate-level data extracted from a report.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement published in 2009,1 which includes a 27-item checklist and flow diagram, was developed principally for systematic reviews and meta-analyses of randomized trials that use aggregate data, generally extracted from published reports. It therefore does not cover some important aspects of the IPD approach to systematic review and meta-analysis, particularly the methods used to obtain, check, and synthesize the IPD, and to handle studies for which IPD were not available. PRISMA-IPD was developed to address these issues.

Methods

PRISMA-IPD was developed based on the methodological framework for guideline development published by the EQUATOR Network.2 The standard PRISMA Statement1 was used as a starting point, and initial work to adapt and build on this was led by a steering group aided by a small project group.

The project group conducted a review of how systematic reviews and IPD meta-analyses are currently reported to update previous evaluations and guidance in this area.3,4 An initial draft adaptation of the standard PRISMA checklist was then prepared by the steering and project groups. This formed the basis of an electronic questionnaire distributed to 95 members of the Cochrane IPD Meta-analysis Methods Group, 88 members of the Society for Research Synthesis Methods, 4 members of both, and 4 additional individuals invited to help develop the extension. Recipients were asked to send the questionnaire to others who might be interested. Links to the survey were placed on the websites of the Cochrane IPD Meta-analysis Methods Group, the Centre for Reviews and Dissemination (CRD), and the Systematic Reviews journal, as well as on the Cochrane Facebook page. The CRD also advertised the survey using Twitter. The survey remained open for 14 days.

The questionnaire sought feedback on the appropriateness of standard PRISMA Statement items to IPD and on suggested changes and additions. It included 4 items with suggested wording changes (1, 3, 6, 18), 8 items with additional proposed elements (11, 14, 15, 16, 17, 19, 20, 25), and 1 item with both (10). Proposed wording changes were rated “appropriate” or “other change required,” and suggested additional components were rated as “not required,” “possible,” “desirable,” or “essential.” Fourteen items had no suggested changes at that stage (2, 4, 5, 7, 8, 9, 12, 13, 21, 22, 23, 24, 26, 27).

A 1-day international workshop was convened in York, United Kingdom, in March 2013. The 26 participants included systematic reviewers with experience in IPD synthesis (n = 21), clinicians (n = 6), methodologists (n = 20), and journal editors (n = 10). Survey feedback was presented, and each PRISMA item was discussed in detail and agreement on inclusion and wording reached by consensus. The checklist prepared during the workshop was then circulated for further comment to participants and to 3 people unable to attend. It continued to be refined iteratively by the steering group and subsequently the wider development group.

Results
Survey Response

Fifty-three responses to the questionnaire were received, a 28% response rate based on direct invitations (numbers of respondents not invited directly are unknown). Of 38 respondents who answered demographic questions, there were 28 systematic reviewers, 9 health care practitioners, 1 policy maker, 27 methodologists, 13 statisticians, and 3 publishers. Twenty-seven reported membership of the Cochrane Collaboration and 11 indicated that they were not members. Twelve reported no practical experience with IPD systematic reviews, whereas 5 had completed 1, 10 had completed between 2 and 5, and 11 had completed more than 5 such reviews.

Respondents supported suggested wording changes for 4 items (>80% scored as appropriate) and required further change for 1 item (31% scored as appropriate). Suggested additions concerning data checking, prespecification, statistical analysis, study variables sought, numbers of participants, IPD-based description of clinical characteristics, reasons for nonprovision of IPD, how studies with and without IPD were analyzed together, and the effect of unavailable trials and missing IPD were supported (>70%). Providing particular data-checking details and production of forest plots for all analyses were not supported (<70%). These results were used only as a starting point for deliberation at the workshop.

Workshop Response

At the workshop 25 checklist items were modified during discussion, much of which centered on striking a balance between being consistent with the standard PRISMA Statement and covering all aspects of reporting pertinent to IPD. Iterative modification and refinement of almost all items by the steering and wider development groups continued until September 2014.

Final PRISMA-IPD Checklist

The Table presents the final PRISMA-IPD checklist adapted and extended from the original PRISMA Statement, as agreed by consensus of those involved in its development. This includes provision for additional information required to describe adequately the IPD approach or where some rewording provides clarity, particularly in the context of IPD. PRISMA-IPD contains 23 items in which the wording has been modified and 3 new items on methods of checking data integrity (A1), on methods of exploring variation (A2), and on reporting any important issues identified as a result of data checking (A3). The other additional item (A4) has been created as a consequence of rearranging items relating to interpretation of results.

To clarify differences, eTable 1 in the Supplement presents the PRISMA and PRISMA-IPD checklists side by side. eTable 2 in the Supplement provides examples from previously published work illustrating how reports may be suitably worded. The Figure presents a modified version of the PRISMA flow diagram, which may also be downloaded from the PRISMA (http://www.prisma-statement.org) and EQUATOR (http://www.equator-network.org) websites, as may the checklist.

Discussion

Systematic reviews and meta-analyses of individual participant data have been recognized as a gold standard approach from the early days of systematic review.5,6 They offer many advantages over analyses that use aggregate data extracted from publications.4,7 These include the potential to avoid bias arising from the absence of unpublished studies and unreported outcomes,7,8 checking and transforming data to common scores or measures and standardizing analyses across studies,7 and the possibility of handling missing data within studies more appropriately.9 Individual-level information enables more flexible and robust analyses than are possible with aggregate study results, including the ability to deal appropriately with time-to-event and longitudinal data. IPD meta-analyses also may enhance evidence synthesis more widely; for example, they may help quantify potentially causal associations from multiple observational studies9 or develop risk prediction models.10,11

With an established history in reviews of interventions in cancer12 and cardiovascular disease,13 the IPD approach is being used increasingly4 and across a broadening range of health care areas. However, it represents a minority of the systematic reviews undertaken,3,4 perhaps owing to the time and resources required to build collaborations of study investigators to share data and agree on analyses to be performed. If IPD becomes more readily available as a result of current initiatives aiming to make provision of clinical trial data for research purposes a legal, regulatory, or ethical requirement,14,15 it is likely that in the future more systematic reviews will access and analyze IPD. This will likely include synthesis of IPD released in controversial areas where transparent, complete, and high-quality reporting is essential.

As with all areas of research, systematic reviews and meta-analyses of IPD could be better reported,16 making it easier for readers to understand, critique, and implement findings. Standard PRISMA guidelines are geared toward systematic reviews based on aggregate data and so lack reference to some important aspects of the IPD approach. The PRISMA-IPD extension was therefore developed to provide a framework for full and transparent reporting of methods used in the collection, checking, and meta-analysis of IPD.

Main Modifications to the PRISMA Checklist
Structured Abstract

Based on the PRISMA extension for abstracts17 tailored to the IPD approach, a structured abstract (item 2) should include important details of methods and results. Although journal format and word limits may make it difficult to include all the outlined information, as much relevant information as possible should be succinctly summarized.

Rationale

It may be useful to explain briefly the benefits that the IPD approach brings to the review, as well as the specific rationale for the research question being addressed (item 3). This may highlight inadequacies of any existing systematic reviews of aggregate data and may be particularly important in areas in which the approach is not yet well established and the target audience is not familiar with its strengths. Providing such information may also be helpful in persuading readers of the robustness and consequent value of findings, which may help encourage their uptake and use in practice, including within clinical guidelines.18 For clarity, it is always helpful for reviews that update or build on previously published reviews to explicitly make the link with prior versions.

Protocols and Registration

Access to IPD both permits more flexible and powerful analysis and provides an opportunity for data manipulation. Thus, the production of, and adherence to, a protocol that includes a detailed analysis plan is perhaps even more important than for a standard systematic review. Deviation from the planned analyses may be necessary and may even improve on what was intended. However, transparency is important, for example in stating which reported analyses were preplanned and which were primary and secondary outcomes. Such statements can be supported by referral to the protocol. Because systematic review protocols are increasingly being registered (eg, in PROSPERO [http://www.crd.york.ac.uk/PROSPERO/]) and published formally, a recommendation to include a citation or link to registration records and formal published protocols has been added (item 5). It also may be helpful to include a copy of the protocol and data request forms as an appendix to the published report.

Eligibility Criteria

Inclusion criteria are generally developed as for a standard systematic review. However, the IPD approach provides an opportunity to use only a subset of the enrolled population within a study. For example, in a review of pediatric research interventions it may be possible to include the IPD from just the children enrolled in a study that recruited both adults and children. Where any such additional inclusion/exclusion criteria are applied at the participant level, they should be reported (item 6). As well as ensuring transparency, doing so helps readers avoid confusion if the number of included participants differs markedly from the number reported in the original study publication.

Identifying Studies and Obtaining Data

Direct contact and collaboration with study investigators, including enlisting their help in identifying eligible studies, is a key feature of systematic reviews and meta-analyses of IPD. Item 7 was therefore extended to include additional means of identifying studies. These can be particularly important for identifying data not published at the time of the systematic review. Item 17 has also been extended to include additional information on seeking IPD from the original studies.

Data Collection, Harmonization, and Checking

Data extraction does not usually apply to those studies for which IPD are obtained. The checklist therefore includes a new element under data collection processes (item 10) to capture how IPD were obtained and managed, and item 11 now includes reporting the methods of standardizing or redefining the IPD received.19 New items on methods of exploring data integrity (item A1) and reporting data integrity (item A3) have also been added, reflecting the importance of data checking and correcting any inaccuracies or errors in the IPD supplied.

Risk of Bias Assessment

Risk of bias assessments for included studies (item 12) and presentation of findings (item 19) have been extended to consider direct investigation of the IPD. For example, there might be less concern about potential bias associated with envelope randomization if checking the IPD shows that treatment allocation to study groups is balanced over time, provides reassurance that envelopes have been used in the planned sequence with none discarded, and that there are no important imbalances in patient characteristics across allocation groups. Conversely, it may be necessary to highlight concerns revealed only by the IPD. For example, if checking IPD randomization sequence reveals an alternating pattern of allocation for a trial that reported apparently sound methods of randomization, this should be noted. Obtaining the full study protocol and direct contact with the participating study investigators also may provide additional information to inform assessment of risk of bias.

Handling Trials for Which IPD Were Unavailable

An important aspect of validity is the completeness and representativeness of the data collected. It is therefore important to provide information on the numbers of studies and participants for which IPD were sought and obtained (item 17), to report whether there is potential risk of bias associated with nonavailability of IPD (items 15 and 22), and to compare results from analyses that include and exclude studies for which IPD were not available (item 23). For the latter, 1-stage and 2-stage meta-analysis models (discussed below) can be used to combine aggregate data (from studies not providing IPD) with the available IPD,20 and both sources of evidence can be distinctly displayed on forest and funnel plots.16 This allows the effect of non-IPD studies on meta-analysis conclusions to be quantified and transparently displayed.

Synthesis Methods

IPD enables more flexible and potentially more powerful statistical analyses than are possible with aggregate data. However, it also creates potential for “data dredging,” whereby reviewers explore numerous outcomes and subgroups to find those that yield interesting results. This made it important to add a recommendation to report all outcomes analyzed and whether these were prespecified (item 13) and to record all subgroup analyses conducted and whether these were prespecified (item A2).

Methods of synthesis are not always reported fully or well.3,4 Item 14 has been extended to list aspects that should be addressed in analyses of clinical trial data. A variety of analytic models can be used including (1) those that first generate estimates of effectiveness (aggregate data) for each study separately and then combine these summary statistics using standard meta-analysis methods (commonly termed a 2-stage approach), and (2) those that estimate the overall meta-analytic effect from all data in all studies simultaneously (commonly termed a 1-stage approach). If a 1-stage modeling approach is used, it is important that the clustering of patients within studies is taken into account and that the data set is not analyzed simplistically as a single “mega trial.”21 Care should be taken to ensure that the model selection process is described adequately and that full specifications are provided. Results should be presented with a nonstatistical audience in mind. For example, coefficients from a 1-stage logistic or Cox regression model relate to a log odds ratio or a log hazard ratio, respectively; converting and reporting these as odds ratios or hazard ratios makes them easier to understand.

It may be helpful to present results as both relative and absolute differences (which depends on relative differences and on baseline risk) between, for example, interventions from randomized trials. A large relative benefit may be of little practical importance if the underlying risk and hence absolute improvement is small. Furthermore, because baseline event rates can differ substantially between different types of individual, the same relative effect may translate to different absolute improvements, even when the relative effect of intervention is the same. It therefore can be helpful to present information on relative and absolute differences according to a series of differing baseline risks.22,23

Forest plots enable readers to examine combined estimates, inconsistency across studies, and the precision of individual studies. In common with the standard PRISMA Statement the display of forest plots for key outcomes is advocated, irrespective of the type of approach to statistical analysis.

Exploration of Effectiveness in Different Participant Types

A major motivation for adopting the IPD approach is the ability to explore between-study heterogeneity and participant-level variation in treatment response. The latter is particularly important, as it allows analyses to explore whether there are any particular types (subgroups) of participants who benefit differentially from the intervention under investigation.24 This is reflected through an additional element in stating objectives (item 4) relating to presentation of subgroup hypotheses, a new item (item A2) to describe methods, and an additional element relating to presenting results (item 21). When reporting such analyses it is vital to state whether there is any clear statistical evidence of a difference in outcome by participant characteristics. As well as being of clinical relevance, any such variation may explain heterogeneity in results between trials.

PRISMA-IPD Amendments Relevant to All Systematic Reviews

Some PRISMA-IPD additions are relevant to all systematic reviews. These include statement of any subgroup hypotheses (item 4); citation of published protocols (item 5); description of how the information to be collected (or extracted) was chosen (item 11); statement of all comparisons and outcomes addressed and whether these were primary outcomes (item 13); statement of whether analyses were prespecified (items 13, 16, and 21); reporting of the number of participants for which data were available (item 17); and description of interventions (item 18). The PRISMA group may wish to consider some of these for inclusion in future versions of the standard PRISMA Statement.

Implementation

PRISMA-IPD is intended to apply to IPD meta-analysis primarily within the context of systematic review of randomized trials. It has been developed largely from experience of undertaking reviews of studies of the effects of health care interventions, where the approach was established in the late 1980s.25,26 Most examples are drawn from this literature, and it is anticipated that the checklist will be used mainly in this context. However, much is also relevant to other areas in which IPD synthesis is gaining popularity, including systematic reviews of diagnostic,27 prognostic,10,28,29 observational,30,31 causal,32 or animal33,34 studies. Although not designed specifically for prospective meta-analyses, in which study investigators decide in advance to formally combine individual-level data from each of their studies in a larger meta-analytic synthesis,35 many checklist items will apply. Because systematic reviews and meta-analyses of IPD are often substantial projects, it may not always be possible to address all items in detail within journal article word limits. In this case, further information should be made available as supplementary material.

The PRISMA-IPD checklist should be largely self-explanatory and represents the minimum amount of information that should be reported to provide a full and transparent account of how the review was conducted. It will sometimes be necessary to include further information not covered by PRISMA-IPD items to deal with nonstandard issues that occurred during the review process and convey nuances of findings. Syntheses of study designs other than randomized trials may require further or different information. Authors and peer reviewers are encouraged to use the checklist to improve reporting and journal editors to include it in their endorsement of PRISMA.

Limitations

Development of the PRISMA-IPD Statement (as for the standard PRISMA Statement) was evidence-based where possible and was otherwise based on opinions gathered from persons with relevant expertise and experience. Response to the survey was limited (53 respondents [28% response rate]). This may reflect it being open for only a short time (necessitated by a fixed workshop date) as well as its specialist focus. However, as its stated purpose was to provide a starting point for discussion at the workshop, this is not considered a major limitation. The checklist has not been formally evaluated prior to proposed implementation. Whether use of PRISMA-IPD will improve reporting quality requires evaluation in future research.

Conclusions

PRISMA-IPD provides guidelines for reporting systematic reviews and meta-analyses of IPD. Future research is needed to determine whether this approach will lead to improved reporting of this type of research.

Back to top
Article Information

Corresponding Author: Lesley A. Stewart, PhD, Centre for Reviews and Dissemination, University of York, York, UK YO10 5DD.

Author Contributions: Dr L. A. Stewart had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: L. A. Stewart, Clarke, Rovers, Riley, G. Stewart, Tierney.

Acquisition, analysis, or interpretation of data: L. A. Stewart, Clarke, Rovers, Riley, Simmonds, G. Stewart, Tierney.

Drafting of the manuscript: L. A. Stewart, Clarke, Rovers, Riley, Simmonds, G. Stewart, Tierney.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Simmonds.

Obtained funding: L. A. Stewart.

Administrative, technical, or material support: Simmonds, G. Stewart.

Study supervision: L. A. Stewart, Clarke, Rovers, Riley, Tierney.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr L. A. Stewart reported that she is employed as Director of the Centre for Reviews and Dissemination at the University of York and is in receipt of research funding to carry out systematic reviews and meta-analyses; has previously received research funding to explore and develop methods for systematic reviews and meta-analysis of IPD; is co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods group; and has previously published on and advocated IPD approaches to systematic review. Dr Clarke reported that he is co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods group and is involved in the conduct of IPD analyses. Dr Rovers reported that she is a co-convenor of the Cochrane Collaboration IPD Meta-analysis Group and has undertaken several IPD reviews. Dr Riley reported that he has conducted and published IPD meta-analyses and related methodological research and guidance on the IPD approach to systematic review and receives funding for these activities. Dr Tierney reported that she is co-convenor of the Cochrane Collaboration IPD Methods Group; has conducted and published IPD meta-analyses, related methodological research, and guidance on the IPD approach to systematic review; and receives funding for these activities. No other disclosures were reported.

Members of the PRISMA-IPD Development Group:Project Group: Mark Simmonds, Gavin Stewart, Lesley Stewart. Steering Group: Mike Clarke, Richard Riley, Maroeska Rovers, Lesley Stewart (chair), Jayne Tierney. PRISMA-IPD Development Group (additional to project and steering groups): Doug Altman (Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom), Lisa Askie (NHMRC Clinical Trials Centre, University of Sydney, Australia), Sarah Burdett (MRC Clinical Trials Unit, UCL, London United Kingdom), Lelia Duley (University of Nottingham, United Kingdom), Jonathan Emberson (Clinical Trial Service Unit & Epidemiological Studies Unit, Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom), Francois Gueffier (Faculty of Medicine, University of Lyon, Lyon, France), Amanda Kerr (Clinical Trial Service Unit & Epidemiological Studies Unit [CTSU], Nuffield Department of Population Health, Oxford University, Oxford, United Kingdom), Karmela Krleža-Jerić (Ottawa Group, IMPACT, and Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia), Toby Lasserson (Cochrane Editorial Unit, Cochrane Collaboration, London, United Kingdom), David Moher (Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada), Karel Moons (Julius Center For Health Sciences and Primary Care, UMC Utrecht, the Netherlands), Bob Phillips (Centre for Reviews and Dissemination, University of York, York, United Kingdom), Pascal Piedbois (Centre Paul Strauss, Strasbourg, France), Jean-Pierre Pignon (Meta-analysis unit, Gustave Roussy, Villejuif, France), Christopher H. Schmid (Department of Biostatistics and Center for Evidence Based Medicine, Brown University School of Public Health, Providence Rhode Island), Catrin Tudur Smith (Department of Biostatistics and North West Hub for Trials Methodology Research, University of Liverpool, Liverpool, United Kingdom), Claire Vale (MRC Clinical Trials Unit, UCL, London, United Kingdom), Emma Veitch (PLOS ONE, Cambridge, United Kingdom), Keith Wheatley (Cancer Research UK Clinical Trials Unit, University of Birmingham, Birmingham, United Kingdom).

Doug Altman is a member of the PRISMA group and author of the PRISMA Statement and is also a member of the EQUATOR Steering Group. David Moher is a member of the PRISMA group and author of the PRISMA Statement, a member of the EQUATOR Steering Group, and is funded through a University Research Chair. Toby Lasserson is an employee of the Cochrane Collaboration and a member of the group developing Cochrane’s own set of standards for conducting and reporting intervention reviews (MECIR) and for its plain language summaries (PLEACS).

Funding/Support: The PRISMA-IPD workshop was funded through a National Institute for Health Research (NIHR) Senior Investigator personal award (Dr L. Stewart). Preparatory work was partially supported by the same award and by the MRC as part of the MRC-National Institute for Health Research Methodology Research Programme (grant ID 88053).

Role of Funders/Sponsors: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The views expressed are those of the authors and not necessarily those of the Cochrane Collaboration, MRC, National Health Service, the NIHR, or the Department of Health.

Additional Contributions: We are grateful to Alison Tonks, MB, ChB (BMJ), and Iain Hrynaszkiewicz, MA (Nature Publishing Group & Palgrave Macmillan), who attended and participated in discussion and debate at the workshop, and to Marc Buyse, ScD (IDDI), who commented on the draft statement. We thank Alison Smith, MSc, and Dean Langan, MSc, for undertaking data extraction for the review of current individual patient data (IPD) publishing practice; Vanda Castle and Jackie Richmond for workshop organization; and John Jackson, BSc, and Vanda Castle for editorial assistance, all of whom are or were members of staff at the Centre for Reviews and Dissemination, University of York. Thanks also to Larysa Rydzewska (MRC Clinical Trials Unit, University College, London) for liaising with the Cochrane IPD Meta-analysis Methods Group, providing editorial assistance, and sourcing example material. None of these individuals received compensation for their contributions.

Correction: This article was corrected online on May 22, 2015, to correct errors in the text and Figure.

References
1.
Moher  D, Liberati  A, Tetzlaff  J, Altman  DG; PRISMA Group.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.  PLoS Med. 2009;6(7):e1000097.PubMedGoogle ScholarCrossref
2.
Moher  D, Schulz  KF, Simera  I, Altman  DG.  Guidance for developers of health research reporting guidelines.  PLoS Med. 2010;7(2):e1000217.PubMedGoogle ScholarCrossref
3.
Simmonds  MC, Higgins  JP, Stewart  LA, Tierney  JF, Clarke  MJ, Thompson  SG.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice.  Clin Trials. 2005;2(3):209-217.PubMedGoogle ScholarCrossref
4.
Riley  RD, Lambert  PC, Abo-Zaid  G.  Meta-analysis of individual participant data: rationale, conduct, and reporting.  BMJ. 2010;340:c221.PubMedGoogle ScholarCrossref
5.
Chalmers  I; Cochrane Collaboration.  The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care.  Ann N Y Acad Sci. 1993;703:156-163.PubMedGoogle ScholarCrossref
6.
Stewart  LA, Clarke  MJ; Cochrane Working Group.  Practical methodology of meta-analyses (overviews) using updated individual patient data.  Stat Med. 1995;14(19):2057-2079.PubMedGoogle ScholarCrossref
7.
Stewart  LA, Tierney  JF.  To IPD or not to IPD? advantages and disadvantages of systematic reviews using individual patient data.  Eval Health Prof. 2002;25(1):76-97.PubMedGoogle ScholarCrossref
8.
Stewart  L, Tierney  J, Burdett  S. Do systematic reviews based on individual patient data offer a means of circumventing biases associated with trial publications? In: Rothstein  H, Sutton  A, Borenstein  M, eds.  Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, United Kingdom: John Wiley & Sons; 2005:261-286.
9.
Burgess  S, White  IR, Resche-Rigon  M, Wood  AM.  Combining multiple imputation and meta-analysis with individual participant data.  Stat Med. 2013;32(26):4499-4514.PubMedGoogle ScholarCrossref
10.
Debray  TP, Moons  KG, Ahmed  I, Koffijberg  H, Riley  RD.  A framework for developing, implementing, and evaluating clinical prediction models in an individual patient data meta-analysis.  Stat Med. 2013;32(18):3158-3180.PubMedGoogle ScholarCrossref
11.
Phillips  RS, Sutton  AJ, Riley  RD, Chisholm  JC, Picton  SV, Stewart  LA; PICNICC Collaboration.  Predicting infectious complications in neutropenic children and young people with cancer (IPD protocol).  Syst Rev. 2012;1:8.PubMedGoogle ScholarCrossref
12.
Clarke  M, Stewart  L, Pignon  JP, Bijnens  L.  Individual patient data meta-analysis in cancer.  Br J Cancer. 1998;77(11):2036-2044.PubMedGoogle ScholarCrossref
13.
Selker  HP, Griffith  JL, Beshansky  JR,  et al.  Patient-specific predictions of outcomes in myocardial infarction for real-time emergency use: a thrombolytic predictive instrument.  Ann Intern Med. 1997;127(7):538-556.PubMedGoogle ScholarCrossref
14.
Krumholz  HM.  Why data sharing should be the expected norm.  BMJ. 2015;350:h599.PubMedGoogle ScholarCrossref
15.
Lo  B.  Sharing clinical trial data: maximizing benefits, minimizing risk.  JAMA. 2015;313(8):793-794.PubMedGoogle ScholarCrossref
16.
Ahmed  I, Sutton  AJ, Riley  RD.  Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey.  BMJ. 2012;344:d7762.PubMedGoogle ScholarCrossref
17.
Beller  EM, Glasziou  PP, Altman  DG,  et al; PRISMA for Abstracts Group.  PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts.  PLoS Med. 2013;10(4):e1001419.PubMedGoogle ScholarCrossref
18.
Vale  CL, Rydzewska  LHM, Rovers  MM, Emberson  JR, Gueyffier  F, Stewart  LA; Cochrane IPD Meta-analysis Methods Group.  Uptake of systematic reviews and meta-analyses based on individual participant data in clinical practice guidelines: descriptive study.  BMJ. 2015;350:h1088.PubMedGoogle ScholarCrossref
19.
Schmid  CH, Landa  M, Jafar  TH,  et al; Angiotensin-Converting Enzyme Inhibition in Progressive Renal Disease (AIPRD) Study Group.  Constructing a database of individual clinical trials for longitudinal analysis.  Control Clin Trials. 2003;24(3):324-340.PubMedGoogle ScholarCrossref
20.
Riley  RD, Lambert  PC, Staessen  JA,  et al.  Meta-analysis of continuous outcomes combining individual patient data and aggregate data.  Stat Med. 2008;27(11):1870-1893.PubMedGoogle ScholarCrossref
21.
Abo-Zaid  G, Guo  B, Deeks  JJ,  et al.  Individual participant data meta-analyses should not ignore clustering.  J Clin Epidemiol. 2013;66(8):865-873.PubMedGoogle ScholarCrossref
22.
Non–small Cell Lung Cancer Collaborative Group.  Chemotherapy in non–small cell lung cancer: a meta-analysis using updated data on individual patients from 52 randomised clinical trials.  BMJ. 1995;311(7010):899-909.PubMedGoogle ScholarCrossref
23.
Steyerberg  EW, Moons  KG, van der Windt  DA,  et al; PROGRESS Group.  Prognosis Research Strategy (PROGRESS) 3: prognostic model research.  PLoS Med. 2013;10(2):e1001381.PubMedGoogle ScholarCrossref
24.
Hingorani  AD, Windt  DA, Riley  RD,  et al; PROGRESS Group.  Prognosis research strategy (PROGRESS) 4: stratified medicine research.  BMJ. 2013;346:e5793.PubMedGoogle ScholarCrossref
25.
Early Breast Cancer Trialists' Collaborative Group.  Effects of adjuvant tamoxifen and of cytotoxic therapy on mortality in early breast cancer: an overview of 61 randomized trials among 28,896 women.  N Engl J Med. 1988;319(26):1681-1692.PubMedGoogle ScholarCrossref
26.
Advanced Ovarian Cancer Trialists’ Group.  Chemotherapy in advanced ovarian cancer: an overview of randomised clinical trials.  BMJ. 1991;303(6807):884-893.PubMedGoogle ScholarCrossref
27.
Mant  J, Doust  J, Roalfe  A,  et al.  Systematic review and individual patient data meta-analysis of diagnosis of heart failure, with modelling of implications of different diagnostic strategies in primary care.  Health Technol Assess. 2009;13(32):1-207.PubMedGoogle ScholarCrossref
28.
Abo-Zaid  G, Sauerbrei  W, Riley  RD.  Individual participant data meta-analysis of prognostic factor studies: state of the art?  BMC Med Res Methodol. 2012;12:56.PubMedGoogle ScholarCrossref
29.
Ahmed  I, Debray  TP, Moons  KG, Riley  RD.  Developing and validating risk prediction models in an individual participant data meta-analysis.  BMC Med Res Methodol. 2014;14:3.PubMedGoogle ScholarCrossref
30.
Danesh  J, Lewington  S, Thompson  SG,  et al; Fibrinogen Studies Collaboration.  Plasma fibrinogen level and the risk of major cardiovascular diseases and nonvascular mortality: an individual participant meta-analysis.  JAMA. 2005;294(14):1799-1809.PubMedGoogle Scholar
31.
van der A  DL, Rovers  MM, Grobbee  DE,  et al.  Mutations in the HFE gene and cardiovascular disease risk: an individual patient data meta-analysis of 53 880 subjects.  Circ Cardiovasc Genet. 2008;1(1):43-50.PubMedGoogle ScholarCrossref
32.
Burgess  S, Thompson  SG; Genetics Collaboration CC.  Methods for meta-analysis of individual participant data from mendelian randomisation studies with binary outcomes [published online June 19, 2012].  Stat Methods Med Res. doi:10.1177/0962280212451882.Google Scholar
33.
Bath  PM, Gray  LJ, Bath  AJ, Buchan  A, Miyata  T, Green  AR; NXY-059 Efficacy Meta-analysis in Individual Animals With Stroke Investigators.  Effects of NXY-059 in experimental stroke: an individual animal meta-analysis.  Br J Pharmacol. 2009;157(7):1157-1171.PubMedGoogle ScholarCrossref
34.
Schmidt  AF, Nielen  M, Klungel  OH,  et al; V.S.S.O. Investigators.  Prognostic factors of early metastasis and mortality in dogs with appendicular osteosarcoma after receiving surgery: an individual patient data meta-analysis.  Prev Vet Med. 2013;112(3-4):414-422.PubMedGoogle ScholarCrossref
35.
Berlin  JA, Ghersi  D. Preventing publication bias: registries and prospective meta-analysis. In: Rothstein  HR, Sutton  AJ, Borenstein  M, eds.  Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. Chichester, United Kingdom: John Wiley & Sons; 2005:35-48.
×