Strengthening the Reporting of Observational Studies in Epidemiology Using Mendelian Randomization: The STROBE-MR Statement

Key Points

Question  What information should be included in reports of mendelian randomization (MR) studies?

Findings  An international expert committee, informed by the methodological framework for guideline development of the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network and using a consensus process, developed the STROBE-MR checklist, which includes 20 main items and 30 subitems for reporting the results of MR studies.

Meaning  Use of the STROBE-MR reporting guideline for MR studies could facilitate evaluation by editors, peer reviewers, researchers, clinicians, and other readers, and enhance the interpretation of their results.

Importance  Mendelian randomization (MR) studies use genetic variation associated with modifiable exposures to assess their possible causal relationship with outcomes and aim to reduce potential bias from confounding and reverse causation.

Objective  To develop the STROBE-MR Statement as a stand-alone extension to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guideline for the reporting of MR studies.

Design, Setting, and Participants  The development of the STROBE-MR Statement followed the Enhancing the Quality and Transparency of Health Research (EQUATOR) framework guidance and used the STROBE Statement as a starting point to draft a checklist tailored to MR studies. The project was initiated in 2018 by reviewing the literature on the reporting of instrumental variable and MR studies. A group of 17 experts, including MR methodologists, MR study design users, developers of previous reporting guidelines, and journal editors, participated in a workshop in May 2019 to define the scope of the Statement and draft the checklist. The draft checklist was published as a preprint in July 2019 and discussed on the preprint platform, in social media, and at the 4th Mendelian Randomization Conference. The checklist was then revised based on comments, further refined through 2020, and finalized in July 2021.

Findings  The STROBE-MR checklist is organized into 6 sections (Title and Abstract, Introduction, Methods, Results, Discussion, and Other Information) and includes 20 main items and 30 subitems. It covers both 1-sample and 2-sample MR studies that assess 1 or multiple exposures and outcomes, and addresses MR studies that follow a genome-wide association study and are reported in the same article. The checklist asks authors to justify why MR is a helpful method to address the study question and state prespecified causal hypotheses. The measurement, quality, and selection of genetic variants must be described and attempts to assess validity of MR-specific assumptions should be well reported. An item on data sharing includes reporting when the data and statistical code required to replicate the analyses can be accessed.

Conclusions and Relevance  STROBE-MR provides guidelines for reporting MR studies. Improved reporting of these studies could facilitate their evaluation by editors, peer reviewers, researchers, clinicians, and other readers, and enhance the interpretation of their results.

Mendelian randomization is a method that uses genetic variation associated with modifiable exposures (or risk factors) to assess their possible causal relationship with outcomes, and aims to reduce bias from confounding, including reverse causation, in epidemiological studies.^1-3 Mendelian randomization studies use genetic variants, typically single-nucleotide polymorphisms (SNPs), that are reliably associated with exposures of interest but do not vary with the correlated lifestyle or socioeconomic factors that may confound conventional observational associations. In recent decades, the number of published mendelian randomization studies has increased rapidly, from 1 report in 2003 to more than 800 articles in 2020. Recent studies have addressed topics as diverse as IL-6 receptor inhibition and prognosis of COVID-19,⁴ cannabis use and risk of schizophrenia,⁵ and association of education and intelligence with risk of Alzheimer disease.⁶

The use of genetic variants for testing hypotheses of causal inference regarding the potential relationship between modifiable exposures and health outcomes depends on the gene-environment equivalence assumption, that modification of the exposure by genetic variation will have the same downstream influence on the outcome as if the exposure were modified through an environmental intervention (including lifestyle and pharmaceutical factors).⁷ As noted by Emdin et al,² “[m]endelian randomization rests on 3 main assumptions: (1) the genetic variant is associated with the risk factor; (2) the genetic variant is not associated with confounders; and (3) the genetic variant influences the outcome only through the risk factor.”

Mendelian randomization can be applied within the instrumental variable framework, if specific assumptions are met. Assumptions are violated if, for example, there is horizontal pleiotropy, in which the genetic variant influences the outcome independently of the risk factor. Also, the variant could be in linkage disequilibrium with another variant that is associated with the outcome or could vary by ancestry.^8,9 Misleading inferences can be generated in mendelian randomization analyses if the relationships relating the exposure to the outcome are misspecified, for example, if the genotype directly affects the outcome, which then affects the putative exposure (ie, reverse causation).

Despite increasing relevance and popularity of mendelian randomization studies, their reporting is often incomplete, which may limit the credibility of potential causal inference.^10-12 Reporting guidelines, such as the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline,¹³ can improve reporting quality.¹⁴ STROBE-MR was developed as a guideline for reporting mendelian randomization studies. It is accompanied by an Explanation and Elaboration (E&E) document,¹⁵ which gives the rationale for each item in the checklist and examples of good reporting. This Special Communication describes the development of the STROBE-MR Statement and includes the checklist of recommended items for inclusion in reports of mendelian randomization studies.

With increasing evidence that reporting quality of many mendelian randomization studies is inadequate,^10-12 the core group (V.W.S., R.C.R., G.D.S., M.E., J.B.R.) established the STROBE-MR project in 2018. Development of STROBE-MR was informed by the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network’s methodological framework for guideline development.¹⁴ First, the literature on the reporting of instrumental variable and mendelian randomization studies was reviewed to identify deficiencies in reporting and previous guidance. Keywords in the article title (instrumental variable OR mendelian randomization) were combined with reporting in the title or abstract. The search was performed in the PubMed database and updated several times during the process of developing the guidance, with the last update on July 8, 2021. Additional literature searches were conducted to identify methodological articles or examples of good reporting.

Second, funding was obtained for the guideline initiative and, specifically, for a face-to-face meeting. A list of potential participants was developed based on the networks of the project’s core group and literature searches. Experts in various fields were invited, ranging from mendelian randomization methodologists and authors of previous reporting guidelines to frequent mendelian randomization study design users and scientific journal editors. The list of the 17 meeting participants is available on Open Science Framework (https://osf.io/fpb8g/).¹⁶

In preparation for the meeting, the core group generated a draft checklist with items that should be reported in mendelian randomization studies. Built on STROBE, the list focused on mendelian randomization–specific assumptions, methods of assessment of their potential violations, and the reporting of data and data sources used in mendelian randomization studies. The draft checklist was tested on a random sample of original articles published in 2018, reporting on mendelian randomization study. PubMed was searched for eligible articles using the MESH term mendelian randomization analysis and free text terms mendelian randomization and mendelian randomisation in the title or abstract.

Of 404 potentially eligible articles (as of April 1, 2019), 20 were randomly selected and scanned to determine whether they contained original research. If not, additional articles were randomly selected until 20 reports of mendelian randomization studies were assessed (of which 10 used a 1-sample and 10 a 2-sample mendelian randomization study design). The sample size of 20 was a pragmatic choice of number of publications that could be reviewed in detail before the meeting in May 2019. The purpose of this pilot review was to better understand the clarity of items on the first checklist draft and to support the literature on reporting deficiencies. It was not intended to be a systematic review. The draft checklist was sent to meeting participants on May 1, 2019, along with the meeting agenda, the EQUATOR guidance on developing reporting guidelines,¹⁴ the STROBE Statement¹³ and E&E document,¹⁷ and other background articles. A formal Delphi process was not performed.

The literature review and pilot study results were presented at a 2-day face-to-face meeting, which took place May 16-17, 2019, at the Medical Research Council Integrative Epidemiology Unit at the University of Bristol. The meeting described the rationale and steps of developing reporting guidelines and included sessions on the empirical evidence on the reporting of instrumental variable and mendelian randomization studies, discussions of the draft checklist, and a session on the publication of STROBE-MR. Selected items were discussed in more detail and a meeting participant was assigned to every item according to expertise and availability. These “item leaders” were tasked with finalizing the wording and covering the item in the E&E document.¹⁵ The agenda of the meeting and other meeting materials are available on Open Science Framework.¹⁶

The draft checklist was published as a preprint in July 2019¹⁸ and debated on the preprint platform, Twitter, and at the 4th Mendelian Randomization Conference.¹⁹ As of July 2021, the preprint had been viewed more than 5000 times and downloaded more than 3500 times.¹⁸ All the comments received at the Mendelian Randomization Conference or obtained from the preprint platform and tweets were subsequently discussed in the core team’s video conferences during 2019 and 2020, and implemented in the checklist, as appropriate.

In parallel, the group developed the E&E document. The item leaders wrote a paragraph explaining the rationale for each of their items and provided examples of good reporting for inclusion in the E&E document.¹⁵ A glossary of commonly used terms in mendelian randomization studies was also prepared, drawing on the work of Davies et al³ and Wade et al.²⁰ Supplementary educational material was included relating to the key assumptions underlying mendelian randomization studies, their assessment and falsification, and the interpretation of potential causal effect estimates. An abridged version of the STROBE-MR glossary is included in the present article (Table 1).

The group agreed that the guidelines should apply to studies that use properties of germline genetic variation to strengthen potential causal inference of modifiable exposures on outcomes. The guidelines are tailored to the majority of such studies that use an instrumental variable framework. They cover both 1-sample and 2-sample mendelian randomization studies that assess 1 or multiple exposures and outcomes and mendelian randomization studies that follow a genome-wide association study and are reported in the same article. For mendelian randomization studies that do not use instrumental variable estimation (eg, some studies of gene-by-environment interaction²¹), some items of the STROBE-MR checklist will not be applicable. The STROBE-MR guidelines do not apply to genome-wide association studies, which are covered by STREGA²² (Strengthening the Reporting of Genetic Association Studies), sequencing studies, expression studies, or the traditional observational epidemiology studies covered by STROBE.¹³

The literature review identified only a few articles that specifically addressed the reporting of instrumental variable or mendelian randomization studies. Most articles were reviewed before the May 2019 meeting^10-12,23,24 and discussed at the meeting. Two more recent articles published after the meeting were also reviewed.^25,26 Systematic reviews of reporting quality in mendelian randomization studies indicate that many reports of mendelian randomization studies do not clearly state or examine the various assumptions of mendelian randomization methods and report insufficient details on the data sources.^10-12 Testing the draft checklist on the sample of 20 mendelian randomization articles published in 2018 confirmed these deficiencies. For example, none of the 10 2-sample mendelian randomization studies described the underlying exposure and outcome study populations in detail, compared them, or discussed sample overlap. Only 10 studies (50%) reported efforts to test and correct for possible pleiotropy. Details on the imputation of missing genetic data or minor allele frequency cutoffs were missing in 11 of the 20 articles reviewed (55%). Information on the strength of genetic instrument was reported in 10 (50%) of the 20 studies. Ten (50%) of the surveyed publications lacked a discussion of clinical or public health relevance, and 13 (65%) did not address the generalizability of the study results. The pilot testing of the draft checklist informed discussions on revising and refining the STROBE-MR checklist and helped assess clarity of the items. The presentation on the review made at the meeting is available on Open Science Framework.¹⁶

The STROBE-MR checklist consists of 20 items (Table 2) that should be addressed when reporting a mendelian randomization study. Similar to the STROBE checklist,¹³ the items are grouped into sections Title and Abstract (item 1), Introduction (items 2-3), Methods (items 4-9), Results (items 10-13), Discussion (items 14-17), and Other Information (items 18-20). Some items contain several subitems related to issues within the same topic. In using the checklist, authors should address all items and subitems, even if some information will have to be reported in the supplementary materials due to space restrictions.

Compared with the original STROBE checklist,¹³ which included 22 items and 18 subitems, STROBE-MR has 2 fewer items (20) but 12 more subitems (30). Only 1 subitem (No. 6d) remained unchanged (Table 2). All other items and subitems were modified to address requirements specific to reporting of mendelian randomization studies. Briefly, as suggested by Hernán,²⁷ in the Introduction the authors should address whether potential causality between exposure and outcome is plausible, justify why mendelian randomization helps to address the question, and describe the causal hypotheses. In the Methods section, investigators should describe the setting, participants, measurement, quality control and selection of genetic variants, and the diagnostic criteria for the outcome of interest for each data source used. Authors should state the 3 core instrumental variable assumptions for the main analysis (relevance, independence, and exclusion restriction; see Table 1), state assumptions for any additional or sensitivity analysis, and provide a detailed description of the statistical methods and statistics used.

In the Results section, authors should summarize the number of individuals at each stage (eg, by using a flow diagram) and the phenotypic exposures and outcomes. The justification for similarity of the genetic variant–exposure associations between the exposure and outcome samples and the overlapping number of individuals in both samples should be reported for 2-sample mendelian randomization studies. Reporting of the main results should include the associations between genetic variant(s) and exposure and between the genetic variant(s) and outcome, as well as the mendelian randomization estimates of the relationship between the exposure and outcome. STROBE-MR gives much emphasis on the transparent reporting of assessments of the validity of the assumptions and sensitivity analyses to assess the robustness of the main results to violations of the assumptions.

In the Discussion section, after summarizing the main results, authors should discuss limitations of the study, focusing on the validity of the instrumental variable assumptions, other sources of potential bias, and imprecision. They should provide a cautious overall interpretation of results and discuss the underlying biological mechanisms that could drive a potential causal relationship between the investigated exposure and the outcome. The clinical or public health policy relevance should be addressed. Under Other Information, authors should provide information to allow others access to the data used and the statistical code needed to replicate the analyses.

STROBE-MR was developed to guide authors in reporting mendelian randomization studies, supporting editors and reviewers when considering such studies for publication, and helping readers when critically appraising published articles to decide whether the results are valid and useful. The STROBE checklist was used as the point of departure, thoroughly modifying and adapting it to mendelian randomization studies through an open process in accordance with the guidance for developers of reporting guidelines.¹⁴ The relevant empirical evidence on the reporting of mendelian randomization studies was reviewed, and the first draft of the STROBE-MR checklist was piloted on recently published mendelian randomization studies, with consecutive drafts subjected to an extensive iterative process of consultation. Thus, the checklist presented herein is based on input from a large number of individuals with diverse backgrounds and perspectives. The comprehensive E&E document,¹⁵ which is intended for use alongside the checklist, also benefited greatly from this consultation process.

The publication of the STROBE-MR Statement, together with the E&E document,¹⁵ is a first step toward implementing these reporting guidelines. Next steps include encouraging journals to endorse and support adherence to this guideline, for example, by making materials available on the EQUATOR Network website.²⁸ Furthermore, this group will collaborate with researchers to translate the reporting guideline into different languages; the STROBE Statement has been translated from English into Chinese, Spanish, German, Italian, Japanese, Persian, Portuguese, and Greek.²⁸

Mendelian randomization is an active and constantly developing field, and these guidelines will therefore likely require periodic updates. Specifically, it is anticipated that increases in data availability, including from novel global sequencing and genome-wide genotyping, within-family designs, and efforts to link health care databases, may require updates to these guidelines. As methods continue to be developed to address new or existing challenges, the guidelines will also be updated accordingly. This group will continue to monitor the literature to help maintain the guidance, particularly the checklist.

The STROBE-MR Statement should not be interpreted as an attempt to prescribe reporting mendelian randomization studies in a rigid format that codifies style, methods, or terminology. The intention is solely to provide guidance on how to report mendelian randomization research clearly and comprehensively. The checklist items should be addressed in sufficient detail and with clarity somewhere in an article, but the order and format for presenting information depend on author preferences and journal style. Moreover, reporting guidelines are not recommendations for designing or conducting studies, although they may contribute to improving methodology.¹⁴ While clarity of reporting is a prerequisite to evaluation, the checklist is not an instrument to evaluate mendelian randomization research quality, and it should not be used for this purpose.²⁹

This statement has several limitations. First, the statement attempts to provide comprehensive reporting guidelines and describe their generation, but does not describe all methods that can be used to assess all of the assumptions that are required within a mendelian randomization study. Nevertheless, the major assumptions and commonly used methods to assess their validity have been described. Second, as with any consensus document, feedback from the community has been sought and incorporated, and future iterations of this document will be updated to incorporate further feedback from individuals interested in undertaking and interpreting mendelian randomization studies. Thus, the process of properly describing and reporting mendelian randomization studies is an evolving field and will be updated as appropriate. Third, a systematic review has not been undertaken to better understand reporting deficiencies in mendelian randomization studies.

STROBE-MR provides guidelines for reporting mendelian randomization studies. Improved reporting of these studies could facilitate their evaluation by editors, peer reviewers, researchers, clinicians, and other readers, and enhance the interpretation of their results.

Corresponding Author: J. Brent Richards, MD, Jewish General Hospital, 3755 Côte-Ste-Catherine Rd, Ste H-413 Montréal, QC H3T 1E2, Canada (contact@strobe-mr.org).

Accepted for Publication: September 24, 2021.

Author Contributions: Drs Skrivankova and Egger had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Davey Smith, Egger, and Richards contributed equally to this article.

Concept and design: Skrivankova, Richmond, Yarmolinsky, Swanson, VanderWeele, Timpson, Dimou, Langenberg, Golub, Tybjaerg-Hansen, Davey Smith, Egger, Richards.

Acquisition, analysis, or interpretation of data: Skrivankova, Woolf, Davies, Swanson, Higgins, Dimou, Loder, Gallo, Egger, Richards.

Drafting of the manuscript: Skrivankova, Woolf, Yarmolinsky, VanderWeele, Timpson, Dimou, Langenberg, Davey Smith, Egger, Richards.

Critical revision of the manuscript for important intellectual content: Skrivankova, Richmond, Woolf, Yarmolinsky, Davies, Swanson, VanderWeele, Higgins, Dimou, Langenberg, Golub, Loder, Gallo, Tybjaerg-Hansen, Egger, Richards.

Statistical analysis: Skrivankova.
Obtained funding: Davey Smith, Egger.

Administrative, technical, or material support: Skrivankova, Richmond, Woolf, Yarmolinsky, Davey Smith, Egger, Richards.

Supervision: Davey Smith, Egger, Richards.

Conflict of Interests Disclosures: Dr VanderWeele reported receiving grants from the National Cancer Institute (R01CA222147) during the conduct of the study. Dr Gallo reported reimbursement of travel expenses from the University of Bristol during the conduct of the study. Dr Tybjaerg-Hansen reported consultancies or talks for Akcea, AstraZeneca, Draupnir Bio, Novartis, Sanofi, Silence Therapeutics, and Regeneron. Dr Egger reported receiving grants from the Swiss National Science Foundation (189498) during the conduct of the study. Dr Egger convened the STROBE group and is a member of the CONSORT group. Dr Richards reported receiving personal fees from GlaxoSmithKline and Deerfield Capital and grants from GlaxoSmithKline, Eli Lilly, and Biogen outside the submitted work. No other disclosures were reported.

Funding/Support: The STROBE-MR meeting was funded through a Swiss National Science Foundation personal award (Dr Egger, grant No. 17481). Preparatory work was supported by the same award and by the Medical Research Council Integrative Epidemiology Unit (MC_UU_00011/1), University of Bristol, Bristol, United Kingdom, and the Institute of Social and Preventive Medicine, University of Bern, Switzerland.

Role of Funders/Sponsors: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr Golub, JAMA Deputy Editor, was not involved in the evaluation or review of this manuscript, or in the decision regarding publication. Where authors are identified as personnel of the International Agency for Research on Cancer/World Health Organization, the authors alone are responsible for the views expressed in this article and they do not necessarily represent the decisions, policy, or views of the International Agency for Research on Cancer/World Health Organization. The views expressed in this article are those of the authors and do not necessarily represent those of the Swiss National Science Foundation, National Health Service, the National Institute for Health Research, or the UK Department of Health and Social Care.

Additional Contributions: We are grateful to all the colleagues who commented on previous versions of the STROBE-MR checklist. We are also grateful to the Medical Research Council Integrative Epidemiology Unit within the Bristol Medical School, University of Bristol, United Kingdom, for hosting the May 2019 workshop.