[Skip to Navigation]
Sign In
Table 1.  Characteristics of 197 Randomized Clinical Trials That Received Ethical Approval in 2012 and Were Registered in More Than 1 Clinical Trial Registry
Characteristics of 197 Randomized Clinical Trials That Received Ethical Approval in 2012 and Were Registered in More Than 1 Clinical Trial Registry
Table 2.  Variables, Results, and Links to Published Articles Reported in the Clinical Trial Registries
Variables, Results, and Links to Published Articles Reported in the Clinical Trial Registries
Table 3.  Reliability of Clinical Trial Registries Measured as Agreement for Specific Trial Characteristics Across Different Trial Registries
Reliability of Clinical Trial Registries Measured as Agreement for Specific Trial Characteristics Across Different Trial Registries
Table 4.  Analysis Stratified by Sponsor Assessing the Reliability of Clinical Trial Registries
Analysis Stratified by Sponsor Assessing the Reliability of Clinical Trial Registries
Table 5.  Agreement Between Information in the Registry and Information Reported in the Published Trials Across Registries
Agreement Between Information in the Registry and Information Reported in the Published Trials Across Registries
1.
Burns  PB, Rohrich  RJ, Chung  KC.  The levels of evidence and their role in evidence-based medicine.   Plast Reconstr Surg. 2011;128(1):305-310. doi:10.1097/PRS.0b013e318219c171 PubMedGoogle ScholarCrossref
2.
Murad  MH, Asi  N, Alsawas  M, Alahdab  F.  New evidence pyramid.   Evid Based Med. 2016;21(4):125-127. doi:10.1136/ebmed-2016-110401 PubMedGoogle ScholarCrossref
3.
Dickersin  K, Rennie  D.  Registering clinical trials.   JAMA. 2003;290(4):516-523. doi:10.1001/jama.290.4.516 PubMedGoogle ScholarCrossref
4.
De Angelis  CD, Drazen  JM, Frizelle  FA,  et al; International Committee of Medical Journal Editors.  Is this clinical trial fully registered? a statement from the International Committee of Medical Journal Editors.   Lancet. 2005;365(9474):1827-1829. doi:10.1016/S0140-6736(05)66588-9 PubMedGoogle ScholarCrossref
5.
Sim  I, Chan  AW, Gülmezoglu  AM, Evans  T, Pang  T.  Clinical trial registration: transparency is the watchword.   Lancet. 2006;367(9523):1631-1633. doi:10.1016/S0140-6736(06)68708-4 PubMedGoogle ScholarCrossref
6.
Mayo-Wilson  E, Li  T, Fusco  N,  et al.  Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy.   J Clin Epidemiol. 2017;91:95-110. doi:10.1016/j.jclinepi.2017.07.014 PubMedGoogle ScholarCrossref
7.
Chan  AW, Pello  A, Kitchen  J,  et al.  Association of trial registration with reporting of primary outcomes in protocols and publications.   JAMA. 2017;318(17):1709-1711. doi:10.1001/jama.2017.13001 PubMedGoogle ScholarCrossref
8.
Goldacre  B, Drysdale  H, Dale  A,  et al.  COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time.   Trials. 2019;20(1):118. doi:10.1186/s13063-019-3173-2 PubMedGoogle ScholarCrossref
9.
Chan  AW, Hróbjartsson  A, Haahr  MT, Gøtzsche  PC, Altman  DG.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.   JAMA. 2004;291(20):2457-2465. doi:10.1001/jama.291.20.2457 PubMedGoogle ScholarCrossref
10.
Nguyen  VT, Rivière  P, Ripoll  P,  et al; COVID-NMA Consortium Team.  Research response to coronavirus disease 2019 needed better coordination and collaboration: a living mapping of registered trials.   J Clin Epidemiol. 2021;130:107-116;130:107-116. doi:10.1016/j.jclinepi.2020.10.010 PubMedGoogle ScholarCrossref
11.
Kouzy  R, Abi Jaoude  J, Garcia  CJ, El Alam  MB, Taniguchi  CM, Ludmir  EB.  Characteristics of the multiplicity of randomized clinical trials for coronavirus disease 2019 launched during the pandemic.   JAMA Netw Open. 2020;3(7):e2015100. doi:10.1001/jamanetworkopen.2020.15100 PubMedGoogle Scholar
12.
Janiaud  P, Axfors  C, Van’t Hooft  J,  et al.  The worldwide clinical trial research response to the COVID-19 pandemic—the first 100 days.   F1000Res. 2020;9:1193. doi:10.12688/f1000research.26707.1 PubMedGoogle ScholarCrossref
13.
Heale  R, Twycross  A.  Validity and reliability in quantitative studies.   Evid Based Nurs. 2015;18(3):66-67. doi:10.1136/eb-2015-102129 PubMedGoogle ScholarCrossref
14.
Gryaznov  D, Odutayo  A, von Niederhäusern  B,  et al.  Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to Spirit Recommendations (ASPIRE) study and associated projects.   Trials. 2020;21(1):896. doi:10.1186/s13063-020-04808-y PubMedGoogle ScholarCrossref
15.
Page  MJ, Moher  D, Bossuyt  PM,  et al.  PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews.   BMJ. 2021;372(n160):n160. doi:10.1136/bmj.n160 PubMedGoogle Scholar
16.
World Health Organization. WHO Trial Registration Data Set (version 1.3.1). Accessed November 13, 2020. https://www.who.int/ictrp/network/trds/en/
17.
World Health Organization. What registries and registrants can do to facilitate unambiguous identification. Accessed April 19, 2021. https://www.who.int/clinical-trials-registry-platform/unambiguous-trial-identification/what-registries-and-registrants-can-do-to-facilitate-unambiguous-identification
18.
World Health Organization. International Standards for Clinical Trial Registries: the registration of all interventional trials is a scientific, ethical and moral responsibility, version 3.0. Accessed May 27, 2021. https://apps.who.int/iris/handle/10665/274994
19.
Hartung  DM, Zarin  DA, Guise  JM, McDonagh  M, Paynter  R, Helfand  M.  Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications.   Ann Intern Med. 2014;160(7):477-483. doi:10.7326/M13-0480 PubMedGoogle ScholarCrossref
20.
Riveros  C, Dechartres  A, Perrodeau  E, Haneef  R, Boutron  I, Ravaud  P.  Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals.   PLoS Med. 2013;10(12):e1001566. doi:10.1371/journal.pmed.1001566 PubMedGoogle Scholar
21.
Kasenda  B, von Elm  E, You  J,  et al.  Prevalence, characteristics, and publication of discontinued randomized trials.   JAMA. 2014;311(10):1045-1051. doi:10.1001/jama.2014.1361 PubMedGoogle ScholarCrossref
22.
Jin  Y, Sanger  N, Shams  I,  et al.  Does the medical literature remain inadequately described despite having reporting guidelines for 21 years?—a systematic review of reviews: an update.   J Multidiscip Healthc. 2018;11:495-510. doi:10.2147/JMDH.S155103 PubMedGoogle ScholarCrossref
23.
Viergever  RF, Karam  G, Reis  A, Ghersi  D.  The quality of registration of clinical trials: still a problem.   PLoS One. 2014;9(1):e84727. doi:10.1371/journal.pone.0084727 PubMedGoogle Scholar
24.
Tse  T, Fain  KM, Zarin  DA.  How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider.   BMJ. 2018;361:k1452. doi:10.1136/bmj.k1452 PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Statistics and Research Methods
    November 1, 2021

    Reliability of Trial Information Across Registries for Trials With Multiple Registrations: A Systematic Review

    Author Affiliations
    • 1Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
    • 2Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
    • 3Clinical Trial Unit, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
    • 4Swiss Tropical and Public Health Institute, Basel, Switzerland
    • 5Department of Neurosurgery and Department of Biomedicine, University Hospital Basel, University of Basel, Basel, Switzerland
    • 6Oxford Clinical Trials Research Unit, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
    • 7Cochrane Hungary, Clinical Centre of the University of Pécs, Medical School, University of Pécs, Pécs, Hungary
    • 8Institute for Evidence in Medicine (for Cochrane Germany Foundation), Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
    • 9Department of Medicine, Women’s College Research Institute, Women’s College Hospital, University of Toronto, Toronto, Ontario, Canada
    • 10Applied Health Research Centre, Li Ka Shing Knowledge Institute of St Michael's Hospital, Toronto, Ontario, Canada
    • 11Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
    JAMA Netw Open. 2021;4(11):e2128898. doi:10.1001/jamanetworkopen.2021.28898
    Key Points

    Question  Are clinical trial registry data for trials with multiple registrations reliable?

    Findings  In this systematic review of 197 randomized clinical trials registered in more than 1 trial registry, sponsor and funder had the highest agreement level (90%) among registries. Primary outcome had agreement of 78%; trial status, 46%; and target sample size, 63%.

    Meaning  The findings suggest that there is low reliability of key characteristics in clinical trial registries, raising concerns about the usefulness of the information provided in the registries.

    Abstract

    Importance  Clinical trial registries are important for gaining an overview of ongoing research efforts and for deterring and identifying publication bias and selective outcome reporting. The reliability of the information in trial registries is uncertain.

    Objective  To assess the reliability of information across registries for trials with multiple registrations.

    Evidence Review  For this systematic review, 360 protocols of randomized clinical trials (RCTs) approved by research ethics committees in Switzerland, the UK, Canada, and Germany in 2012 were evaluated. Clinical trial registries were searched from March to September 2019 for corresponding registrations of these RCTs. For RCTS that were recorded in more than 1 clinical trial registry, key trial characteristics that should be identical among all trial registries (ie, sponsor, funding source, primary outcome, target sample size, trial status, date of first patient enrollment, results available, and main publication indexed) were extracted in duplicate. Agreement between the different trial registries for these key characteristics was analyzed descriptively. Data analyses were conducted from May 1 to November 30, 2020. Representatives from clinical trial registries were interviewed to discuss the study findings between February 1 and March 31, 2021.

    Findings  The analysis included 197 RCTs registered in more than 1 trial registry (151 in 2 registries and 46 in 3 registries), with 188 trials in ClinicalTrials.gov, 185 in the European Union Drug Regulating Authorities Clinical Trials Database (EudraCT), 20 in ISRCTN, and 47 in other registries. The agreement of key information across all registries was as follows: 178 of 197 RCTs (90%; 95% CI, 85%-94%) for sponsor, 18 of 20 (90%; 95% CI, 68%-99%) for funding source (funding was not reported on ClinicalTrials.gov), 154 of 197 (78%; 95% CI, 72%-84%) for primary outcome, 90 of 197 (46%; 95% CI, 39%-53%) for trial status, 122 of 194 (63%; 95% CI, 56%-70%) for target sample size, and 43 of 57 (75%; 95% CI, 62%-86%) for the date of first patient enrollment when the comparison time was increased to 30 days (date of first patient enrollment was not reported on EudraCT). For results availability in trial registries, agreement was 122 of 197 RCTs (62%; 95% CI, 55%-69%) for summary data reported in the registry and 91 of 197 (46%; 95% CI, 39%-53%) for whether a published article with the main results was indexed. Different legal requirements were stated as the main reason for inconsistencies by representatives of clinical trial registries.

    Conclusions and Relevance  In this systematic review, for a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of the information provided in these registries. Further harmonization across clinical trial registries may be necessary to increase their usefulness.

    Introduction

    Randomized clinical trials (RCTs) are at the top of classical textbook evidence-based medicine pyramids.1,2 However, RCTs reach their full potential only if they are designed, conducted, and reported appropriately. In an article in 2003, Dickersin and Rennie3 highlighted the lack of transparent overview of clinical research and concluded that “a comprehensive register of initiated clinical trials, with each trial assigned a unique identifier, would inform reviewers, physicians, and others (eg, consumers) about which trials had been started and directly address the problem of publication bias.” In 2005, the International Committee of Medical Journal Editors4 reported that newly implemented trials (ie, those started after July 2005) would be considered for publication only if they were prospectively registered before enrollment of the first trial participant.

    At present, clinical trial registries are well established and are supposed to provide a comprehensive overview of all ongoing RCTs on a specific topic to help deter unnecessary duplication of research and to estimate and deter publication bias.5 Furthermore, outcomes must be prespecified in clinical trial registries, which should discourage authors from cherry-picking results in RCTs.6,7 Therefore, clinical trial registries can be used as a tool to assess whether outcomes are reported in the final published article as previously specified or whether the primary outcome was changed and new outcomes were introduced.8,9 The ongoing COVID-19 pandemic has further underlined the importance of clinical trial registries for providing an overview of ongoing research efforts and, thus, for creating synergies.10-12 Nevertheless, the reliability of the information found on clinical trial registration websites remains uncertain.13 Because some trials are registered in more than 1 clinical trial registry (eg, owing to different requirements by the sponsor and funder or by different participating countries), we empirically assessed the reliability of information on RCTs available in multiple clinical trial registries.

    Methods
    Search and Data Extraction

    For this systematic review, we used data from a previous study14 on 360 RCT protocols (excluding pilot and phase 1 studies) that were approved by research ethics committees in 2012 in Switzerland, the UK, Germany, and Canada. We searched for registration records of each of these RCTs from March to 2019 using the following platforms: the World Health Organization (WHO) International Clinical Trials Registry Platform, ClinicalTrials.gov, the European Union Drug Regulating Authorities Clinical Trials Database (EudraCT), the ISRCTN registry, and the Google web search engine. We used registration numbers provided in study protocols (when available) or combinations of the population, intervention, control, or primary outcome as search terms together with the name of the principal investigator. Other clinical trial platforms (eg, German Clinical Trials Register, Clinical Trials Registry–India) were considered if they were identified through the aforementioned search platforms (ie, WHO search portal or Google web search). This study followed the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guideline.15

    When an RCT was identified in a trial registry, we extracted key trial characteristics (ie, sponsor, funding source, primary outcome, target sample size, recruitment status, and date of first patient enrollment). The key trial characteristics were selected from the WHO Trial Registration Data Set,16 excluding items that could or should be different among different registries (eg, registration number, date of registration, and contact for public queries) and items that would have required substantially more resources to conduct a meaningful comparison (eg, inclusion and exclusion criteria, secondary outcomes). Furthermore, we checked whether the results of the study and a link to the main published article (containing primary results) were available in the registry records. The searches and data extraction were performed in duplicate by some of us (B.S., V.L.G., K.K., A.T.H., N.G., I.R.M., H.L., A.M., S.L., E.N., and A.B.), and disagreements were resolved by discussion.

    Data Analysis

    Data analyses were conducted from May 1 to November 30, 2020. To assess the reliability of the trial information, we analyzed whether the key trial characteristics were identical in all identified trial registries. Furthermore, we evaluated, in a 1-to-1 comparison of registries, the agreement between the information available in the most commonly identified primary clinical trial registries (ie, ClinicalTrials.gov vs EudraCT, ClinicalTrials.gov vs other registries, and EudraCT vs other registries). Because clinical trial registries used different terms to describe trial status, the following categories were created: (1) completed, (2) ongoing (with or without recruitment), (3) terminated, and (4) unclear. The trial status on EudraCT was reported by each participating country separately. If the status was recorded as ongoing in any participating country, we judged the overall status to be ongoing for the study. When some countries stated that the trial was terminated and others indicated that it was completed, we judged the status to be unclear. For the variable date of first patient enrolled, we added an additional analysis to assess whether the indicated date was in agreement among the different registries when the comparison time was increased to 30 days. For comparison of the primary outcomes, we compared the time point at which the outcome was measured and the analysis metric (eg, change from baseline, final values) in addition to the type of outcome. We judged the primary outcome to be “potentially identical, but some details missing in 1 registry,” if 1 of these outcome characteristics was reported in 1 registry but missing from another. We conducted a stratified analysis to explore whether there was a difference in registry agreement between industry-sponsored and investigator-sponsored RCTs. The sample of analyzed studies was reduced for a few trial characteristics that were not systematically reported in some registries (ie, funding source and date when first patient was enrolled). In response to peer-review feedback, we compared the primary outcome and target sample size for RCTs with discrepancies among trial registries with those reported in the published article with the main results. All analyses were conducted using Stata, version 16.1 (StataCorp LLC) and were descriptive (including 95% CIs) without any formal hypothesis testing.

    Interviews With Clinical Trial Registry Representatives

    Representatives from the 7 trial registries (ClinicalTrials.gov, EudraCT, ISRCTN registry, the German Clinical Trials Register, the Clinical Trials Registry–India, the Australian New Zealand Clinical Trials Registry, the Japan Primary Registries Network) included in this study were contacted by email. We presented our results and invited them to a short interview to discuss the following key points: (1) Do you plan to work toward more harmonization of registry items across registries (eg, plan to add items for trial characteristics that are present in other registries)? (2) Do you know of any developments since 2012 that could have improved the reliability of information available in clinical trial registries? (3) What measures could be undertaken to improve the agreement of clinical trial registries? Could other stakeholders, such as funding agencies, research ethics committees, or academic institutions, have a role here? and (4) Are you aware of other studies that investigated the consistency of RCT information across registries? Interviews were recorded with the oral permission of clinical trial registry representatives. Answers were summarized qualitatively, focusing on the 4 aforementioned key points.

    Results
    Reliability of Available Information in Clinical Trial Registries

    From the sample of 360 RCT protocols approved in 2012, 197 were registered in more than 1 clinical trial registry. Of those 197 RCTs, 151 (77%) were identified in 2 registries and 46 (23%) in 3 registries (Table 1). The RCTs were registered in the following clinical trial registries: ClinicalTrials.gov (188 [95%]), EudraCT (185 [94%]), ISRCTN registry (20 [10%]), and other registries (47 [24%]), including the German Clinical Trials Register (n = 33), the Clinical Trials Registry–India (n = 11), the Australian New Zealand Clinical Trials Registry (n = 2), and the Japan Primary Registries Network (n = 1). The RCT protocols were originally approved in Switzerland (92 [47%]), the UK (58 [29%]), Germany (29 [15%]), and Canada (18 [9%]). Of the 197 trials, 171 (87%) included multiple centers, and 155 (79%) were industry sponsored.

    Most of the variables of interest were well reported (ie, >90%) in the clinical trial registries (Table 2). The 2 exceptions were date of first patient enrollment, which was not reported on EudraCT, and funding source, which was in a merged entry field on ClinicalTrials.gov under “Collaborators” and was also not consistently available on EudraCT (165 of 185 RCTs [89%]). The main publication was indexed in 101 of 188 ClinicalTrials.gov registrations (54%), 8 of 20 ISRCTN registrations (40%), 15 of 185 EudraCT registrations (8%), and 2 of 47 other registrations (4%). Results were posted on 127 of 185 EudraCT registrations (69%) and 107 of 188 ClinicalTrials.gov registrations (57%) but on none of the ISRCTN or other registrations (Table 2).

    The proportion of trials with consistent information across all registries was 178 of 197 (90%; 95% CI, 85%-94%) for sponsor, 18 of 20 (90%; 95% CI, 68%-99%) for funding source (funding was not reported on ClinicalTrials.gov), and 154 of 197 (78%; 95% CI, 72%-84%) for the primary outcome (Table 3). The primary outcomes of another 17 of 197 RCTs (9%; 95% CI, 5%-13%) were potentially identical, but some key characteristics (eg, time point of measurement or assessment) were missing in at least 1 registry. Furthermore, the agreement across all registries was 90 of 197 RCTs (46%; 95% CI, 39%-53%) for the trial status, 122 of 194 (63%; 95% CI, 39%-53%) for the target sample size, and 1 of 57 (2%; 95% CI, 0%-9%) for the date of first patient enrollment (date of first patient enrollment was not reported on EudraCT). When the time was increased to 30 days, the agreement for date of first patient enrollment increased to 43 of 57 (75%; 95% CI, 62%-86%). The results from the 1-to-1 comparison were in line with the comparison across all registries (Table 3).

    The agreement across all registries with respect to results availability was 122 of 197 RCTs (62%; 95% CI, 55%-69%) for summary data reported in the registry and 91 of 197 (46%; 95% CI, 39%-53%) for whether a published article reporting the main results was listed. When we considered only RCTs with results available in at least 1 registry (ie, excluding RCTs without any results in any registry), the agreement decreased to 67 of 142 (47%; 95% CI, 39%-53%). For indexed publications, the agreement decreased to 11 of 114 (10%; 95% CI, 5%-17%) when we only considered RCTs with a publication indexed in at least 1 registry. These findings appeared to be consistent between subgroups of industry-sponsored and investigator-sponsored trials (Table 4). When trials with inconsistent registry entries were assessed for sample size or primary outcome, the agreement between the information reported in the publication and that reported in trial registries was low to moderate for all trial registries (Table 5).

    Perspectives of Clinical Trial Registry Representatives

    Representatives from ClinicalTrials.gov, EudraCT, ISRCTN, the German Clinical Trials Register, and the Australian New Zealand Clinical Trials Registry agreed to participate in a short interview from February 1 to March 31, 2021. The representatives reported that there were ongoing efforts to harmonize clinical trial registries, referring to the implemented standardized WHO Trial Registration Data Set16 and to regular meetings organized by the WHO, as well as bilateral meetings among trial registry representatives. Even though these efforts have been made and continue, different legislation requirements were mentioned as the main hurdle to achieve better harmonization among trial registries. One representative stated, “The main challenge is that even when there is a desire to harmonize, each region’s legal requirements take priority and can’t easily be changed.” Another stated, “Although sponsors are required by law to submit information in our database that is in line with the content of their trials’ protocols and results, there is no legal basis that would require sponsors to submit the same information as it was previously submitted in other databases.” All representatives assumed that lack of regular updates to all trial registries was the main reason for the inconsistencies among trial registries. Representatives emphasized that it is the responsibility of the study sponsor to make sure that the information in study registries is correct and up to date. With regard to some initiatives that might have improved the situation, most representatives mentioned that regular reminders were implemented not only to add study results but also to update trial characteristics. One representative commented, “If you don’t ask them, they don’t tell you about the changes.” In addition to the option to upload study results (possible for all registries but introduced after 2012 for some registries), most registries provide the option to upload study protocols, which could help to ensure that important information, such as the primary outcome, remains consistent among different sources.

    When asked what other stakeholders could contribute to improve the situation, 2 representatives mentioned independently that journals should do some basic checks, such as assessing whether the registration actually exists. One representative stated, “We see that publications exist with a provisional registration number but the registration is not publicly available.” Another representative stated that better practices in clinical trials need to be promoted at all levels: “We hear that primary outcomes aren’t even well specified in the primary protocol document itself. So it’s really difficult to have a well-defined primary outcome on the registry when you are working with some material that is suboptimal.” In addition, it was mentioned that it would be good to link final reports from research ethics committees with trial registries. None of the representatives were aware of a similar study assessing the agreement of study information across registries.

    Discussion

    Clinical trial registries were established to identify and deter publication bias and selective reporting of outcomes as well as to provide an overview of ongoing research efforts. To fulfill this purpose, the information in clinical trial registries should be reliable.13 Although the WHO states that “Trials should only be included on more than 1 registry if it is absolutely necessary,”17 country-specific requirements often force investigators to register a trial in multiple trial registries (ie, multinational studies). We used the availability of multiple trial registrations to assess the agreement of information in clinical trial registries. Our study findings suggest that the consistency of key information across registries may be variable for a given trial. For none of the examined characteristics was the agreement above 90%. Only 2 characteristics, sponsor and funder, reached an agreement level of 90%. For all other characteristics, the agreement was lower. This is especially concerning for important key characteristics, such as primary outcome (78% agreement), trial status (46% agreement), and target sample size (63% agreement). The differences in regard to completeness of study results in trial registries may be attributable to some registries having introduced the option to upload study results after 2012; also, listing within the registry whether the study results were published has been mandatory only since 2018.18 Interviews with representatives from clinical trial registries showed that although some efforts to harmonize registries have been made, these efforts have mainly focused on the presentation of study results rather than on trial characteristics. In addition, interviewees suggested that different legal requirements in different countries hamper the harmonization of trial registries. Although several studies8,19,20 have revealed that there may be large discrepancies between information reported in a clinical trial registry and information that is later published, this is, to our knowledge, the first investigation assessing the consistency of records across trial registries for the same trial.

    Limitations

    This study has limitations. First, the sample consisted mostly of multicenter industry trials, and the size is relatively small, considering the vast amount of registered RCTs; therefore, it might not be fully representative. This finding raises the question of whether investigator-initiated trials or single-center trials might perform better. Our subgroup analysis suggested that the performance of investigator-sponsored trials was equally poor. This finding is in line with several research studies21,22 that found that industry trials and multicenter trials often performed similarly or even better when adequate reporting was assessed (eg, publishing study results; adherence to Consolidated Standards of Reporting Trials [CONSORT] guidelines). Second, the sample consisted of RCTs that received ethical approval in 2012. The situation may have changed since then, and characteristics of RCTs that are currently registered may have become more reliable. For example, as mentioned by the representatives of trial registries in the interviews, some registries have started to facilitate uploading of trial results (ie, ISRCTN, EudraCT, and the Australian New Zealand Clinical Trials Registry) and introduced or will introduce more user-friendly platforms (EudraCT, the German Clinical Trials Register); others started sending reminders about the trial status (ClinicaTrial.gov, the Australian New Zealand Clinical Trials Registry) or have restructured how primary endpoints should be reported after a study revealed that the reporting quality was relatively poor (ClinicalTrials.gov).23,24 However, as Tse et al24 stated when discussing the quality of trial registry information, “quality control review cannot ensure the veracity of the submitted information.” Other practices, such as uploading the study protocol, might be more beneficial for the agreement of trial information across registries. This practice was partially implemented in ClinicalTrials.gov, where since 2017, the trial results reported on the registry have been required to include the trial protocol (optional during registration). Third, we extracted data from multiple registries that were available on the day of extraction and did not consider older available versions of registry entries. It is likely that discrepancies arose because some registries were updated while others were not. From a user’s perspective, we believe that this is irrelevant and therefore not a real limitation because a user (eg, scientist or patient) should be able to trust the information of a trial registry at any given time point. Fourth, investigators may have entered country-specific values for some characteristics (eg, sample size, recruitment of first patient). We believe that this was not likely because most definitions of trial characteristics clearly stated that data for the overall trial were required (eg, “Planned number of subjects to be included in the whole clinical trial,” as defined on EudraCT). Entering of country-specific data would be misleading from a user’s perspective because none of the trial characteristics were labeled as being country specific. Fifth, assessment of which registry was most accurate could be done only in a limited scope on a small sample in this study (Table 5). Larger studies would be required to make a meaningful direct comparison among trial registries.

    Conclusions

    In this systematic review, for a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of clinical trial registries. Further investigation and harmonization efforts across clinical trial registries appear to be necessary to increase their usefulness. Interventions that might increase the reliability include uploading of study protocols (entire protocols or key parts) or linking trial registries through platforms (eg, WHO International Clinical Trials Registry Platform) and implementing automated systems that detect inconsistencies. These 2 interventions may help avoid differences in characteristics that are specified before the start of the trial (ie, uploading trial protocol to specify sponsor, funder, target sample size, and primary outcome) and differences that could be introduced during the course of the trial (ie, actively comparing entries across registries for recruitment of first patient, trial status, and availability of study results and providing links to publication of main results).

    Back to top
    Article Information

    Accepted for Publication: July 22, 2021.

    Published: November 1, 2021. doi:10.1001/jamanetworkopen.2021.28898

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Speich B et al. JAMA Network Open.

    Corresponding Author: Benjamin Speich, PhD, Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, Spitalstrasse 12, 4056 Basel, Switzerland (benjamin.speich@usb.ch).

    Author Contributions: Drs Hopewell and Briel were co–last authors. Dr Speich had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Speich, Hopewell, Briel.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Speich, Gryaznov, Marian, Hopewell.

    Critical revision of the manuscript for important intellectual content: Speich, Gloy, Klatte, Gryaznov, Taji Heravi, Ghosh, Marian, Lee, Mansouri, Lohner, Saccilotto, Nury, Chan, Blümle, Odutayo, Briel.

    Statistical analysis: Speich, Gryaznov, Ghosh, Marian.

    Obtained funding: Speich.

    Administrative, technical, or material support: Speich, Gloy, Ghosh, Marian, Lee, Mansouri, Saccilotto, Briel.

    Supervision: Speich, Hopewell, Briel.

    Conflict of Interest Disclosures: Dr Gryaznov reported being currently employed by Idorsia Pharmaceuticals Ltd outside the submitted work. Ms Marian reporting receiving a National Institute for Health Research (UK) Pre-Doctoral Fellowship during the conduct of the study. Dr Lohner reported receiving a János Bolyai Research Scholarship to conduct evidence-based medicine–related methodological research from the Hungarian Academy of Sciences and financial support from Alexander von Humboldt Foundation for the research stay at the Institute for Evidence in Medicine, University of Freiburg, Germany, during the conduct of the study. No other disclosures were reported.

    Funding/Support: Dr Speich was supported by Advanced Postdoc.Mobility (P300PB_177933) and Return Postdoc.Mobility (P4P4PM_194496) grants from the Swiss National Science Foundation.

    Role of the Funder/Sponsor: The Swiss National Science Foundation had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Group Members: The Adherence to Spirit Recommendations (ASPIRE) Study Group members are listed in the Supplement.

    Additional Contributions: We thank the clinical trial representatives for participating in the semi-structured interviews and all participating research ethics committees from Germany (Freiburg), Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St Gallen, Thurgau, and Zurich), Canada (Hamilton), and the UK (National Health Service Health Research Authority) for their support and cooperation and for granting us access to approved study protocols from 2012. None of the clinical trial representatives or members of the research ethics committees were compensated for their participation.

    References
    1.
    Burns  PB, Rohrich  RJ, Chung  KC.  The levels of evidence and their role in evidence-based medicine.   Plast Reconstr Surg. 2011;128(1):305-310. doi:10.1097/PRS.0b013e318219c171 PubMedGoogle ScholarCrossref
    2.
    Murad  MH, Asi  N, Alsawas  M, Alahdab  F.  New evidence pyramid.   Evid Based Med. 2016;21(4):125-127. doi:10.1136/ebmed-2016-110401 PubMedGoogle ScholarCrossref
    3.
    Dickersin  K, Rennie  D.  Registering clinical trials.   JAMA. 2003;290(4):516-523. doi:10.1001/jama.290.4.516 PubMedGoogle ScholarCrossref
    4.
    De Angelis  CD, Drazen  JM, Frizelle  FA,  et al; International Committee of Medical Journal Editors.  Is this clinical trial fully registered? a statement from the International Committee of Medical Journal Editors.   Lancet. 2005;365(9474):1827-1829. doi:10.1016/S0140-6736(05)66588-9 PubMedGoogle ScholarCrossref
    5.
    Sim  I, Chan  AW, Gülmezoglu  AM, Evans  T, Pang  T.  Clinical trial registration: transparency is the watchword.   Lancet. 2006;367(9523):1631-1633. doi:10.1016/S0140-6736(06)68708-4 PubMedGoogle ScholarCrossref
    6.
    Mayo-Wilson  E, Li  T, Fusco  N,  et al.  Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy.   J Clin Epidemiol. 2017;91:95-110. doi:10.1016/j.jclinepi.2017.07.014 PubMedGoogle ScholarCrossref
    7.
    Chan  AW, Pello  A, Kitchen  J,  et al.  Association of trial registration with reporting of primary outcomes in protocols and publications.   JAMA. 2017;318(17):1709-1711. doi:10.1001/jama.2017.13001 PubMedGoogle ScholarCrossref
    8.
    Goldacre  B, Drysdale  H, Dale  A,  et al.  COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time.   Trials. 2019;20(1):118. doi:10.1186/s13063-019-3173-2 PubMedGoogle ScholarCrossref
    9.
    Chan  AW, Hróbjartsson  A, Haahr  MT, Gøtzsche  PC, Altman  DG.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.   JAMA. 2004;291(20):2457-2465. doi:10.1001/jama.291.20.2457 PubMedGoogle ScholarCrossref
    10.
    Nguyen  VT, Rivière  P, Ripoll  P,  et al; COVID-NMA Consortium Team.  Research response to coronavirus disease 2019 needed better coordination and collaboration: a living mapping of registered trials.   J Clin Epidemiol. 2021;130:107-116;130:107-116. doi:10.1016/j.jclinepi.2020.10.010 PubMedGoogle ScholarCrossref
    11.
    Kouzy  R, Abi Jaoude  J, Garcia  CJ, El Alam  MB, Taniguchi  CM, Ludmir  EB.  Characteristics of the multiplicity of randomized clinical trials for coronavirus disease 2019 launched during the pandemic.   JAMA Netw Open. 2020;3(7):e2015100. doi:10.1001/jamanetworkopen.2020.15100 PubMedGoogle Scholar
    12.
    Janiaud  P, Axfors  C, Van’t Hooft  J,  et al.  The worldwide clinical trial research response to the COVID-19 pandemic—the first 100 days.   F1000Res. 2020;9:1193. doi:10.12688/f1000research.26707.1 PubMedGoogle ScholarCrossref
    13.
    Heale  R, Twycross  A.  Validity and reliability in quantitative studies.   Evid Based Nurs. 2015;18(3):66-67. doi:10.1136/eb-2015-102129 PubMedGoogle ScholarCrossref
    14.
    Gryaznov  D, Odutayo  A, von Niederhäusern  B,  et al.  Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to Spirit Recommendations (ASPIRE) study and associated projects.   Trials. 2020;21(1):896. doi:10.1186/s13063-020-04808-y PubMedGoogle ScholarCrossref
    15.
    Page  MJ, Moher  D, Bossuyt  PM,  et al.  PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews.   BMJ. 2021;372(n160):n160. doi:10.1136/bmj.n160 PubMedGoogle Scholar
    16.
    World Health Organization. WHO Trial Registration Data Set (version 1.3.1). Accessed November 13, 2020. https://www.who.int/ictrp/network/trds/en/
    17.
    World Health Organization. What registries and registrants can do to facilitate unambiguous identification. Accessed April 19, 2021. https://www.who.int/clinical-trials-registry-platform/unambiguous-trial-identification/what-registries-and-registrants-can-do-to-facilitate-unambiguous-identification
    18.
    World Health Organization. International Standards for Clinical Trial Registries: the registration of all interventional trials is a scientific, ethical and moral responsibility, version 3.0. Accessed May 27, 2021. https://apps.who.int/iris/handle/10665/274994
    19.
    Hartung  DM, Zarin  DA, Guise  JM, McDonagh  M, Paynter  R, Helfand  M.  Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications.   Ann Intern Med. 2014;160(7):477-483. doi:10.7326/M13-0480 PubMedGoogle ScholarCrossref
    20.
    Riveros  C, Dechartres  A, Perrodeau  E, Haneef  R, Boutron  I, Ravaud  P.  Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals.   PLoS Med. 2013;10(12):e1001566. doi:10.1371/journal.pmed.1001566 PubMedGoogle Scholar
    21.
    Kasenda  B, von Elm  E, You  J,  et al.  Prevalence, characteristics, and publication of discontinued randomized trials.   JAMA. 2014;311(10):1045-1051. doi:10.1001/jama.2014.1361 PubMedGoogle ScholarCrossref
    22.
    Jin  Y, Sanger  N, Shams  I,  et al.  Does the medical literature remain inadequately described despite having reporting guidelines for 21 years?—a systematic review of reviews: an update.   J Multidiscip Healthc. 2018;11:495-510. doi:10.2147/JMDH.S155103 PubMedGoogle ScholarCrossref
    23.
    Viergever  RF, Karam  G, Reis  A, Ghersi  D.  The quality of registration of clinical trials: still a problem.   PLoS One. 2014;9(1):e84727. doi:10.1371/journal.pone.0084727 PubMedGoogle Scholar
    24.
    Tse  T, Fain  KM, Zarin  DA.  How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider.   BMJ. 2018;361:k1452. doi:10.1136/bmj.k1452 PubMedGoogle Scholar
    ×