[Skip to Navigation]
Sign In
Figure 1.  Time Intervals for Various Trial Milestones
Time Intervals for Various Trial Milestones

The current study addresses the “site oversight” portion of the trial approval process. In the schematic, the regulatory approval and the contract execution processes occur concurrently and thus may have different end times. IRB indicates institutional review board.

Figure 2.  Time to Milestone by 2004-2007 vs 2008-2012 Trials
Time to Milestone by 2004-2007 vs 2008-2012 Trials

The time to various milestones (A, regulatory approval; B, contract execution; C, activation; D, enrollment of first participant; and E, overall start-up) are stratified by early trials from 2004-2007 (Assessment of Pexelizumab in Acute Myocardial Infarction [APEX-AMI], Improved Reduction of Outcomes: Vytorin Efficacy International Trial [IMPROVE-IT], Rivaroxaban vs Warfarin in Nonvalvular Atrial Fibrillation [ROCKET-AF], Acute Study of Clinical Effectiveness of Nesiritide in Decompensated Heart Failure [ASCEND-AF], and Thrombin Receptor Antagonist for Clinical Event Reduction in Acute Coronary Syndrome [TRACER]) and contemporary trials from 2008-2012 (Trial Evaluating Cardiovascular Outcomes with Sitagliptin [TECOS], Exenatide Study of Cardiovascular Event Lowering [EXSCEL], Evaluation of Cardiovascular Outcomes After an Acute Coronary Syndrome During Treatment With Alirocumab [ODYSSEY], and Examining Use of Ticagrelor in Peripheral Artery Disease [EUCLID]).

Table 1.  Trial Attributes and No. of North American Sites by Milestone
Trial Attributes and No. of North American Sites by Milestone
Table 2.  Time to Trial Start-up Milestones, at All Sites and the Top 10% of Sitesa
Time to Trial Start-up Milestones, at All Sites and the Top 10% of Sitesa
Table 3.  Time to Milestone by IRB Type
Time to Milestone by IRB Type
1.
Jones  WS, Roe  MT, Antman  EM,  et al.  The changing landscape of randomized clinical trials in cardiovascular disease.   J Am Coll Cardiol. 2016;68(17):1898-1907. doi:10.1016/j.jacc.2016.07.781 PubMedGoogle ScholarCrossref
2.
Luce  BR, Kramer  JM, Goodman  SN,  et al.  Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change.   Ann Intern Med. 2009;151(3):206-209. doi:10.7326/0003-4819-151-3-200908040-00126 PubMedGoogle ScholarCrossref
3.
Eisenstein  EL, Lemons  PW  II, Tardiff  BE, Schulman  KA, Jolly  MK, Califf  RM.  Reducing the costs of phase III cardiovascular clinical trials.   Am Heart J. 2005;149(3):482-488. doi:10.1016/j.ahj.2004.04.049 PubMedGoogle ScholarCrossref
4.
Snyderman  R.  The clinical researcher—an “emerging” species.   JAMA. 2004;291(7):882-883. doi:10.1001/jama.291.7.882 PubMedGoogle ScholarCrossref
5.
Fanaroff  AC, Califf  RM, Lopes  RD.  High-quality evidence to inform clinical practice.   Lancet. 2019;394(10199):633-634. doi:10.1016/S0140-6736(19)31256-5 PubMedGoogle ScholarCrossref
6.
Fanaroff  AC, Califf  RM, Lopes  RD.  New approaches to conducting randomized controlled trials.   J Am Coll Cardiol. 2020;75(5):556-559. doi:10.1016/j.jacc.2019.11.043 PubMedGoogle ScholarCrossref
7.
Zwarenstein  M, Treweek  S, Gagnier  JJ,  et al; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group.  Improving the reporting of pragmatic trials: an extension of the CONSORT statement.   BMJ. 2008;337:a2390. doi:10.1136/bmj.a2390 PubMedGoogle ScholarCrossref
8.
Tunis  SR, Stryer  DB, Clancy  CM.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.   JAMA. 2003;290(12):1624-1632. doi:10.1001/jama.290.12.1624 PubMedGoogle ScholarCrossref
9.
Fanaroff  AC, Califf  RM, Windecker  S, Smith  SC  Jr, Lopes  RD.  Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008-2018.   JAMA. 2019;321(11):1069-1080. doi:10.1001/jama.2019.1122 PubMedGoogle ScholarCrossref
10.
Fanaroff  AC, Fudim  M, Califf  RM, Windecker  S, Smith  SC  Jr, Lopes  RD.  Levels of evidence supporting drug, device, and other recommendations in the American Heart Association/American College of Cardiology guidelines.   Am Heart J. 2020;226:4-12. doi:10.1016/j.ahj.2020.05.003 PubMedGoogle ScholarCrossref
11.
GUSTO investigators.  An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction.   N Engl J Med. 1993;329(10):673-682. doi:10.1056/NEJM199309023291001 PubMedGoogle ScholarCrossref
12.
Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO) IIa investigators.  Randomized trial of intravenous heparin versus recombinant hirudin for acute coronary syndromes.   Circulation. 1994;90(4):1631-1637. doi:10.1161/01.CIR.90.4.1631 PubMedGoogle ScholarCrossref
13.
Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO) IIb investigators.  A comparison of recombinant hirudin with heparin for the treatment of acute coronary syndromes.   N Engl J Med. 1996;335(11):775-782. doi:10.1056/NEJM199609123351103 PubMedGoogle ScholarCrossref
14.
Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO III) Investigators.  A comparison of reteplase with alteplase for acute myocardial infarction.   N Engl J Med. 1997;337(16):1118-1123. doi:10.1056/NEJM199710163371603 PubMedGoogle ScholarCrossref
15.
Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) Trial Investigators.  Inhibition of platelet glycoprotein IIb/IIIa with eptifibatide in patients with acute coronary syndromes.   N Engl J Med. 1998;339(7):436-443. doi:10.1056/NEJM199808133390704 PubMedGoogle ScholarCrossref
16.
 International, randomized, controlled trial of lamifiban (a platelet glycoprotein IIb/IIIa inhibitor), heparin, or both in unstable angina. The PARAGON Investigators. Platelet IIb/IIIa Antagonism for the Reduction of Acute coronary syndrome events in a Global Organization Network.   Circulation. 1998;97(24):2386-2395. doi:10.1161/01.CIR.97.24.2386 PubMedGoogle ScholarCrossref
17.
Global Organization Network (PARAGON)-B Investigators.  Randomized, placebo-controlled trial of titrated intravenous lamifiban for acute coronary syndromes.   Circulation. 2002;105(3):316-321. doi:10.1161/hc0302.102573 PubMedGoogle ScholarCrossref
18.
 Comparison of sibrafiban with aspirin for prevention of cardiovascular events after acute coronary syndromes: a randomised trial. The SYMPHONY Investigators. Sibrafiban versus Aspirin to Yield Maximum Protection from Ischemic Heart Events Post-acute Coronary Syndromes.   Lancet. 2000;355(9201):337-345. doi:10.1016/S0140-6736(99)11179-6 PubMedGoogle ScholarCrossref
19.
Ferguson  JJ, Califf  RM, Antman  EM,  et al; SYNERGY Trial Investigators.  Enoxaparin vs unfractionated heparin in high-risk patients with non–ST-segment elevation acute coronary syndromes managed with an intended early invasive strategy: primary results of the SYNERGY randomized trial.   JAMA. 2004;292(1):45-54.PubMedGoogle Scholar
20.
Armstrong  PW, Granger  CB, Adams  PX,  et al; APEX AMI Investigators.  Pexelizumab for acute ST-elevation myocardial infarction in patients undergoing primary percutaneous coronary intervention: a randomized controlled trial.   JAMA. 2007;297(1):43-51. doi:10.1001/jama.297.1.43 PubMedGoogle Scholar
21.
Pfeffer  MA, McMurray  JJ, Velazquez  EJ,  et al; Valsartan in Acute Myocardial Infarction Trial Investigators.  Valsartan, captopril, or both in myocardial infarction complicated by heart failure, left ventricular dysfunction, or both.   N Engl J Med. 2003;349(20):1893-1906. doi:10.1056/NEJMoa032292 PubMedGoogle ScholarCrossref
22.
Cannon  CP, Blazing  MA, Giugliano  RP,  et al; IMPROVE-IT Investigators.  Ezetimibe added to statin therapy after acute coronary syndromes.   N Engl J Med. 2015;372(25):2387-2397. doi:10.1056/NEJMoa1410489PubMedGoogle ScholarCrossref
23.
Patel  MR, Mahaffey  KW, Garg  J,  et al; ROCKET AF Investigators.  Rivaroxaban versus warfarin in nonvalvular atrial fibrillation.   N Engl J Med. 2011;365(10):883-891. doi:10.1056/NEJMoa1009638PubMedGoogle ScholarCrossref
24.
O’Connor  CM, Starling  RC, Hernandez  AF,  et al.  Effect of nesiritide in patients with acute decompensated heart failure.   N Engl J Med. 2011;365(1):32-43. doi:10.1056/NEJMoa1100171PubMedGoogle ScholarCrossref
25.
Tricoci  P, Huang  Z, Held  C,  et al; TRACER Investigators.  Thrombin-receptor antagonist vorapaxar in acute coronary syndromes.   N Engl J Med. 2012;366(1):20-33. doi:10.1056/NEJMoa1109719PubMedGoogle ScholarCrossref
26.
Green  JB, Bethel  MA, Armstrong  PW,  et al; TECOS Study Group.  Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes.   N Engl J Med. 2015;373(3):232-242. doi:10.1056/NEJMoa1501352PubMedGoogle ScholarCrossref
27.
Holman  RR, Bethel  MA, Mentz  RJ,  et al; EXSCEL Study Group.  Effects of once-weekly exenatide on cardiovascular outcomes in type 2 diabetes.   N Engl J Med. 2017;377(13):1228-1239. doi:10.1056/NEJMoa1612917PubMedGoogle ScholarCrossref
28.
Schwartz  GG, Steg  PG, Szarek  M,  et al; ODYSSEY OUTCOMES Committees and Investigators.  Alirocumab and cardiovascular outcomes after acute coronary syndrome.   N Engl J Med. 2018;379(22):2097-2107. doi:10.1056/NEJMoa1801174PubMedGoogle ScholarCrossref
29.
Hiatt  WR, Fowkes  FG, Heizer  G,  et al; EUCLID Trial Steering Committee and Investigators.  Ticagrelor versus clopidogrel in symptomatic peripheral artery disease.   N Engl J Med. 2017;376(1):32-40. doi:10.1056/NEJMoa1611688PubMedGoogle ScholarCrossref
30.
Krafcik  BM, Doros  G, Malikova  MA.  A single center analysis of factors influencing study start-up timeline in clinical trials.   Future Sci OA. 2017;3(4):FSO223. doi:10.4155/fsoa-2017-0025 PubMedGoogle Scholar
31.
Abu-Shaheen  A, Al Badr  A, Al Fayyad  I, Al Qutub  A, Faqeih  EA, Al-Tannir  M.  Streamlining and cycle time reduction of the startup phase of clinical trials.   Trials. 2020;21(1):115. doi:10.1186/s13063-020-4079-8 PubMedGoogle ScholarCrossref
32.
National Institutes of Health. Single IRB policy for multi-site research. Accessed November 11, 2020. https://grants.nih.gov/policy/humansubjects/single-irb-policy-multi-site-research.htm
33.
Clinical Trials Transformation Initiative. Who we are. Accessed November 1, 2020. https://www.ctti-clinicaltrials.org/who-we-are
34.
Glickman  SW, McHutchison  JG, Peterson  ED,  et al.  Ethical and scientific implications of the globalization of clinical research.   N Engl J Med. 2009;360(8):816-823. doi:10.1056/NEJMsb0803929 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Cardiology
    July 23, 2021

    Assessment of North American Clinical Research Site Performance During the Start-up of Large Cardiovascular Clinical Trials

    Author Affiliations
    • 1The Ohio State University Wexner Medical Center, Division of Cardiovascular Medicine, Columbus
    • 2Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina
    • 3Verana Health, San Francisco, California
    • 42M Clinical, Arlington, Texas
    • 5Stanford Center for Clinical Research, Department of Medicine, Stanford University, Stanford, California
    • 6Verily Life Sciences and Google Health, San Francisco, California
    JAMA Netw Open. 2021;4(7):e2117963. doi:10.1001/jamanetworkopen.2021.17963
    Key Points

    Question  What are the start-up times needed to reach various milestones for North American research sites in large cardiovascular trials, and how do these milestones vary by time and regulatory process?

    Findings  In this cohort study including data from 9 clinical trials, the median start-up time (from study protocol delivery to first participant enrollment) was 255 days, which significantly improved from 267 days for trials in 2004-2007 to 237 days for trials in 2008-2012. Sites using a central vs local regulatory process had a significantly reduced start-up time of 199 vs 287 days, respectively.

    Meaning  In addition to providing benchmark metrics, these data demonstrate modest improvement over time and suggest that use of central institutional review boards may enhance trial efficiency.

    Abstract

    Importance  Randomized clinical trials (RCTs) are critical in advancing patient care, yet conducting such large-scale trials requires tremendous resources and coordination. Clinical site start-up performance metrics can provide insight into opportunities for improved trial efficiency but have not been well described.

    Objective  To measure the start-up time needed to reach prespecified milestones across sites in large cardiovascular RCTs in North America and to evaluate how these metrics vary by time and type of regulatory review process.

    Design, Setting, and Participants  This cohort study evaluated cardiovascular RCTs conducted from July 13, 2004, to February 1, 2017. The RCTs were coordinated by a single academic research organization, the Duke Clinical Research Institute. Nine consecutive trials with completed enrollment and publication of results in their target journal were studied. Data were analyzed from December 4, 2019, to January 11, 2021.

    Exposures  Year of trial enrollment initiation (2004-2007 vs 2008-2012) and use of a central vs local institutional review board (IRB).

    Main Outcomes and Measures  The primary outcome was the median start-up time (from study protocol delivery to first participant enrollment) as compared by trial year and type of IRB used. The median start-up time for the top 10% of sites was also reported. Secondary outcomes included time to site regulatory approval, time to contract execution, and time to site activation.

    Results  For the 9 RCTs included, the median site start-up time shortened only slightly over time from 267 days (interquartile range [IQR], 185-358 days) for 2004-2007 trials to 237 days (IQR, 162-343 days) for 2008-2012 trials (overall median, 255 days [IQR, 177-350 days]; P < .001). For the top 10% of sites, median start-up time was 107 days (IQR, 95-121 days) for 2004-2007 trials vs 104 days (IQR, 84-118 days) for 2008-2012 trials (overall median, 106 days [IQR, 90-120 days]; P = .04). The median start-up time was shorter among sites using a central IRB (199 days [IQR, 140-292 days]) than those using a local IRB (287 days [IQR, 205-390 days]; P < .001).

    Conclusions and Relevance  This cohort study of North American research sites in large cardiovascular RCTs found a duration of nearly 9 months from the time of study protocol delivery to the first participant enrollment; this metric was only slightly shortened during the study period but was reduced to less than 4 months for top-performing sites. These findings suggest that the use of central IRBs has the potential to improve RCT efficiency.

    Introduction

    Over the past 4 decades, large randomized clinical trials (RCTs) have been critical in advancing care for patients with cardiovascular disease; these RCTs remain the foundation for practicing evidence-based, clinical medicine.1 However, conducting such RCTs on a large scale necessitates the coordination of various stakeholders (research organizations, practitioners, patients, industry, and professional societies) and numerous sites enrolling high numbers of research participants. The burden of cost and the length of time required for these types of trials have previously been described as extensive and increasing.2-4

    Despite the importance of RCTs, more than 90% of guideline recommendations made by the largest cardiovascular societies are not validated by such RCTs, leading some to call for new trial design methodologies.1,5,6 The inefficiencies in discovering practice-changing treatment options has led to the prominence of pragmatic clinical trials that facilitate enrollment of a diverse study population, inclusion of health care professionals in community settings, public-private partnerships, and patient-centered outcomes.7-10 Owing to the complexity of conducting large RCTs, several institutions across North America have developed academic research organizations to manage the logistics and challenges that may arise. The Duke Clinical Research Institute (DCRI) is one such organization that has coordinated several large cardiovascular RCTs over the past 3 decades.11-21

    Though structural changes have been made to RCT designs, few studies quantitatively assess whether such modifications to design and regulatory processes improve trial efficiency. In particular, the time taken to enroll a site’s first participant after protocol finalization remains largely unknown but represents an opportunity for improvement. To quantitatively evaluate trial efficiency, a comprehensive assessment of the DCRI’s experience with 9 large cardiovascular clinical trials enrolling participants from 2004 to 2017 was undertaken.

    Methods
    Clinical Trials

    Nine consecutive cardiovascular outcomes trials20,22-29 coordinated by the DCRI with available data were selected for this analysis. These trials—with the earliest trial beginning enrollment on July 13, 2004, and the latest trial ending enrollment on February 1, 2017—were managed by global collaborations between pharmaceutical companies and leading academic institutions. In addition, the trials covered a wide range of cardiovascular diagnoses inclusive of acute coronary syndrome, atrial fibrillation, heart failure, prevention, type 2 diabetes, and peripheral artery disease (Table 1). For consideration of inclusion, trials had to have completed enrollment with publication of results in their target journal. Although these trials were conducted globally, the present analysis reviewed those sites managed within North America. All trial protocols were approved by multiple parties, including the US Food and Drug Administration, pharmaceutical company committees, and large academic steering committees, before being sent to sites in North America. The DCRI was the primary or coprimary coordinating center for each of the trials, with the rights held by the DCRI and global academic leaders for analysis and publication of the clinical trial results. Operational metrics were collected for each trial for each DCRI-managed North American site. For analysis purposes, the included trials were stratified into 2 groups based on trial enrollment start year: early trials dated from July 13, 2004, to December 18, 2007 (Assessment of Pexelizumab in Acute Myocardial Infarction [APEX-AMI], Improved Reduction of Outcomes: Vytorin Efficacy International Trial [IMPROVE-IT], Rivaroxaban vs Warfarin in Nonvalvular Atrial Fibrillation [ROCKET-AF], Acute Study of Clinical Effectiveness of Nesiritide in Decompensated Heart Failure [ASCEND-HF], and Thrombin Receptor Antagonist for Clinical Event Reduction in Acute Coronary Syndrome [TRACER]),20,22-25 and later trials dated from December 1, 2008, to December 1, 2012 (Trial Evaluating Cardiovascular Outcomes with Sitagliptin [TECOS], Exenatide Study of Cardiovascular Event Lowering [EXSCEL], Evaluation of Cardiovascular Outcomes After an Acute Coronary Syndrome During Treatment With Alirocumab [ODYSSEY], and Examining Use of Ticagrelor in Peripheral Artery Disease [EUCLID]).26-29 This study was considered exempt from obtaining participant consent by the institutional review board (IRB) of Duke University due to the aggregation of operational data that did not involve specific participant data. This study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Operational Metrics and Definitions

    Numerous factors comprise the efficiency of overall time between site contact and enrollment conclusion. The current study examined several key operational start-up metrics within the purview of an individual site that achieved each milestone; these metrics were routinely and systematically collected for each site managed by the DCRI during trial start-up. These metrics comprised the start-up time, defined as the time from when the final study protocol was sent to the site to when the first participant was enrolled. As outlined graphically in Figure 1, in addition to start-up time, 3 milestones were monitored, including time from when the protocol was sent to the site to each of the following: (1) regulatory approval at the site (defined as the site having obtained ethics committee approval and having completed the necessary regulatory documents for study drug shipment), (2) contract execution, and (3) activation (defined as the site having authorization to enroll, which incorporates both regulatory approval and contract execution). Furthermore, the time from activation to first participant enrollment was collected. Finally, the time for regulatory approval was stratified based on the use of a central vs local IRB process.

    Statistical Analysis

    Categorical variables were presented as counts with percentages, and continuous variables were reported as medians with interquartile ranges. Cumulative distribution function plots were used to summarize the distribution of the time to each milestone for each individual trial and for early vs contemporary trials. Box plots were generated showing the time to reach either regulatory approval or contract execution for each individual trial and for early vs contemporary trials. The Wilcoxon rank sum procedure was used to test for a difference in the median time to each milestone across the 2 temporarily stratified trial groupings. The χ2 test was used to assess the difference in the proportion of sites that had contract execution before regulatory approval between early and contemporary trials. Time was measured in number of days. Unless otherwise noted, all hypothesis tests were 2-sided, and P < .05 was interpreted as statistically significant. Analyses were interpreted as exploratory, and thus no formal adjustments for multiple hypothesis testing were made. Missing values were excluded from descriptive statistical summaries. All analyses were performed from December 4, 2019, to January 11, 2021, using SAS, version 9.4 (SAS Institute Inc).

    Results

    The start of enrollment across the 9 selected trials20,22-29 ranged from July 13, 2004, to December 1, 2012 (Table 1). The times to various trial start-up milestones are described in Table 2 for all sites and for the top 10% of sites for each trial for each milestone. The median overall start-up time was 255 days (IQR, 177-350 days), which improved to 237 days (IQR, 162-343 days) for 2008-2012 trials compared with 267 days (IQR, 185-358 days) for 2004-2007 trials (P < .001). For the top 10% of sites, median start-up time was 107 days (IQR, 95-121 days) for 2004-2007 trials to 104 days (IQR, 84-118 days) for 2008-2012 trials (overall median, 106 days [IQR, 90-120 days]; P = .04). The median time to complete regulatory approval for all sites across all trials was 132 days (IQR, 78-209 days). This process improved to 105 days (IQR, 51-177 days) for 2008-2012 trials compared with 149 days (IQR, 97-224 days) for 2004-2007 trials (P < .001). Similarly, the median time to contract execution was 143 days (IQR, 74-250 days) overall and improved to 119 days (IQR, 59-223 days) for 2008-2012 trials compared with 167 days (IQR, 92-262 days) for 2004-2007 trials (P < .001). Median time to activation was 171 days (IQR, 114-246 days) and was followed by an additional 66 days (IQR, 33-124 days) to first participant enrollment. These milestones improved to 149 days (IQR, 97-217 days) and 70 days (IQR, 37-134 days), respectively, for 2008-2012 trials (P < .001) compared with 189 days (IQR, 129-265 days) and 64 days (IQR, 32-117 days), respectively, for 2004-2007 trials (P = .01). Comparable data for the top 10% of enrolling sites can be found in Table 2. The cumulative distribution function plots for 2004-2007 and 2008-2012 trials for each milestone are highlighted in Figure 2 (individual trial cumulative distribution function plots can be found in eFigures 1-5 in the Supplement).

    To explore whether contract execution or regulatory document completion was associated with the pacing of start-up times, the median times for contract execution and regulatory approval are shown, depending on which activity was first completed, in eTable 1 in the Supplement. For sites completing regulatory approval first, an additional 46 days (IQR, 13-155 days) were needed for contract execution. For sites with contract execution first, an additional 30 days (IQR, 12-63 days) were needed for regulatory approval. These times were slightly faster for both metrics for the 2008-2012 trials compared with the 2004-2007 trials (regulatory approval first: 2008-2012 trials, 33 days [IQR, 8-111 days] vs 2004-2007 trials, 60 days [IQR, 20-210 days]; P < .001; and contract execution first: 2008-2012 trials, 27 days [IQR, 8-61 days] vs 2004-2007 trials, 33 days [IQR, 14-63 days]; P = .03). Individual trial data can be found in eTable 2 in the Supplement. Additionally, the number of sites completing their contracts before obtaining regulatory approval was 1104 (49.6%) for all sites, which was fewer at 427 (46.6%) for 2008-2012 trials compared with 680 (52.0%) for 2004-2007 trials (P = .02). The box plot distributions of days to regulatory approval and contract execution are shown in eFigure 6, with individual trials shown in eFigure 7 in the Supplement.

    To compare whether the type of IRB regulatory approval process was associated with the time to IRB regulatory approval or overall start-up, the use of a central or local IRB process was compared across IRB type and across 2004-2007 vs 2008-2012 trials (Table 3). Individual trial data can be found in eTables 3 and 4 in the Supplement.

    For the regulatory analysis, 2225 sites were included in the analysis, with 798 sites (35.9%) using a central IRB, 1209 sites (54.3%) using a local IRB, and 218 sites (9.8%) without information specifying the type of IRB used. The median time to regulatory approval in sites using a central IRB was 78 days (IQR, 45-124 days), which was more than half the time seen in sites using a local IRB (165 days [IQR, 112-244 days]; P < .001). The time to regulatory approval was faster for sites using a central IRB regardless of whether they were 2004-2007 or 2008-2012 trials (2004-2007 trials with central IRB, 92 days [IQR, 64-146 days] vs local IRB, 161 days [IQR, 110-237 days]; P < .001; and 2008-2012: central IRB, 71 days [IQR, 41-112 days] vs local IRB, 181 days [IQR, 121-274 days]; P < .001), and contemporary trials more often used a central IRB process (540 sites [58.9%] vs 258 sites [19.7%]). The median time to overall start-up was also significantly less for sites using a central vs local IRB (199 days [IQR, 140-292 days] vs 287 days [IQR, 205-390 days]; P < .001).

    Discussion

    In this study of 9 large cardiovascular RCTs20,22-29 conducted at the DCRI, we found that the overall site start-up time in North America was nearly 9 months, although top-performing sites for each trial completed their start-up in less than 4 months. There was a significant but modest improvement in this metric when comparing early to contemporary trials, possibly related to the outpatient setting in which these later trials were conducted. Contemporary trials more frequently used a central IRB and had faster times to regulatory approval when compared with earlier trials and sites using a local IRB. To our knowledge, this study for the first time quantitatively characterizes these performance metrics and provides insight into target areas for improvement.

    With increasing global burden of cardiovascular disease, RCTs remain the foundation for answering challenging clinical questions and investigating new therapies. Novel approaches are needed to improve trial efficiency while maintaining rigorous oversight, ensuring protection of human participants, and keeping the interests of industry and academia transparent. The balance of various stakeholders with diverse interests is evidenced by the 255-day median start-up time observed in our sample. Krafcik et al30 described a 319-day median start-up time in a review of 38 heterogenous studies ranging from 2004 to 2016 in Boston, Massachusetts. Similarly, a 174-day start-up time across 13 Saudi clinical trials was reported by Abu-Shaheen et al,31 with a significant portion of that time dedicated to obtaining local IRB approval. To limit loss of resources spent on the contractual process without regulatory approval and vice versa, many institutions complete the regulatory approval process and contract negotiations consecutively rather than concurrently. In addition, numerous delays can be encountered. For example, sites are frequently approached to participate in multiple trials and must determine which to conduct at their institutions; protocols may change before activation and require additional review; and centers with large portfolios often have internal review committees that deliberate over how to allocate resources and select patient populations for new projects. Not addressed in the current study is the financial and human capital required for site training, which has taken on an even greater burden in recent years with the use of centralized systems to manage and track individual training.

    In agreement with previously published data,30 our data suggest improved overall efficiency with the use of central IRBs, and the rationale for multiple sites reviewing clinical trial protocols through individual IRBs should be reconsidered. Such multicenter clinical trial protocols are often reviewed by several independent parties, including academic leaders, data monitoring committees, and regulatory agencies. As promoted by the National Institutes of Health, the use of a single IRB system that streamlines local IRB reviews for approved protocols32 can be a significant step toward enhancing regulatory efficiency while preserving ethical principles and participant protections.

    Contract negotiations and budget development have also become more complex, with common key areas of contract disagreement including terms of confidentiality, rights to data and publication, and intellectual property. Often, these negotiations occur repeatedly with subsequent protocols with the same pharmaceutical sponsor and academic group, which can cause delay. Such delays can be improved by using master service agreements and proactively establishing alternate language and form commitments for sponsors and sites during contract negotiations to reduce turnaround times on proposed changes.

    Though our data suggest modest improvement in trial start-up efficiency metrics over the past 15 years, more effort is needed to expedite this process while preserving individual stakeholder interests. All key parties in clinical research, including pharmaceutical companies, clinical researchers, participants, and regulatory agencies, have a vested interest in optimization, but there is often conflict and inertia that continue to make progress challenging. Pharmaceutical companies must ensure that trials can meet regulatory scrutiny while protecting shareholder value and pharmaceutical portfolios; academic institutions need specific research rights to protect institutional integrity and tax-exempt status, which requires rights to trial data sets and publication of results; and regulatory agencies are focused on drug safety and patient advocacy. Multiple collaborative approaches have ensued, including collegial forums, such as the Clinical Trial Transformation Initiative formed in 2007 by the Center for Drug Evaluation and Research.33 Broader use of master service agreements, streamlined protocol templates, and efforts such as the DCRI’s Rapid Start Up initiative, which uses prenegotiated, rapidly replicated contract language for key areas, can all serve to reduce delays and improve efficiency. Furthermore, although there is interest in globalization of clinical research by exporting research efforts to areas with fewer cost, regulation, and contractual barriers, there should be an equal effort in validating and enhancing North American trial operations metrics.34

    Our data from the top 10% of sites consistently show that sites can proceed through the site initiation procedures efficiently. However, additional work is needed to evaluate the specific measures that these sites take to achieve their level of productivity beyond the use of central IRBs. Large RCTs should continue to track and report their site metrics to provide objective data to promote an evidence-based operational plan and support trial design changes. Recent scientific efforts to curb the public health effects of the SARS-CoV-2 pandemic demonstrate the ability of large, complex systems to work efficiently across the sometimes diverging interests of various global stakeholders. The epidemic of cardiovascular disease is arguably of equal imminent importance. Although tremendous public financial and human resources are dedicated to improving its morbidity and mortality, a concerted and efficient system for implementing these resources has yet to be developed. In the coming years, clinical and regulatory communities will continue to require outcomes studies to identify beneficial therapies and drive integration of these interventions into clinical guidelines and practice. These data provide insight into how RCT design and development can be optimized to efficiently and effectively deliver the results needed for practicing evidence-based, clinical medicine.

    Limitations

    This study had several limitations. The included trials were inherently heterogenous with regard to enrollment environment (acute inpatient vs outpatient), time of follow-up, and area of cardiovascular research, which may have limited the strength of the observed comparisons. This limitation is mitigated some in that all trials were coordinated through a single academic research organization. Further, there were no direct comparisons for start-up metrics between sites that were managed by contract research organizations vs those managed by the sponsor or various site-specific features (eg, academic vs nonacademic or population density). The current study was also unable to account for the inherent delay associated with intermittent meetings of local IRB review committees, which disproportionately affects sites using this regulatory process. Additionally, sites that received protocols and contracts but did not enroll any patients represent significant cost and resource usage but were not captured in this study. Finally, given the publication delay, we were unable to include more contemporary and ongoing trials in the current study.

    Conclusions

    In this cohort study, research sites in North America that participated in large cardiovascular RCTs required nearly 9 months from time of initial contact to first participant enrollment, although these measures have modestly improved over time. The use of central IRBs may enhance RCT start-up efficiency, but more work is needed to ensure the timely implementation of a research protocol while protecting the interests of various stakeholders.

    Back to top
    Article Information

    Accepted for Publication: May 5, 2021.

    Published: July 23, 2021. doi:10.1001/jamanetworkopen.2021.17963

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Goyal A et al. JAMA Network Open.

    Corresponding Author: W. Schuyler Jones, MD, Duke Clinical Research Institute, Duke University School of Medicine, Durham, NC 27710 (schuyler.jones@duke.edu).

    Author Contributions: Drs Goyal and Jones had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Goyal, Granger, Alexander, Rao, Roe, Berdan, Reist, Mahaffey, Califf, Hernandez, Jones.

    Acquisition, analysis, or interpretation of data: Goyal, Schibler, Alhanti, Hannan, Blazing, Lopes, Alexander, Peterson, Green, Rorick, Berdan, Reist, Mahaffey, Harrington, Patel, Jones.

    Drafting of the manuscript: Goyal, Schibler.

    Critical revision of the manuscript for important intellectual content: Goyal, Alhanti, Hannan, Granger, Blazing, Lopes, Alexander, Peterson, Rao, Green, Roe, Rorick, Berdan, Reist, Mahaffey, Harrington, Califf, Patel, Hernandez, Jones.

    Statistical analysis: Goyal, Schibler, Alhanti, Jones.

    Obtained funding: Mahaffey, Jones.

    Administrative, technical, or material support: Goyal, Alhanti, Blazing, Green, Roe, Rorick, Berdan, Reist, Califf, Patel, Jones.

    Supervision: Alhanti, Roe, Mahaffey, Harrington, Patel, Jones.

    Conflict of Interest Disclosures: Dr Granger reported receiving personal fees from Boehringer Ingelheim and Bristol Myers Squibb/Pfizer outside the submitted work. Dr Lopes reported receiving personal fees from Bayer (consulting), Boehringer Ingelheim (consulting), Bristol Myers Squibb (consulting), Daiichi Sankyo (consulting), GlaxoSmithKline, Medtronic (consulting), Merck (consulting), Pfizer (consulting), Portola (consulting), and Sanofi (consulting) and grants from Pfizer, Bristol Myers Squibb, GlaxoSmithKline, Medtronic, and Sanofi outside the submitted work. Dr Alexander reported receiving personal fees from AbbVie Data Safety Monitoring Board and Pfizer (honoraria), consulting fees from Bristol Myers Squibb Institutional Research Grant, and grants from Bayer Institutional Research Grant and CSL Behring Institutional Research Grant outside the submitted work. Dr Peterson reported receiving grants from Amgen, Janssen, Bristol Myers Squibb, and Esperion and personal fees from Cerner (consulting) and Livongo (consulting) outside the submitted work. Dr Rao reported receiving research funding from Bayer Institutional and Shockwave Institutional during the conduct of the study. Dr Green reported receiving grants from Boehringer Ingelheim/Lilly Alliance, Sanofi/Lexicon, Merck, Roche, and GlaxoSmithKline and personal fees from Boehringer Ingelheim/Lilly Alliance, Sanofi, AstraZeneca, Pfizer, Hawthorne Effect, Regeneron, and Novo Nordisk outside the submitted work. Dr Harrington reported receiving grants for randomized clinical trial planning from Janssen and Bristol Myers Squibb during the conduct of the study; personal fees from Gilead Scientific (consulting) and SignalPath (board member, randomized clinical trial software) outside the submitted work; and serving on the Board of Directors (unpaid) for the American Heart Association. Dr Califf reported receiving an employee salary from Verily Life Sciences and Google Health and serving on the Cytokinetics Board and Centessa Board during the conduct of the study. Dr Patel reported receiving grants from AstraZeneca, Bayer, Janssen, Procyrion Inc, and Heart Flow and serving on the advisory board for Bayer, Janssen, Mytonomy, and Procyrion Inc outside the submitted work. Dr Hernandez reported receiving personal fees from AstraZeneca, Amgen, Bayer, Boston Scientific, Cytokinetics, Myokardia, and Bristol Myers Squibb and grants from Boehringer Ingelheim, American Regent, Verily, and Merck outside the submitted work. No other disclosures were reported.

    Funding/Support: The individual studies were funded by industry sponsors, and this analysis was supported by the Duke Clinical Research Institute.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We thank Karen Pieper, MS, and Terry Flora, BS, of the Duke Clinical Research Institute for their support in helping us access the necessary data for this study, as well as the dedicated sites and willing participants that made these projects possible. No compensation was received by the nonauthor contributors.

    References
    1.
    Jones  WS, Roe  MT, Antman  EM,  et al.  The changing landscape of randomized clinical trials in cardiovascular disease.   J Am Coll Cardiol. 2016;68(17):1898-1907. doi:10.1016/j.jacc.2016.07.781 PubMedGoogle ScholarCrossref
    2.
    Luce  BR, Kramer  JM, Goodman  SN,  et al.  Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change.   Ann Intern Med. 2009;151(3):206-209. doi:10.7326/0003-4819-151-3-200908040-00126 PubMedGoogle ScholarCrossref
    3.
    Eisenstein  EL, Lemons  PW  II, Tardiff  BE, Schulman  KA, Jolly  MK, Califf  RM.  Reducing the costs of phase III cardiovascular clinical trials.   Am Heart J. 2005;149(3):482-488. doi:10.1016/j.ahj.2004.04.049 PubMedGoogle ScholarCrossref
    4.
    Snyderman  R.  The clinical researcher—an “emerging” species.   JAMA. 2004;291(7):882-883. doi:10.1001/jama.291.7.882 PubMedGoogle ScholarCrossref
    5.
    Fanaroff  AC, Califf  RM, Lopes  RD.  High-quality evidence to inform clinical practice.   Lancet. 2019;394(10199):633-634. doi:10.1016/S0140-6736(19)31256-5 PubMedGoogle ScholarCrossref
    6.
    Fanaroff  AC, Califf  RM, Lopes  RD.  New approaches to conducting randomized controlled trials.   J Am Coll Cardiol. 2020;75(5):556-559. doi:10.1016/j.jacc.2019.11.043 PubMedGoogle ScholarCrossref
    7.
    Zwarenstein  M, Treweek  S, Gagnier  JJ,  et al; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group.  Improving the reporting of pragmatic trials: an extension of the CONSORT statement.   BMJ. 2008;337:a2390. doi:10.1136/bmj.a2390 PubMedGoogle ScholarCrossref
    8.
    Tunis  SR, Stryer  DB, Clancy  CM.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.   JAMA. 2003;290(12):1624-1632. doi:10.1001/jama.290.12.1624 PubMedGoogle ScholarCrossref
    9.
    Fanaroff  AC, Califf  RM, Windecker  S, Smith  SC  Jr, Lopes  RD.  Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008-2018.   JAMA. 2019;321(11):1069-1080. doi:10.1001/jama.2019.1122 PubMedGoogle ScholarCrossref
    10.
    Fanaroff  AC, Fudim  M, Califf  RM, Windecker  S, Smith  SC  Jr, Lopes  RD.  Levels of evidence supporting drug, device, and other recommendations in the American Heart Association/American College of Cardiology guidelines.   Am Heart J. 2020;226:4-12. doi:10.1016/j.ahj.2020.05.003 PubMedGoogle ScholarCrossref
    11.
    GUSTO investigators.  An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction.   N Engl J Med. 1993;329(10):673-682. doi:10.1056/NEJM199309023291001 PubMedGoogle ScholarCrossref
    12.
    Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO) IIa investigators.  Randomized trial of intravenous heparin versus recombinant hirudin for acute coronary syndromes.   Circulation. 1994;90(4):1631-1637. doi:10.1161/01.CIR.90.4.1631 PubMedGoogle ScholarCrossref
    13.
    Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO) IIb investigators.  A comparison of recombinant hirudin with heparin for the treatment of acute coronary syndromes.   N Engl J Med. 1996;335(11):775-782. doi:10.1056/NEJM199609123351103 PubMedGoogle ScholarCrossref
    14.
    Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO III) Investigators.  A comparison of reteplase with alteplase for acute myocardial infarction.   N Engl J Med. 1997;337(16):1118-1123. doi:10.1056/NEJM199710163371603 PubMedGoogle ScholarCrossref
    15.
    Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) Trial Investigators.  Inhibition of platelet glycoprotein IIb/IIIa with eptifibatide in patients with acute coronary syndromes.   N Engl J Med. 1998;339(7):436-443. doi:10.1056/NEJM199808133390704 PubMedGoogle ScholarCrossref
    16.
     International, randomized, controlled trial of lamifiban (a platelet glycoprotein IIb/IIIa inhibitor), heparin, or both in unstable angina. The PARAGON Investigators. Platelet IIb/IIIa Antagonism for the Reduction of Acute coronary syndrome events in a Global Organization Network.   Circulation. 1998;97(24):2386-2395. doi:10.1161/01.CIR.97.24.2386 PubMedGoogle ScholarCrossref
    17.
    Global Organization Network (PARAGON)-B Investigators.  Randomized, placebo-controlled trial of titrated intravenous lamifiban for acute coronary syndromes.   Circulation. 2002;105(3):316-321. doi:10.1161/hc0302.102573 PubMedGoogle ScholarCrossref
    18.
     Comparison of sibrafiban with aspirin for prevention of cardiovascular events after acute coronary syndromes: a randomised trial. The SYMPHONY Investigators. Sibrafiban versus Aspirin to Yield Maximum Protection from Ischemic Heart Events Post-acute Coronary Syndromes.   Lancet. 2000;355(9201):337-345. doi:10.1016/S0140-6736(99)11179-6 PubMedGoogle ScholarCrossref
    19.
    Ferguson  JJ, Califf  RM, Antman  EM,  et al; SYNERGY Trial Investigators.  Enoxaparin vs unfractionated heparin in high-risk patients with non–ST-segment elevation acute coronary syndromes managed with an intended early invasive strategy: primary results of the SYNERGY randomized trial.   JAMA. 2004;292(1):45-54.PubMedGoogle Scholar
    20.
    Armstrong  PW, Granger  CB, Adams  PX,  et al; APEX AMI Investigators.  Pexelizumab for acute ST-elevation myocardial infarction in patients undergoing primary percutaneous coronary intervention: a randomized controlled trial.   JAMA. 2007;297(1):43-51. doi:10.1001/jama.297.1.43 PubMedGoogle Scholar
    21.
    Pfeffer  MA, McMurray  JJ, Velazquez  EJ,  et al; Valsartan in Acute Myocardial Infarction Trial Investigators.  Valsartan, captopril, or both in myocardial infarction complicated by heart failure, left ventricular dysfunction, or both.   N Engl J Med. 2003;349(20):1893-1906. doi:10.1056/NEJMoa032292 PubMedGoogle ScholarCrossref
    22.
    Cannon  CP, Blazing  MA, Giugliano  RP,  et al; IMPROVE-IT Investigators.  Ezetimibe added to statin therapy after acute coronary syndromes.   N Engl J Med. 2015;372(25):2387-2397. doi:10.1056/NEJMoa1410489PubMedGoogle ScholarCrossref
    23.
    Patel  MR, Mahaffey  KW, Garg  J,  et al; ROCKET AF Investigators.  Rivaroxaban versus warfarin in nonvalvular atrial fibrillation.   N Engl J Med. 2011;365(10):883-891. doi:10.1056/NEJMoa1009638PubMedGoogle ScholarCrossref
    24.
    O’Connor  CM, Starling  RC, Hernandez  AF,  et al.  Effect of nesiritide in patients with acute decompensated heart failure.   N Engl J Med. 2011;365(1):32-43. doi:10.1056/NEJMoa1100171PubMedGoogle ScholarCrossref
    25.
    Tricoci  P, Huang  Z, Held  C,  et al; TRACER Investigators.  Thrombin-receptor antagonist vorapaxar in acute coronary syndromes.   N Engl J Med. 2012;366(1):20-33. doi:10.1056/NEJMoa1109719PubMedGoogle ScholarCrossref
    26.
    Green  JB, Bethel  MA, Armstrong  PW,  et al; TECOS Study Group.  Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes.   N Engl J Med. 2015;373(3):232-242. doi:10.1056/NEJMoa1501352PubMedGoogle ScholarCrossref
    27.
    Holman  RR, Bethel  MA, Mentz  RJ,  et al; EXSCEL Study Group.  Effects of once-weekly exenatide on cardiovascular outcomes in type 2 diabetes.   N Engl J Med. 2017;377(13):1228-1239. doi:10.1056/NEJMoa1612917PubMedGoogle ScholarCrossref
    28.
    Schwartz  GG, Steg  PG, Szarek  M,  et al; ODYSSEY OUTCOMES Committees and Investigators.  Alirocumab and cardiovascular outcomes after acute coronary syndrome.   N Engl J Med. 2018;379(22):2097-2107. doi:10.1056/NEJMoa1801174PubMedGoogle ScholarCrossref
    29.
    Hiatt  WR, Fowkes  FG, Heizer  G,  et al; EUCLID Trial Steering Committee and Investigators.  Ticagrelor versus clopidogrel in symptomatic peripheral artery disease.   N Engl J Med. 2017;376(1):32-40. doi:10.1056/NEJMoa1611688PubMedGoogle ScholarCrossref
    30.
    Krafcik  BM, Doros  G, Malikova  MA.  A single center analysis of factors influencing study start-up timeline in clinical trials.   Future Sci OA. 2017;3(4):FSO223. doi:10.4155/fsoa-2017-0025 PubMedGoogle Scholar
    31.
    Abu-Shaheen  A, Al Badr  A, Al Fayyad  I, Al Qutub  A, Faqeih  EA, Al-Tannir  M.  Streamlining and cycle time reduction of the startup phase of clinical trials.   Trials. 2020;21(1):115. doi:10.1186/s13063-020-4079-8 PubMedGoogle ScholarCrossref
    32.
    National Institutes of Health. Single IRB policy for multi-site research. Accessed November 11, 2020. https://grants.nih.gov/policy/humansubjects/single-irb-policy-multi-site-research.htm
    33.
    Clinical Trials Transformation Initiative. Who we are. Accessed November 1, 2020. https://www.ctti-clinicaltrials.org/who-we-are
    34.
    Glickman  SW, McHutchison  JG, Peterson  ED,  et al.  Ethical and scientific implications of the globalization of clinical research.   N Engl J Med. 2009;360(8):816-823. doi:10.1056/NEJMsb0803929 PubMedGoogle ScholarCrossref
    ×