One hundred fifteen estimates were included from the Global Burden of Disease database. Data for each stroke type are presented separately.
One hundred fifteen trials were included. AIS indicates acute ischemic stroke; ICH, intracerebral hemorrhage; SAH, subarachnoid hemorrhage; TIA, transient ischemic attack.
eMethods. Variable Definitions and Sensitivity Analyses
eFigure 1. Forest plot with the random effects, pooled enrolled enrollment disparity difference (EDD) for all trials (n=106) and for trial subgroups when subarachnoid hemorrhage trials were excluded.
eFigure 2. Temporal trend by publication date in the representation of women in randomized controlled trials of acute stroke with subarachnoid hemorrhage trials included (n=115).
eFigure 3. Temporal trend by publication date in the representation of women in randomized controlled trials of acute stroke with subarachnoid hemorrhage trials dropped (n=106).
eAppendix. Sensitivity Analyses.
eFigure 4. Sensitivity analysis using Global Burden of Disease data to account for the representation of women in the underlying stroke populations.
eFigure 5. Sensitivity analysis using published incidence data to account for the representation of women in the underlying stroke populations.
eFigure 6. Forest plot with the random effects, pooled participation to prevalence ratio (PPR) for all trials and for trial subgroups, estimated as part of a sensitivity analysis (n=115 trials).
Customize your JAMA Network experience by selecting one or more topics from the list below.
Strong B, Pudar J, Thrift AG, et al. Sex Disparities in Enrollment in Recent Randomized Clinical Trials of Acute Stroke: A Meta-analysis. JAMA Neurol. 2021;78(6):666–677. doi:10.1001/jamaneurol.2021.0873
Relative to their representation in underlying stroke populations, are women underenrolled in contemporary randomized clinical trials of acute stroke therapies?
In this meta-analysis of 115 acute stroke trials published in the last decade, relative to their representation in underlying stroke populations, women were underenrolled by 5.3 percentage points across all studies. The use of an upper age limit of 80 years as an exclusion criterion was associated with significantly less enrollment of women after multivariable adjustment.
Per these findings, further efforts including changes to eligibility criteria are needed to ensure increased participation of women in acute stroke trials.
The underenrollment of women in randomized clinical trials represents a threat to the validity of the evidence supporting clinical guidelines and potential disparities in access to novel treatments.
To determine whether women were underenrolled in contemporary randomized clinical trials of acute stroke therapies published in 9 major journals after accounting for their representation in underlying stroke populations.
MEDLINE was searched for acute stroke therapeutic trials published between January 1, 2010, and June 11, 2020.
Eligible articles reported the results of a phase 2 or 3 randomized clinical trial that enrolled patients with stroke and/or transient ischemic attack and examined a therapeutic intervention initiated within 1 month of onset.
Data extraction was performed by 2 independent authors in duplicate. Individual trials were matched to estimates of the proportion of women in underlying stroke populations using the Global Burden of Disease database.
Main Outcomes and Measures
The primary outcome was the enrollment disparity difference (EDD), the absolute difference between the proportion of trial participants who were women and the proportion of strokes in the underlying disease populations that occurred in women. Random-effects meta-analyses of the EDD were performed, and multivariable metaregression was used to explore the associations of trial eligibility criteria with disparity estimates.
The search returned 1529 results, and 115 trials (7.5%) met inclusion criteria. Of 121 105 randomized patients for whom sex was reported, 52 522 (43.4%) were women. The random-effects summary EDD was −0.053 (95% CI, −0.065 to −0.040), indicating that women were underenrolled by 5.3 percentage points. This disparity persisted across virtually all geographic regions, intervention types, and stroke types, apart from subarachnoid hemorrhage (0.117 [95% CI, 0.084 to 0.150]). When subarachnoid hemorrhage trials were excluded, the summary EDD was −0.067 (95% CI, −0.078 to −0.057). In the multivariable metaregression analysis, an upper age limit of 80 years as an eligibility criterion was associated with a 6–percentage point decrease in the enrollment of women.
Conclusions and Relevance
Further research is needed to understand the causes of the underenrollment of women in acute stroke trials. However, to maximize representation, investigators should avoid imposing age limits on enrollment.
The inadequate enrollment of women in randomized clinical trials (RCTs) has been a long-standing issue in clinical medicine.1 Despite efforts to increase their representation, recent analyses provide evidence that disparities persist in the participation of women in clinical trials of various cardiovascular diseases, including stroke.2-9 In the past decade, significant progress in acute stroke therapy has been made, including treatment of large-vessel occlusions with mechanical thrombectomy,10-12 reduction in the risk of recurrent stroke with dual antiplatelet therapy,13,14 and advances in neuroprotection.15 However, given the well-documented importance of sex in the epidemiology and pathophysiology of stroke,16 the possible underenrollment of women in these and other stroke trials represents a threat to their generalizability and in turn to the validity of the evidence base with regards to the treatment of women. It also introduces the potential for unequal access to novel treatments.
Furthermore, previous studies of enrollment disparities in stroke trials were limited by inadequate consideration of the representation of women in the underlying stroke populations,2-4 which is likely to vary by geographic location and stroke type.17 These studies also did not investigate associations between disparity measures and eligibility criteria that are likely to affect the enrollment of women. Thus, the objectives of this study were to (1) determine, after accounting for the representation of women in the underlying disease populations, whether a sex disparity in enrollment exists in acute stroke RCTs published in major clinical journals in the last decade and (2) explore whether the magnitude of the sex disparity is associated with eligibility criteria that could differentially affect the participation of men and women.
Working with an experienced medical librarian to develop the initial set of search terms, we searched MEDLINE (PubMed) for RCTs published between January 1, 2010, and June 11, 2020, using the following query: (Stroke OR “Stroke” [MeSH]) AND acute AND (“Randomized Controlled Trial” [Publication Type] OR “Randomized Controlled Trials as Topic” [MeSH] OR “Controlled Clinical Trial” [Publication Type] OR randomized [TIAB] OR randomized [TIAB] OR randomly [TIAB] OR Trial*). Because this strategy did not capture all hemorrhagic stroke trials, we conducted additional searches using the MeSH terms “cerebral hemorrhage” and “subarachnoid hemorrhage.” Eligible studies were those meeting the following criteria:
Final primary results of a phase 2 or 3 RCT design with a planned sample size of 100 patients or more. Cluster RCTs and noninferiority trials were included.
The trial population was patients with acute stroke of any type (ie, acute ischemic stroke [AIS], transient ischemic attack [TIA], intracerebral hemorrhage [ICH], or subarachnoid hemorrhage [SAH]) enrolled within 1 month of ictus.
Treatment (either pharmacological or nonpharmacological) compared with a control (placebo, usual care, or nonstandard comparator) was initiated within 1 month of stroke onset. Trials testing detection methods, educational interventions, or techniques for improvement of quality of care were excluded. Secondary prevention trials were included if treatment was initiated within 1 month of onset. For simplicity, they are also described as trials of acute stroke therapies.
Published in 1 of 9 major journals (BMJ, Circulation, JAMA, JAMA Neurology, Lancet, Lancet Neurology, NEJM, Neurology, and Stroke) in the last 10 years (January 2010-June 2020). Unpublished trials were not included.
The reference lists of all articles ultimately deemed eligible were also manually reviewed to identify any additional relevant RCTs (ie, snowballing). We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines18 to report our findings. This meta-analysis was not registered.
Two authors (B.S. and J.P.) independently screened the title and abstract of each result and determined whether it was eligible for full-text review. For those studies that passed the initial screening, final determinations of eligibility were also done in duplicate based on the full-text article. All eligible trials then underwent duplicate, independent data abstraction. Discrepancies in study eligibility determinations or data abstractions were resolved by consensus and, when necessary, by consultation with the senior author (M.J.R.). The following characteristics were abstracted from eligible trials: the number of enrolled women; number of enrolled men; number of enrolling clinical sites; period of enrollment; geographic location of patient enrollment or clinical sites; enrollment numbers for each country or region; details of the experimental and control interventions (intervention type); primary end point; preplanned and final sample size; involvement of industry; mean or median age of participants; enrollment of different stroke types (frequencies); whether 1 or more women were represented in trial leadership; and eligibility criteria pertaining to age, stroke severity, prestroke disability, time from onset, qualification for specific medical treatments (ie, endovascular therapy [EVT] and intravenous thrombolysis [IVT]), and qualifying comorbidities (eg, hypertension, diabetes). These eligibility criteria were selected based on the assumption that they could potentially affect enrollment by sex. Because treatment efficacy was not under study, we did not assess risk of bias. More detailed variable definitions are provided in the eMethods in the Supplement.
To determine the representation of women in underlying stroke populations, the Global Burden of Disease (GBD) 2017 database (https://gbd2017.healthdata.org/gbd-search/), which provides data for individual countries, as well as administrative units within some nations, was queried.17 Individual trials were matched to GBD incidence data on the basis of their included stroke type, geographic area (the smallest area or region in which >80% of trial enrollment took place), and the middle year of the trial enrollment period. We abstracted the number of incident strokes in women and associated 95% uncertainty interval, as well as the number of incident strokes in men and associated 95% uncertainty interval.
To quantify the enrollment of women, we used a modified version of the enrollment disparity difference (EDD), a quantitative measure previously developed to characterize enrollment disparities in RCTs of lung cancer therapies.19 For each RCT, we first calculated the proportion of trial participants who were women (PPW). We then calculated the proportion of strokes occurring in women in the underlying stroke populations (PSW) using GBD data. We defined the EDD as the difference between the PPW and PSW (ie, EDD = PPW – PSW), such that a negative EDD value is indicative of an underenrollment of women (and a positive EDD value indicates overenrollment).
To quantify the imprecision of the EDD estimates, we computed the standard error (SE) of the EDD by estimating the SEs of the PPW and PSW and performing the following calculation:
For the PPW, we estimated the SE using the standard formula for the SE of a proportion, where n is the number of patients enrolled in the trial:
However, for the PSW, the number of men and women with stroke were estimates, which brought additional uncertainty. To account for this, we used the number of men and women with stroke and the associated 95% uncertainty intervals reported in the GBD database to fit γ distributions for each region-specific, stroke-specific, and time-specific estimate. We subsequently drew 100 000 samples from each of the γ distributions, computed the corresponding PSW, and used the PSWs from these samples to estimate the SE.
We conducted a random-effects meta-analysis of the EDDs of individual trials to summarize the overall sex disparity among trials of acute stroke. A random-effects model, rather than assuming a single true estimate of the outcome, calculates the mean from what is assumed to be a distribution of estimates20 and is appropriate for quantifying enrollment disparities, which are likely to vary substantially between trials because of variability in the composition of underlying clinical populations and specification of inclusion and exclusion criteria. Heterogeneity was quantified with the use of the I2 statistic.21 Subgroup analyses were performed for the following trial characteristics: geographic region (Americas [mostly North America], Asia Pacific, Europe, or multiregion), stroke type (AIS or TIA, ICH, mixed ischemic and hemorrhagic strokes, or SAH), sample size (<250 patients, 250-750 patients, or >750 patients), industry involvement, intervention type (EVT; IVT; secondary prevention, if initiated within 30 days of onset; surgery; or other), and representation of 1 or more women in a leadership role. Temporal trends in trial enrollment were examined by fitting a metaregression model of the EDDs, with year of publication as the independent variable.
To investigate the association of various trial eligibility criteria with the enrollment of women, we conducted a random-effects multivariable metaregression of study-specific EDD estimates using the Knapp-Hartung modification to variance estimation.22 We assessed a prespecified list of eligibility criteria as potential covariates to minimize the probability of a type 1 error, as recommended by Thompson and Higgins.23 The following eligibility criteria were considered independent variables: the highest permitted age of participants, the highest permitted prestroke-modified Rankin Scale score, longest permitted time from stroke onset to a clinically important event (eg, enrollment, treatment, arrival), eligible range of stroke severity (ie, both mild and severe strokes, only mild strokes, or only severe strokes included), a requirement for IVT eligibility, a requirement for EVT eligibility, and a requirement for presentation with hypertension (the most common qualifying comorbidity). More detailed definitions of these variables are provided in the eMethods in the Supplement. Eligibility criteria that were associated with the EDD at P < .25 were selected for inclusion in the multivariable metaregression model. To ensure parsimony, independent variables that were not significant (ie, P > .05) and had only a minimal influence on other covariates were dropped from the model. No adjustment was made for multiple comparisons. We used the metan package in Stata version 16.1 (StataCorp) to perform the meta-analysis and the metareg package to conduct the metaregression.
To verify that our conclusions were not dependent on the use of GBD data, we conducted a sensitivity analysis using an alternative source of data to estimate the PSW. We matched individual trials with greater than 80% case enrollment in a single country to published incidence studies that were conducted in the same country within 10 years of the trial enrollment period. Trials with largely multinational enrollment were therefore excluded from this analysis. We performed a random-effects meta-analysis on trials with which an incidence study could be matched and compared the pooled EDD using GBD data to the pooled EDD using published incidence data. Since several previous studies reporting on sex differences in trial enrollment calculated the participation-to-prevalence ratio,3,5,9 we also conducted a sensitivity analysis of all trials using the participation-to-prevalence ratio instead of the EDD. Methods for calculating this ratio are described in the eMethods of the Supplement. The analytic code and data used in the analysis are publicly available at the Open Science Framework (https://osf.io/fht2u/).
Of 1529 total search results, 115 (7.5%)11-15,24-133 were deemed eligible and included in the meta-analysis (Figure 1). The characteristics of these 115 trials are presented in Table 1. Overall, the trials provided data on the sex of 121 105 randomized patients with acute stroke or TIA, 52 522 of whom were women (43.4%). The PPW ranged from 22.4% to 79.0% (median, 45.0% [interquartile range (IQR), 39.0%-50.1%]). There was also large variation in the GBD-based PSW estimates that were matched to the 115 individual stroke trials; PSW estimates ranged from 41.4% (China; 2014; ICH) to 67.2% (Japan; 2011; SAH) with a median of 50.8% (IQR, 49.1%-53.3%).17 The distribution of PSWs for each stroke type is presented in Figure 2.
The random-effects pooled EDD of the 115 trials was −0.053 (95% CI, −0.065 to −0.040), indicating that women were underenrolled by an absolute difference of 5.3 percentage points relative to their representation in the underlying stroke populations. However, there was substantial variability between trials in the participation of women (I2, 84.4%). Results of the subgroup analyses are presented in Figure 3. Stroke type was identified as a significant source of heterogeneity: the summary EDD for trials enrolling SAHs was 0.117 (95% CI, 0.084-0.150); this highly positive EDD estimate indicates that women were considerably overenrolled in these 9 studies. Conversely, among the remaining 106 trials, women were underenrolled by approximately 6.7 percentage points relative to the underlying stroke populations (summary EDD, −0.067 [95% CI, −0.078 to −0.057]) (eFigure 1 in the Supplement). The largest sex disparity was observed among trials of secondary prevention therapies,13,14,24,45,46,61,72,73,125,130,132 which showed a large, negative EDD value (summary EDD, −0.117 [95% CI, −0.148 to −0.085]). When temporal trends were investigated using a univariable metaregression model, there was a statistically significant decrease in the representation of women over time (a decrease of 0.005 in the EDD per year [95% CI, −0.010 to 0.000 per year]; P = .05) (eFigure 2 in the Supplement). However, this trend was likely the result of the publication of most of the SAH trials,93,99,101,114,116,117,126,133 which had a significantly higher mean participation of women, in 2015 or earlier. The association of time with the EDD was greatly attenuated and became nonsignificant after adjustment for stroke type and after excluding the 9 SAH trials47,93,99,101,114,116,117,126,133 from the analysis (eFigure 3 in the Supplement).
In the main sensitivity analysis (eAppendix in the Supplement), 49 of the 115 trials were matched to a published incidence study from a single country. Among these 49 studies,11-14,25,27,28,30,31,35,38,40,43-46,48,50,52,55,62,65-68,71,74,75,78,84,88,89,91,96,99-101,105-108,111,112,114,118,119,121,124,131 the pooled, random-effects EDD when the representation of women in the underlying stroke populations (PSW) was based on GBD data was −0.056 (95% CI, −0.077 to −0.035). When estimates of the PSW were calculated using the published incidence studies, the disparity was slightly larger (summary EDD, −0.072 [95% CI, −0.088 to −0.055]), and women were no longer overrepresented in SAH trials. Further information on the results of this sensitivity analysis is provided in eFigures 4 and 5 in the Supplement. When the disparity was quantified as a ratio, the summary participation to prevalence ratio for all 115 trials was 0.89 (95% CI, 0.87-0.92) (eFigure 6 in the Supplement).
When we examined the association of eligibility criteria with the EDD using metaregression, we were only able to include the 80 trials11-15,24-34,36,37,40-42,44-46,48-51,53,55,56,58,59,61-64,67,71-75,77-80,82,84-87,89,90,94-98,103-105,107-109,111,113,115,118-121,123-125,128-130,132 enrolling patients with AIS and TIA, because there was insufficient variation in the eligibility criteria among trials of other stroke types (ie, ICH, SAH, and mixed).35,38,39,43,47,52,54,57,60,65,66,68,70,76,81,83,88,91-93,99-102,106,110,112,114,116,117,122,126,127,131,133 Only the imposition of an upper age limit and requirements for IVT and EVT eligibility were significantly associated with the EDD in the final model, but prestroke disability and limits on stroke severity were retained because they had an influence on the magnitude of the other independent variables (Table 2). The imposition of an upper age limit of 80 years or younger was associated with a 6–percentage point decrease in the enrollment of women, while requirements for IVT eligibility and EVT eligibility were associated with 3–percentage point increases in the enrollment of women after adjustment.
In this analysis of 115 acute stroke trials published between 2010 and 2020 in 9 highly cited medical journals, we found that women were underenrolled by approximately 5 percentage points relative to their representation in underlying disease populations. This finding is especially concerning given that women, on average, experience slightly more incident events than men in the underlying stroke populations, and it indicates that greater efforts are needed to increase their enrollment. The systematic underrepresentation of women in acute stroke trials also raises questions about the representativeness and generalizability of trial results to women. Furthermore, we note that a statistically significant disparity against women was seen in virtually all sensitivity and subgroup analyses. The largest disparity was observed among trials of secondary prevention therapies; while there are likely several factors that contribute to this, the lower representation of women may be partially explained by the exclusion of patients with indications of a stroke of cardioembolic origin from many of the antiplatelet-based secondary prevention trials. Stroke of cardioembolic origin is more common in women,134 and this exclusion likely has a disproportionate effect on them.
In the metaregression analysis, we found that the imposition of an upper age limit on eligibility of 80 years or younger was associated with a 6–percentage point decrease in the representation of women after adjustment. The older age of women at first stroke is a well-documented phenomenon; in a 2009 meta-analysis134 of 59 stroke incidence studies, women were a mean of 4.3 years older than men at the time of their first event. Consequently, upper age limits placed on trial enrollment exclude more women. An illustration of this phenomenon is provided by an analysis of patients with AIS in a large German registry, which showed that an upper age limit of 80 years in a hypothetical trial would exclude 19% of men but 44% of women.135 These findings suggest that an upper age limit on trial enrollment must be medically necessary to be justified. When an upper age limit is being considered, trial investigators should first contemplate whether age is serving as a proxy for some other age-associated characteristic, such as frailty, fall risk, or bleeding risk. In such cases, it may be possible to replace the age exclusion with an exclusion based on a valid risk assessment tool.136
Somewhat surprisingly, we also found that enrollment requirements for IVT and EVT eligibility were associated with a higher representation of women. Our finding with respect to IVT may be reflective of the facts that women are more likely to have a severe stroke137 and mild severity is a common reason for nontreatment with thrombolytic therapy.138 Regarding EVT eligibility, in an analysis of a German administrative database with more than 1 million observations, women were more likely to receive mechanical thrombectomy,139 which aligns with the findings of our metaregression.
We should note, however, that our metaregression model was unable to explain two-thirds of the between-trial variance. This indicates that factors apart from eligibility criteria affect the participation of women in these trials and contribute to their underrepresentation. One factor that may adversely affect the enrollment of women is their willingness to participate. Although not always a consistent finding,140 there is evidence that women are less likely to consent to inclusion in stroke trials than men,141,142 which may be the consequence of a higher perceived risk of trial participation and greater risk aversion among women.143,144 There may also be other factors operating in the prescreening phase that differentially affect the probability of a male or female patient with stroke being screened.3,9 These factors could reflect either a bias in trial design, such as the inclusion of clinical sites that see mostly male patients (such as US Veterans Health Administration facilities) or perhaps differences in the likelihood of hospitalization following stroke. For example, a prior analysis of stroke registry data indicates that elderly women who live in nursing homes are less likely to be admitted to acute care hospitals.145 Ultimately, more high-quality data are needed from screening logs to determine whether women are less likely to be approached for participation in the context of acute stroke trials.
There are several limitations to this study. To begin, our finding that SAH trials overenrolled women by a significant margin should be approached with caution, since we found that this estimate was affected by the source of data used to generate the PSW (eFigures 4 and 5 in the Supplement). Subarachnoid hemorrhage is a relatively rare disease, and a well-designed population-based incidence study in Oxfordshire, United Kingdom, registered only 20 incident events during a 3-year period.146 Consequently, estimates of the PSW in the underlying SAH populations are likely to be imprecise. Second, conclusions drawn from the metaregression model are limited by the potential for residual confounding at either the trial or patient level because of the ecological nature of the analysis.23 Although our findings in conjunction with previous work provide a strong justification to eliminate upper age limits on enrollment, analyses of individual patient screening and recruitment data are needed to fully elucidate the role of these and other factors in the underenrollment of women. Unfortunately, such analyses may not be insightful in practice because of a lack of standardization of screening procedures across clinical sites.147 Our ability to classify trial eligibility criteria, such as stroke severity, was also quite limited and affected by the lack of an accepted definition for mild or severe strokes. Lastly, we only included trials published in 9 major clinical journals and excluded those with a planned sample size of less than 100 patients, which may limit the generalizability of our findings to other acute stroke trials. However, major clinical trials affecting acute stroke care are most likely to be published in these 9 journals.
In summary, we found that women were underenrolled in acute stroke trials published in the last decade by 5 percentage points relative to their representation in underlying stroke populations. This disparity was present for every stroke type except trials enrolling SAHs, which showed an overenrollment of women. We also found that the presence of an upper age limit of 80 years on eligibility was significantly associated with a lower representation of women. Thus, authors imposing an upper age limit as an eligibility criterion should provide a clear rationale for doing so. Further study is needed to monitor the magnitude of sex disparities in clinical trial enrollment, determine what modifications to trial design can feasibly be made to increase the participation of women, and understand the patient-level factors that may contribute to women being less likely to enroll. It will also be important to determine the clinical relevance and impact of the observed enrollment disparities on the validity, representativeness, and generalizability of trial data.
Accepted for Publication: February 19, 2021.
Published Online: April 26, 2021. doi:10.1001/jamaneurol.2021.0873
Corresponding Author: Mathew J. Reeves, BVSc, PhD, Department of Epidemiology and Biostatistics, College of Human Medicine, Michigan State University, B601 West Fee Hall, East Lansing, MI 48824 (email@example.com).
Author Contributions: Mr Strong and Dr Reeves had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Strong, Thrift, Reeves.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Strong.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Strong, de los Campos.
Administrative, technical, or material support: Howard, Reeves.
Conflict of Interest Disclosures: None reported.
Meeting Presentation: The work contained in this manuscript and the work leading up to it have been presented at the International Stroke Conference 2020; February 20, 2020; Los Angeles, California; the European Stroke Organisation–World Stroke Organization Conference 2020; November 7, 2020; virtual; and the International Stroke Conference 2021; March 17, 2021; virtual.
Additional Contributions: We would like to thank to Iris Kovar-Gough, MLIS, Michigan State University, for her assistance in developing the search strategy. No compensation was received for this work.