Key PointsQuestion
What percentage of patients with schizophrenia in the real world are represented in randomized clinical trials (RCTs) and do their outcomes differ from those not represented in RCTs?
Findings
In this study of 25 259 real-world individuals with diagnoses of schizophrenia spectrum disorders recorded in national patient registries in Finland and Sweden, about a fifth were represented in RCTs and their outcomes were better than of those individuals with schizophrenia not meeting RCT inclusion criteria.
Meaning
Future research should consider the heterogeneity of individuals with schizophrenia and the patient groups typically ineligible for participation; RCTs may become more inclusive by representing a broader spectrum of individuals with schizophrenia and by targeting specific currently underrepresented groups.
Importance
Most evidence about efficacy and safety of antipsychotics in schizophrenia spectrum disorders relies on randomized clinical trials (RCTs). However, owing to their strict eligibility criteria, RCTs represent only a part of the real-world population (ie, unselected patients seen in everyday clinical practice), which may result in an efficacy-effectiveness gap.
Objective
To quantify the proportion of real-world individuals with schizophrenia spectrum disorders who would be ineligible for participation in RCTs, and to explore whether clinical outcomes differ between eligible and ineligible individuals.
Design, Setting, and Participants
This study applied eligibility criteria typically used in RCTs for relapse prevention in schizophrenia spectrum disorders to real-world populations. Individuals with diagnoses of schizophrenia spectrum disorders recorded in national patient registries in Finland and Sweden were identified. Individuals who had used antipsychotics continuously for 12 weeks in outpatient care were selected. Individuals were followed up for up to 1 year while they were receiving maintenance treatment with any second-generation antipsychotic (excluding clozapine). Follow-up was censored at treatment discontinuation, initiation of add-on antipsychotics, death, and end of database linkage.
Main Outcomes and Measures
Proportions of RCT-ineligible individuals with schizophrenia spectrum disorders owing to any and specific RCT exclusion criteria. The risk of hospitalization due to psychosis within 1-year follow-up in ineligible vs eligible persons were compared using hazard ratios (HR) and corresponding 95% CIs.
Results
The mean (SD) age in the Finnish cohort (n = 17 801) was 47.5 (13.8) years and 8972 (50.4%) were women; the mean (SD) age in the Swedish cohort (n = 7458) was 44.8 (12.5) years and 3344 (44.8%) were women. A total of 20 060 individuals (79%) with schizophrenia spectrum disorders would be ineligible for RCTs (Finnish cohort: 14 221 of 17 801 [79.9%]; Swedish cohort: 5839 of 7458 [78.3%]). Most frequent reasons for ineligibility were serious somatic comorbidities and concomitant antidepressant/mood stabilizer use. Risks of hospitalization due to psychosis was higher among ineligible than eligible individuals (Finnish cohort: 18.4% vs 17.2%; HR, 1.14 [95% CI, 1.04-1.24]; Swedish cohort: 20.1% vs 14.8%; HR, 1.47 [95% CI, 1.28-1.92]). The largest risks of hospitalization due to psychosis were observed in individuals ineligible owing to treatment resistance, tardive dyskinesia, and history of suicide attempts. Finally, with more ineligibility criteria met, larger risks of hospitalization due to psychosis were observed in both countries.
Conclusions and Relevance
RCTs may represent only about a fifth of real-world individuals with schizophrenia spectrum disorders. Underrepresented (ineligible) patients with schizophrenia spectrum disorders have moderately higher risks of admission due to psychosis while receiving maintenance treatment than RCT-eligible patients. These findings set the stage for future studies targeting real-world populations currently not represented by RCTs.
Most evidence about efficacy and safety of medical treatments is based on randomized clinical trials (RCTs), which are highly standardized, systematic studies. RCT outcomes (efficacy) may differ from the utility of interventions in routine clinical practice (effectiveness), in what has been termed the efficacy-effectiveness gap. Efficacy-effectiveness gaps have been identified in several health care areas, including pneumology,1,2 oncology,3 infectology,4 and internal medicine5; nonpharmacological interventions in psychology6; and antidepressants.7
A possible efficacy-effectiveness gap in effectiveness and safety of antipsychotics in individuals with schizophrenia, which, to our knowledge, has not been investigated so far, may stem from the strict exclusion criteria applied in typical RCTs aiming at marketing approval. Therefore, a broad and diverse set of individuals is excluded from these trials, such as those experiencing suicidal ideations, substance use disorders, or somatic and psychiatric comorbidities. Such excluded people may have different courses of illness and possibly also different treatment outcomes.
Here, we aimed to quantify the real-world population (ie, unselected patients seen in everyday clinical practice) not directly represented in RCTs, ie, those who are ineligible owing to any RCT exclusion criteria, as well as the real-world populations ineligible owing to specific exclusion criteria. Moreover, we assessed whether there are differences in key outcomes between individuals who are potentially eligible and those who are not (overall and for specific RCT exclusion criteria). To answer these research questions, we analyzed data from real-world populations in 2 nationwide registries.
In this analysis, we simulated the application of typical inclusion and exclusion criteria of RCTs conducted in individuals with schizophrenia (eAppendix 1 in the Supplement) to the national patient registries of Finland and Sweden. The protocol for our analysis was registered on the Open-Science Framework prior to analysis on September 15, 2020,8 and we complied with the Reporting of Studies Conducted Using Observational Routinely-Collected Data (RECORD) reporting guideline (eAppendix 2 in the Supplement).9 The Regional Ethics Board of Stockholm approved this research project (decision 2007/762–31). Permissions were also granted by pertinent institutional authorities at the Finnish National Institute for Health and Welfare (permission THL/847/5.05.00/2015), the Social Insurance Institution of Finland (65/522/2015), and Statistics Finland (TK53-1042-15). The study was registry based, and no contact was made with the participants of the study; therefore, according to legislation in both countries, obtaining informed consent from participants was not required.
Real-World Databases Used and Follow-up
We had access to the data extracted from the national patient registries in Finland (January 2005-December 2017) and Sweden (January 2006-December 2016) (eAppendix 1A in the Supplement includes details about the cohorts), which here represent real-world individuals with schizophrenia and schizoaffective disorder (referred to from here on as schizophrenia). Pseudonymized data were originally extracted by the register maintainers via personal identity codes, which enable data linkage between registries of both countries. Personal identity codes were replaced with research identity codes before data were shared with the researchers. We chose 2 registries to assess similarity in findings across countries and reduce the likelihood of chance findings.
In both registries, we first focused on individuals hospitalized at least once owing to schizophrenia and who used second-generation antipsychotics at the start of follow-up because those are the typical interventions in modern RCTs10 and also the most used antipsychotics in Finland and Sweden nowadays.11 We did not consider individuals using clozapine or first-generation antipsychotics because the former is not a first-line treatment but reserved for treatment resistance (here defined as clozapine or electroconvulsive therapy treatment, reported ever before follow-up), and the latter are only rarely used in real-world clinical practice in Finland and Sweden.11 Continuous medication use was derived using the PRE2DUP method from dispensed prescriptions.12
The duration of follow-up was 12 months as this is a typical duration of relapse-prevention RCTs,10 with time zero defined as when the inclusion criteria were fulfilled, ie, after 12 weeks of continuous antipsychotic use in monotherapy as an outpatient. We chose this 12-week criterion to ensure clinical stability of schizophrenia in maintenance treatment with antipsychotics, which is a starting point for relapse prevention trials. We censored follow-up at discontinuation of antipsychotics, hospitalization (other types than analyzed as outcome event), death, after the defined follow-up time, and end of data linkage. For details about patient involvement and data sharing options, see the eMethods in the Supplement.
Outcomes and Statistical Analyses
By applying to these databases the standard RCT inclusion and exclusion criteria mentioned in the eMethods (eAppendix 1A) in the Supplement, we defined populations consisting of:
Individuals potentially eligible for standard RCTs about relapse prevention with antipsychotics (ie, meeting all inclusion criteria but having none of the exclusion criteria);
Individuals ineligible for such an RCT for any reason (ie, meeting all inclusion criteria but having ≥1 exclusion criteria);
Individuals ineligible for such an RCT owing to each specific exclusion criterion (ie, forming subpopulations of ineligible individuals owing to age, substance use, risk of suicide, treatment resistance, serious somatic disease, mood stabilizer or antidepressant use, intellectual disability, tardive dyskinesia, or pregnancy/breastfeeding).
We summarized relevant baseline characteristics and report the distribution of the prescribed antipsychotics (for the most commonly used drugs)11 of the eligible and ineligible populations. Based on previous knowledge,11 we categorized most commonly prescribed antipsychotics in these cohorts as olanzapine, quetiapine, risperidone, and aripiprazole, while the rest were grouped as either any long-acting injectable (LAI) antipsychotic or other oral antipsychotics.
The primary outcome was hospitalization due to psychosis (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision codes F20-F29). Secondary outcomes were hospitalization due to any psychiatric reason (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision codes F00-F99), all-cause hospitalization, the need for add-on antipsychotics, and all-cause discontinuation of antipsychotic use. To verify the robustness of our results for different time points, in addition to the main analyses at 12 months, we conducted analyses for the primary outcome at 6 months and 9 months of follow-up. Additionally, as sensitivity analyses, we applied the same primary outcome analyses to separate cohorts of (1) clozapine users and (2) individuals only treated in outpatient care (eAppendix 1A in the Supplement).
To compare potential differences in the risk of these outcomes between eligible and ineligible individuals, we calculated hazard ratios (HRs) and their 95% CIs using a Cox regression model with eligible individuals as reference. Proportional hazards assumption was tested and complied with by plotting Kaplan-Meier curves and via Schoenfeld residuals. To shed light on the associations of specific exclusion criteria with the primary outcome, we additionally compared individuals with a given exclusion criterion with eligible individuals. We also compared the primary outcome between individuals who met 1, 2, or 3 or more exclusion criteria with those of eligible individuals. We conducted all analyses using SAS statistical software version 9.4 (SAS Institute) between November 2020 and May 2021. We calculated 95% CIs to provide estimates of the accuracy of our population parameters. P values are only reported in Tables as additional measures of the magnitude and precision of the differences, but we do not characterize results as statistically significant or according to some arbitrary P value threshold.
Proportions, Descriptive Statistics, and Antipsychotic Use in Eligible and Ineligible Populations
The mean (SD) age in the Finnish cohort (n = 17 801) was 47.5 (13.8) years, and 8972 (50.4%) were women; the mean (SD) age in the Swedish cohort (n = 7458) was 44.8 (12.5) years, and 3344 (44.8%) were women. In the Finnish cohort, 3580 individuals (20.1%) were eligible for RCT participation; 14 221 (79.9%) met at least 1 exclusion criterion and were thus ineligible. Similarly, in the Swedish cohort, 1619 individuals (21.7%) were eligible for RCTs, and 5839 (78.3%) were ineligible.
There were no major differences in the distribution of age and sex between eligible and ineligible individuals (Table 1). Individuals who were ineligible for RCTs were more likely to use oral quetiapine (Finland: 3735 [26.3%] vs 612 [17.1%]; Sweden: 809 [13.9%] vs 110 [6.8%]; eFigure 1 in the Supplement). LAI antipsychotics were prescribed less frequently to ineligible than to eligible individuals (Finland: 1767 [12.4%] vs 753 [21.0%]; Sweden: 1075 [18.4%] vs 390 [24.1%]; eFigure 1 in the Supplement). In the Swedish data set where information on disability pension was available, ineligible individuals were somewhat more likely (4985 [85.4%]) to receive disability pension than eligible ones (1320 [81.5%]), indicating more severe decline in occupational function.
Ineligibility Reasons and Subpopulations
In the Finnish and Swedish cohorts, 5875 (33.0%) and 2514 (33.7%), respectively, fulfilled only 1 exclusion criterion, while 3271 (18.4%) and 1338 (17.9%), respectively, met 3 or more criteria (Figure, A). The most frequent reasons for ineligibility were serious somatic comorbidities (broad definition: 7202 [51%] and 2866 [49%]; narrow: 5287 [36%] and 1747 [30%] in Finland and Sweden, respectively) and concomitant use of mood stabilizers or antidepressants (7983 [56%] and 3281 [56%]), followed by a history of substance use (3808 [27%] and 1828 [31%]) and suicide risk (1690 [12%] and 1032 [18%]) (Figure, B).
Subpopulations ineligible owing to specific exclusion criteria had variation in age and sex distributions (eTable 1A and B in the Supplement). The proportion of men was highest in those excluded owing to substance use (2510 men [65.9%] in the Finnish cohort and 1230 [67.3%] in the Swedish cohort), whereas those excluded owing to age (almost entirely owing to age >65 years) were mainly women (687 men [38.3%] in the subgroup of the Finnish cohort excluded owing to age and 143 [36.3%] in the Swedish cohort). There was significant overlap between specific exclusion criteria with each other. Besides obvious overlap between broad and narrow definitions of serious somatic comorbidities, those with a history of suicide attempt often also had substance use (Finnish cohort: 882 [52%]; Swedish cohort: 533 [52%]) and mood stabilizers/antidepressants use (1009 [58%] in the Finnish and 603 [60%] in the Swedish cohorts) and those with tardive dyskinesia had serious somatic conditions (24 [60%] and 47 [63%] in the Finnish and Swedish cohorts, respectively) and mood stabilizers/antidepressants use (24 [60%] and 36 [48%] in the Finnish and Swedish cohorts, respectively).
Olanzapine was the most frequently prescribed antipsychotic across specific subpopulations (eFigures 2 and 3 in the Supplement). Quetiapine replaced olanzapine as the most frequently prescribed antipsychotic among those with a history of suicide attempts and among those with tardive dyskinesia in the Finnish cohort. Risperidone was equally commonly prescribed among pregnant or breastfeeding individuals as olanzapine in the Swedish cohort. In Finland, LAI antipsychotics were used in about 10% of all individuals in most subpopulations (except substance use), while in Sweden the use of LAI antipsychotics was more frequent: almost 20% in all groups (except mood stabilizers/antidepressants users).
Primary and Secondary Outcomes in Eligible and Ineligible Populations
During the 12 months of follow-up, individuals who would be ineligible for RCTs were more likely to be hospitalized owing to psychosis, compared with eligible individuals (Finnish cohort: 2609 [18.4%] vs 615 [17.2%]; HR, 1.14 [95% CI, 1.04-1.24]; Swedish cohort: 1174 [20.1%] vs 240 [14.8%]; HR, 1.47 [95% CI, 1.28-1.92]). Similar risk estimates were observed for 6- and 9-month follow-up times (Table 2).
Compared with eligible individuals, individuals who were ineligible for RCTs had increased risks of any psychiatric hospitalization (HR, 1.34 [95% CI, 1.23-1.45]) in the Finnish cohort; HR, 1.74 [95% CI, 1.52-1.99] in the Swedish cohort) and for all-cause hospitalization (HR, 1.55 [95% CI, 1.43-1.68] in the Finnish cohort; HR, 1.77 [95% CI, 1.55-2.03] in the Swedish cohort; Table 3). In the Swedish cohort, ineligible persons had a higher risk for needing an additional antipsychotic than eligible persons (HR, 1.31 [95% CI, 1.15-1.48]), which was not observed in the Finnish cohort (HR, 1.06 [95% CI, 0.96-1.17]). The risk of all-cause antipsychotic discontinuation did not differ between ineligible and eligible individuals (Table 3).
Primary Outcome in Subpopulations Ineligible for Specific Reasons
The largest risks of hospitalization due to psychosis were observed in individuals ineligible owing to treatment resistance, tardive dyskinesia, and history of suicide attempts (Table 4). Finally, with more ineligibility criteria met, larger risks of hospitalization due to psychosis were observed in both countries (eResults and eTables 2 and 3 in the Supplement).
Sensitivity analysis in the cohort of clozapine users uncovered similar proportions of individuals being ineligible (5806 [81.6%] in the Finnish cohort and 1346 [80.2%] in the Swedish cohort) for RCTs as in the main analyses (eTable 4 in the Supplement). Results from the second sensitivity analysis were also similar to the primary analysis: of 4727 individuals treated in outpatient care only, 3508 (74.2%) were ineligible for RCT participation (eTable 5 in the Supplement).
In this study, we applied typical inclusion and exclusion criteria of RCTs to the real-world populations of individuals with schizophrenia in Finnish and Swedish national registries. We found that almost 80% of individuals with schizophrenia would be ineligible to participate in typical RCTs and are therefore not represented in them. The most frequent reasons for ineligibility observed in the 2 cohorts were serious somatic comorbidities and concomitant use of mood stabilizers or antidepressants, followed by history of substance use and risk of suicide. Furthermore, we found that RCT-ineligible real-world individuals had, on average, a moderately higher risk for rehospitalization due to psychosis while receiving maintenance treatment with antipsychotics. This increased risk was observed in several subpopulations, ie, individuals ineligible for specific reasons such as substance use, risk of suicide, treatment resistance, or tardive dyskinesia. Moreover, ineligible individuals appeared to have a higher burden of psychiatric and somatic comorbidities, as indicated by increased psychiatric and any-reason hospitalization rates.
We envision the following implications of our findings. Because we showed that the majority of individuals with schizophrenia are not represented by typical RCTs and that clinical outcomes can differ between eligible and ineligible individuals, targeted RCTs, subgroup analyses of RCTs with broader inclusion criteria, and observational cohorts focusing on underrepresented subpopulations are warranted. To date, only a few RCTs have been conducted in specific patient groups.13-15 Additionally, because approximately 50% of ineligible individuals met somatic comorbidities exclusion criteria in our study, risks of adverse effects and their potential serious consequences as well as the risk of clinically significant pharmacological interactions could be higher in real-world populations than in RCTs. This may require clinical attention and further research after pivotal RCTs and drug market approval.16 The latter could exploit the potential offered by electronic health records for screening and recruiting trial participants and therefore enrich their real-world representativeness. It also underlines the importance of aftermarket/postapproval studies (ie, phase 4 studies) requested by regulators and conducted by pharmaceutical companies to particularly investigate the safety of new treatments in broader populations. Furthermore, we found that choices of antipsychotics were somewhat different between ineligible vs eligible individuals. In previous real-world studies using within-individual designs minimizing selection bias, LAI antipsychotics were associated with lowered risk of rehospitalization whereas quetiapine often was not, compared with no use of antipsychotics.17,18 Ineligible individuals were less likely to use LAI antipsychotics and more likely to use quetiapine than eligible individuals. Reasons for these differences are not fully clear from our data; however, it is possible that LAI antipsychotics are avoided as those are slower to taper, eg, in persons with high risk of extrapyramidal symptoms, or active substance use, which increases the risk of interactive effects leading possibly to respiratory depression or seizures. Quetiapine may be prescribed more often for subgroups presenting more affective symptoms (eg, with suicidal ideation) or for tardive dyskinesia.19 This also describes a fundamental difference between real-world studies (such as the present study) and RCTs: in the real world, treatments are chosen by clinicians by their best judgment and following clinical care guidelines, while in RCTs the treatment is preset by the design. Therefore, to further elucidate antipsychotic use and effectiveness in practice, in addition to typical RCTs, which are important to examine whether a drug works in principle in selection bias–free conditions (efficacy), pragmatic trials (such as STAR*D20 and CATIE21) and observational studies (such as the SOHO study22,23) may in future be of benefit.24 These are performed on less selected populations and resemble clinical practice more closely than typical RCTs. Finally, our results provide estimates for risks of rehospitalization for schizophrenia in different patient populations, which could be used to inform individuals and clinicians about the expected outcome on antipsychotics within 1 year. Of note, individuals with a previous history of substance use, suicide attempt, or clozapine use (as a proxy for treatment resistance) had only a moderately higher risk of rehospitalization for acute psychosis. In this context, it needs to be considered that these estimates only apply to individuals with schizophrenia already stable taking medication for 12 weeks before the start of the 1-year observation period. For some individuals in these subpopulations, it might be difficult to reach this level of stability. Nevertheless, the observed differences in rehospitalization rates between subpopulations call for more specific epidemiological studies on expected absolute risks and predictors of relapse and rehospitalization.
Our analysis is somewhat limited because our selection of individuals eligible for RCTs matches the population in actual RCTs only to a certain extent for different reasons. (1) Participation in a trial requires participants not only to meet eligibility criteria but also to be willing to participate in a trial; the latter might be an important driver of outcomes, which we cannot disentangle in the real-world population. This, in addition to other factors such as dropouts, might explain the difference in rehospitalization rates on antipsychotic maintenance therapy between RCTs (4% at 7-12 [median, 9] months)10 and the real world (here 14% at 9 months in eligible individuals). (2) The selection criteria used for our analysis are typical for a specific, relatively common type of RCTs (ie, relapse prevention of schizophrenia with antipsychotics) and specific real-world samples (Finland and Sweden). Therefore, our results may not be directly generalizable to other types of RCTs or to countries with different health care systems or resources. Although the main results represent only individuals previously treated in inpatient care due to schizophrenia spectrum disorders, additional analyses in the Swedish outpatient cohort showed similar results, allaying concerns about bias resulting from cohort and statistical method selections. (3) Furthermore, inclusion and exclusion criteria vary between RCTs (eAppendix 1B in the Supplement). Some RCTs apply more relaxed criteria than the ones we used, eg, by allowing the participation of individuals with psychiatric or somatic comorbidities, with stable concomitant antidepressant or mood-stabilizing medications, history of suicide attempts (without active suicidal thoughts or behaviors), or substance use (when inactive at the time or when criteria for dependence are not met). Of note, the data from real-world cohorts do not allow one to apply all eligibility criteria exactly as in RCTs because only diagnoses and not clinical ratings are available, and some symptoms are often underreported in diagnostic data (eg, suicidality and substance use). However, previous research has shown that register-based inpatient diagnoses of schizophrenia are valid.25 Consequently, while our estimates refer to standard RCT with rather strict criteria, possibly other RCTs represent more than the 20% of real-world patients. Nonetheless, our results highlight that there is considerable heterogeneity in real-world individuals, which is not addressed by most standard RCTs.
In conclusion, based on comprehensive main and sensitivity analyses leveraging sizeable nationwide cohorts and in line with hypotheses put forward before but backed with less solid evidence,26-31 only a minority (about one-fifth) of real-world individuals with schizophrenia may be eligible for typical RCTs and their clinical outcomes were estimated to differ from ineligible individuals. However, because we did not investigate relative treatment effects (eAppendix 1A in the Supplement) and because the observed differences in absolute risks for clinical outcomes were not extreme, we emphasize that there are no major indications from our research that overall RCT results on efficacy and safety of antipsychotics would not apply to ineligible individuals. Nevertheless, our results indicate that specific subgroups among the majority of real-world individuals ineligible for RCTs can have a different course of illness, which also means they might experience differential treatment benefits. Therefore, in line with previous literature,28,30,32-36 future studies focusing on specific subpopulations, pragmatic trials to investigate treatment strategies, and well-designed observational studies are needed to investigate and improve the outcomes of the many individuals afflicted by schizophrenia and currently underrepresented in research settings.
Corresponding Author: Jurjen J. Luykx, MD, PhD, Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University Medical Centre, Maastricht, the Netherlands, P. Debyelaan 25, 6229 HX Maastricht, the Netherlands (j.luykx@maastrichtuniversity.nl).
Accepted for Publication: November 20, 2021.
Published Online: January 26, 2022. doi:10.1001/jamapsychiatry.2021.3990
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Taipale H et al. JAMA Psychiatry.
Author Contributions: Drs Taipale and Tanskanen had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Taipale, Schneider-Thoma, and Pinzón-Espinosa are co–first authors.
Concept and design: Taipale, Schneider-Thoma, Pinzón-Espinosa, Radua, Cardoner, Cipriani, Vieta, Leucht, Tiihonen, Luykx.
Acquisition, analysis, or interpretation of data: Taipale, Schneider-Thoma, Pinzón-Espinosa, Radua, Efthimiou, Vinkers, Mittendorfer-Rutz, Pintor-Perez, Tanskanen, Tomlinson, Fusar-Poli, Cipriani, Leucht, Tiihonen, Luykx.
Drafting of the manuscript: Taipale, Schneider-Thoma, Pinzón-Espinosa, Efthimiou, Cardoner, Tomlinson, Vieta, Luykx.
Critical revision of the manuscript for important intellectual content: Taipale, Pinzón-Espinosa, Radua, Efthimiou, Vinkers, Mittendorfer-Rutz, Cardoner, Pintor-Perez, Tanskanen, Fusar-Poli, Cipriani, Vieta, Leucht, Tiihonen, Luykx.
Statistical analysis: Taipale, Radua, Efthimiou, Cipriani, Luykx.
Obtained funding: Luykx.
Administrative, technical, or material support: Mittendorfer-Rutz, Tanskanen, Tomlinson, Leucht, Luykx.
Supervision: Radua, Cardoner, Pintor-Perez, Fusar-Poli, Cipriani, Vieta, Leucht, Luykx.
Conflict of Interest Disclosures: Dr Taipale reported grants and personal fee from Janssen Pharmaceuticals, grants from Eli Lilly and Company, and personal fees from Otsuka Pharmaceutical outside the submitted work. Dr Pinzón-Espinosa reported personal fees and nonfinancial support from Lundbeck, Angelini, Neuraxpharm, and Pfizer and nonfinancial support from Janssen Pharmaceuticals and Casen Recordati outside the submitted work. Dr Mittendorfer-Rutz reported grants from Janssen Pharmaceuticals outside the submitted work. Dr Cardoner has received grants and acted as a CME consultant, advisor, or speaker for the following entities in the last 5 years (all unrelated to the present work): Angelini, Esteve, Exeltis, Janssen Pharmaceuticals, Lundbeck, Novartis, Pfizer, and Servier. Dr Tanskanen reported other support from Janssen and Eli Lilly and Company outside the submitted work. Dr Tomlinson reported personal fees from INCiPiT (Italian Network for Paediatric Trials) and Angelini Pharma outside the submitted work. Dr Fusar-Poli reported personal fees from Lundbeck, Angelini Pharma, and Menarini outside the submitted work. Dr Cipriani reported personal fees from INCiPiT (Italian Network for Paediatric Trials) and CARIPLO Foundation and personal fees and grants from Angelini Pharma outside the submitted work. Dr Vieta reported grants from AB-Biotics, Ferrer, GlaxoSmithKline, Janssen Pharmaceuticals, and Sanofi-Aventis; personal fees from AbbVie, Aimentia, Angelini Pharma, Celon Pharma, Sumitomo Dainippon Pharma, Gedeon Richter, Janssen Pharmaceuticals, Organon, Otsuka Pharmaceutical, Sunovion, and Takeda; and received grants and served as consultant, advisor, or CME speaker from GH Research, Lundbeck, and Sage outside the submitted work. Dr Leucht reported personal fees from Alkermes, Angelini Pharma, Eisai, Gedeon Richter, Janssen Pharmaceuticals, Johnson & Johnson, Lundbeck, Medichem, Merck Sharp & Dohme, Otsuka Pharmaceutical, Casen Recordati, Rovi, Sandoz, Sanofi, Sunovion, Teva Pharmaceuticals, Boehringer Ingelheim, and LTS Lohmann outside the submitted work. Dr Tiihonen reported personal fees from Finnish Medicines Agency, European Medicines Agency, Eli Lilly and Company, Janssen-Cilag, Otsuka Pharmaceutical, and Lundbeck and grants from the Stanley Foundation and the Sigrid Jusélius Foundation outside the submitted work. No other disclosures were reported.
Funding/Support: Dr Efthimiou was supported by the Swiss National Science Foundation (Ambizione grant 180083). Dr Cipriani is supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility, by an NIHR Research Professorship (grant RP-2017-08-ST2-006), by the NIHR Oxford and Thames Valley Applied Research Collaboration, and by the NIHR Oxford Health Biomedical Research Centre (grant BRC-1215-20005). Dr Taipale was funded by the Academy of Finland (grants 315969 and 320107). Dr Radua was funded by the Spanish Ministry of Science, Innovation and Universities/Economy and Competitiveness/Instituto de Salud Carlos III (grants CPII19/00009 abd PI19/00394), co-financed by ERDF Funds from the European Commission (A Way of Making Europe). Dr Luykx was supported by a personal Rudolf Magnus Talent Fellowship from the University Medical Center Utrecht (grant H150).
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The views expressed are those of the authors and not necessarily those of the UK National Health Service, the Conflict of Interest Disclosures: National Institute for Health Research, or the UK Department of Health.
Additional Contributions: We are indebted to experienced expert Russel Cummins (Altrecht Mental Health), who took the effort of critically reading our manuscript and providing his take on the current unmet needs in psychosis studies and the potential value of our findings (eAppendix 1 in the Supplement). Compensation was not received.
1.Cramer-van der Welle
CM, Peters
BJM, Schramel
FMNH, Klungel
OH, Groen
HJM, van de Garde
EMW; Santeon NSCLC Study Group; Santeon NSCLC study group. Systematic evaluation of the efficacy-effectiveness gap of systemic treatments in metastatic nonsmall cell lung cancer.
Eur Respir J. 2018;52(6):1801100. doi:
10.1183/13993003.01100-2018PubMedGoogle Scholar 3.Phillips
CM, Parmar
A, Guo
H,
et al. Assessing the efficacy-effectiveness gap for cancer therapies: a comparison of overall survival and toxicity between clinical trial and population-based, real-world data for contemporary parenteral cancer therapeutics.
Cancer. 2020;126(8):1717-1726. doi:
10.1002/cncr.32697PubMedGoogle ScholarCrossref 5.Ankarfeldt
MZ, Adalsteinsson
E, Groenwold
RHH, Ali
MS, Klungel
OH. A systematic literature review on the efficacy-effectiveness gap: comparison of randomized controlled trials and observational studies of glucose-lowering drugs.
Clin Epidemiol. 2017;9:41-51. doi:
10.2147/CLEP.S121991PubMedGoogle ScholarCrossref 7.Schneider
C, Breilmann
J, Reuter
B, Becker
T, Kösters
M. Systematic evaluation of the 'efficacy-effectiveness gap' in the treatment of depression with venlafaxine and duloxetine.
Acta Psychiatr Scand. 2021;144(2):113-124: doi:
10.1111/acps.13293Google ScholarCrossref 8.Luykx
JJ, Pinzón-Espinosa
J, Schneider-Thoma
J,
et al
What is the Impact of Randomized Controlled Trials Inclusion and Exclusion Criteria on Outcomes in People using Antipsychotics in the Real World? A Systematic Analysis in National Registries. OSF Regist; 2020. doi:
10.17605/OSF.IO/ZRBW7 9.Benchimol
EI, Smeeth
L, Guttmann
A,
et al; RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement.
PLoS Med. 2015;12(10):e1001885. doi:
10.1371/journal.pmed.1001885PubMedGoogle Scholar 13.Krause
M, Huhn
M, Schneider-Thoma
J, Bighelli
I, Gutsmiedl
K, Leucht
S. Efficacy, acceptability and tolerability of antipsychotics in patients with schizophrenia and comorbid substance use: a systematic review and meta-analysis.
Eur Neuropsychopharmacol. 2019;29(1):32-45. doi:
10.1016/j.euroneuro.2018.11.1105PubMedGoogle ScholarCrossref 18.Taipale
H, Mehtälä
J, Tanskanen
A, Tiihonen
J. Comparative effectiveness of antipsychotic drugs for rehospitalization in schizophrenia: a nationwide study with 20-year follow-up.
Schizophr Bull. 2018;44(6):1381-1387. doi:
10.1093/schbul/sbx176PubMedGoogle ScholarCrossref 22.Haro
JM, Suarez
D, Novick
D, Brown
J, Usall
J, Naber
D; SOHO Study Group. Three-year antipsychotic effectiveness in the outpatient care of schizophrenia: observational versus randomized studies results.
Eur Neuropsychopharmacol. 2007;17(4):235-244. doi:
10.1016/j.euroneuro.2006.09.005PubMedGoogle ScholarCrossref 25.Isohanni
M, Mäkikyrö
T, Moring
J,
et al. A comparison of clinical and research DSM-III-R diagnoses of schizophrenia in a Finnish national birth cohort: clinical and research diagnoses of schizophrenia.
Soc Psychiatry Psychiatr Epidemiol. 1997;32(5):303-308. doi:
10.1007/BF00789044PubMedGoogle ScholarCrossref 26.Riedel
M, Strassnig
M, Müller
N, Zwack
P, Möller
HJ. How representative of everyday clinical populations are schizophrenia patients enrolled in clinical trials?
Eur Arch Psychiatry Clin Neurosci. 2005;255(2):143-148. doi:
10.1007/s00406-004-0547-5PubMedGoogle ScholarCrossref 27.Freudenthal
R, Marston
L, Stansfeld
JL, Priebe
S, Moncrieff
J. How do participants in clinical trials compare with other patients with schizophrenia?
Contemp Clin Trials Commun. 2021;22:100803. doi:
10.1016/j.conctc.2021.100803PubMedGoogle Scholar 28.Kline
E, Hendel
V, Friedman-Yakoobian
M,
et al. A comparison of neurocognition and functioning in first episode psychosis populations: do research samples reflect the real world?
Soc Psychiatry Psychiatr Epidemiol. 2019;54(3):291-301. doi:
10.1007/s00127-018-1631-xPubMedGoogle ScholarCrossref 30.Kennedy-Martin
T, Curtis
S, Faries
D, Robinson
S, Johnston
J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results.
Trials. 2015;16:495. doi:
10.1186/s13063-015-1023-4PubMedGoogle ScholarCrossref 36.Alphs
L, Benson
C, Cheshire-Kinney
K,
et al. Real-world outcomes of paliperidone palmitate compared to daily oral antipsychotic therapy in schizophrenia: a randomized, open-label, review board-blinded 15-month study.
J Clin Psychiatry. 2015;76(5):554-561. doi:
10.4088/JCP.14m09584PubMedGoogle ScholarCrossref