Evaluation of Differences in Individual Treatment Response in Schizophrenia Spectrum Disorders: A Meta-analysis | Psychiatry and Behavioral Health | JAMA Psychiatry | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Observed Response Suggests Heterogeneity in Treatment Response
Observed Response Suggests Heterogeneity in Treatment Response

A, Thirty patients with schizophrenia were randomized to either the antipsychotic treatment or the control group. Despite the trial being simulated, with a main effect of treatment (Cohen d = 1.32; t27.5 = 3.46; P = .002), it may be tempting to infer from the Positive and Negative Syndrome Scale (PANSS) pretreatment and posttreatment outcome difference scores that some patients in the treatment group responded better than others. This observed pretreatment and posttreatment outcome difference can be misleading given that individual differences might be merely some unexplained components of variance. B, The treatment group patients are ranked according to the observed pretreatment and posttreatment outcome differences and classified as responders or nonresponders based on an arbitrary threshold (dashed light blue line). Although the ranking is not necessary for the classification, it increases the perception of individual differences in response to treatment.

Figure 2.  Consequences of Between-Patient Variation
Consequences of Between-Patient Variation

Two simulated scenarios are shown: one with a constant treatment effect across patients (A-C), and one with a reversed ranking after a control condition is taken into account (D, E). A, Using the same ranking from the simulated parallel trial, we show that ranking is a flawed approach to quantifying symptom improvement. B, When adding a control condition to the initial parallel trial, the seeming differences in improvement from patients in the treatment group vanish. C, The crossover trial simulation eliminates spurious differences in the outcome and reveals no between-patient differences in response. Although seemingly unlikely, such a scenario cannot be ruled out from the results of a parallel group trial. In another scenario, a variable treatment effect is added to a parallel trial (A). D, Differences in improvement may reverse the ranking if patients varied in their response to the treatment compared with controls. E, The patient who appeared to have improved the most in D had actually the smallest net improvement. PANSS indicates Positive and Negative Syndrome Scale.

Figure 3.  Random Within-Patient Fluctuations
Random Within-Patient Fluctuations

Four patients from the simulated trial were measured repeatedly over time using the Positive and Negative Syndrome Scale (PANSS). A, The differences in a patient’s symptom severity scores from time point to time point are independent of any intervention. B, Measuring repeatedly and calculating the means over all time points account for random within-patient fluctuations and reveal that patients had the same mean PANSS score. What differed was the amount of random fluctuation, which might be a highly unlikely scenario but, until tested, cannot be assumed is not true.

Figure 4.  Estimating Treatment-by-Patient Interaction With Repeated Crossover Trials
Estimating Treatment-by-Patient Interaction With Repeated Crossover Trials

The same simulation is shown of 30 patients with schizophrenia in a parallel trial assessed with the Positive and Negative Syndrome Scale (PANSS). A, Patients received both the antipsychotic and placebo drugs in a crossover trial. B, Only by running a crossover trial more than once can we identify whether individual response is a trait or a permanent feature of the patient. C, Associating the net improvement from crossover trials 1 and 2 (solid dots) shows that advantages from the first trial do not replicate in the second (r = 0.02; 95% CI). In this scenario, response to treatment is not a permanent feature of the patient. D, Another scenario using the same crossover trial 1 (A) and repeating it, we might observe that the response was almost identical to the response in crossover trial 1. E, Patients responded similarly in both trials (r = 0.77; 95% CI). In this scenario, we can consider response to treatment as a trait in these patients. Repeatability of the association from one crossover trial to the next in the same sample is the only way to separate treatment-by-patient interaction from random within-patient variation and to determine whether response is a trait or a state.

Figure 5.  Lower Variability in the Treatment Group Compared With the Control Group Across Antipsychotic Drug Parallel Trials
Lower Variability in the Treatment Group Compared With the Control Group Across Antipsychotic Drug Parallel Trials

The forest plot shows the VR together with its 95% CI for treatment vs control across 52 studies. Each study is listed with its respective citation number. The overall VR was lower for treatment compared with control group.

1.
Garver  DL, Holcomb  JA, Christensen  JD.  Heterogeneity of response to antipsychotics from multiple disorders in the schizophrenia spectrum.  J Clin Psychiatry. 2000;61(12):964-972. doi:10.4088/JCP.v61n1213PubMedGoogle ScholarCrossref
2.
Senn  S.  Mastering variation: variance components and personalised medicine.  Stat Med. 2016;35(7):966-977. doi:10.1002/sim.6739PubMedGoogle ScholarCrossref
3.
Senn  S.  Trying to be precise about vagueness.  Stat Med. 2007;26(7):1417-1430. doi:10.1002/sim.2639PubMedGoogle ScholarCrossref
4.
Homan  P, Kane  JM.  Clozapine as an early-stage treatment.  Acta Psychiatr Scand. 2018;138(4):279-280. doi:10.1111/acps.12965PubMedGoogle ScholarCrossref
5.
Hecksteden  A, Kraushaar  J, Scharhag-Rosenberger  F, Theisen  D, Senn  S, Meyer  T.  Individual response to exercise training - a statistical perspective.  J Appl Physiol (1985). 2015;118(12):1450-1459. doi:10.1152/japplphysiol.00714.2014PubMedGoogle ScholarCrossref
6.
Hecksteden  A, Pitsch  W, Rosenberger  F, Meyer  T.  Repeated testing for the assessment of individual response to exercise training.  J Appl Physiol (1985). 2018;124(6):1567-1579. doi:10.1152/japplphysiol.00896.2017PubMedGoogle ScholarCrossref
7.
Dworkin  RH, McDermott  MP, Farrar  JT, O’Connor  AB, Senn  S.  Interpreting patient treatment response in analgesic clinical trials: implications for genotyping, phenotyping, and personalized pain treatment.  Pain. 2014;155(3):457-460. doi:10.1016/j.pain.2013.09.019PubMedGoogle ScholarCrossref
8.
Bennett  JH, ed.  Statistical Inference and Analysis: Selected Correspondence of RA Fisher. Oxford, UK: Clarendon Press; 1990. Cited by: Senn S. Seven myths of randomisation in clinical trials. Stat Med. 2013;32(9): 1439-1450. doi:10.1002/sim.5713
9.
Cortés  J, González  JA, Medina  MN,  et al.  Does evidence support the high expectations placed in precision medicine? A bibliographic review.  F1000 Res. 2019;7:30. doi:10.12688/f1000research.13490.4Google ScholarCrossref
10.
Nakagawa  S, Poulin  R, Mengersen  K,  et al.  Meta-analysis of variation: ecological and evolutionary applications and beyond.  Methods Ecol Evol. 2015;6(2):143-152. doi:10.1111/2041-210X.12309Google ScholarCrossref
11.
Viechtbauer  W.  Conducting meta-analyses in R with the metafor package.  J Stat Softw. 2010;36(3):1-48. doi:10.18637/jss.v036.i03Google ScholarCrossref
12.
Brugger  SP, Howes  OD.  Heterogeneity and homogeneity of regional brain structure in schizophrenia: a meta-analysis.  JAMA Psychiatry. 2017;74(11):1104-1111. doi:10.1001/jamapsychiatry.2017.2663PubMedGoogle ScholarCrossref
13.
Pillinger  T, Osimo  EF, Brugger  S, Mondelli  V, McCutcheon  RA, Howes  OD.  A meta-analysis of immune parameters, variability, and assessment of modal distribution in psychosis and test of the immune subgroup hypothesis [published November 8, 2018].  Schizophr Bull. 2018. doi:10.1093/schbul/sby160PubMedGoogle Scholar
14.
Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle ScholarCrossref
15.
Leucht  S, Davis  JM.  Do antipsychotic drugs lose their efficacy for relapse prevention over time?  Br J Psychiatry. 2017;211(3):127-129. doi:10.1192/bjp.bp.117.201103PubMedGoogle ScholarCrossref
16.
Hedges  LV, Nowell  A.  Sex differences in mental test scores, variability, and numbers of high-scoring individuals.  Science. 1995;269(5220):41-45. doi:10.1126/science.7604277PubMedGoogle ScholarCrossref
17.
Litman  RE, Smith  MA, Doherty  JJ,  et al.  AZD8529, a positive allosteric modulator at the mGluR2 receptor, does not improve symptoms in schizophrenia: a proof of principle study.  Schizophr Res. 2016;172(1-3):152-157. doi:10.1016/j.schres.2016.02.001PubMedGoogle ScholarCrossref
18.
StudyRIS. Office of Clinical Pharmacology and Biopharmacy Review. NDA number: 20272. Janssen-Cilag; data on file 1996. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
19.
Daniel  DG, Zimbroff  DL, Potkin  SG, Reeves  KR, Harrigan  EP, Lakshminarayanan  M; Ziprasidone Study Group.  Ziprasidone 80 mg/day and 160 mg/day in the acute exacerbation of schizophrenia and schizoaffective disorder: a 6-week placebo-controlled trial.  Neuropsychopharmacology. 1999;20(5):491-505. doi:10.1016/S0893-133X(98)00090-6PubMedGoogle ScholarCrossref
20.
Potkin  SG, Litman  RE, Torres  R, Wolfgang  CD.  Efficacy of iloperidone in the treatment of schizophrenia: initial phase 3 studies.  J Clin Psychopharmacol. 2008;28(2)(suppl 1):S4-S11. doi:10.1097/JCP.0b013e3181692787PubMedGoogle ScholarCrossref
21.
Zborowski  J, Schmitz  P, Staser  J,  et al.  Efficacy and safety of sertindole in a trial of schizophrenic patients.  Biol Psychiatry. 1995;9(37):661-662. doi:10.1016/0006-3223(95)94656-HGoogle ScholarCrossref
22.
Marder  SR, Meibach  RC.  Risperidone in the treatment of schizophrenia.  Am J Psychiatry. 1994;151(6):825-835. doi:10.1176/ajp.151.6.825PubMedGoogle ScholarCrossref
23.
Kane  J, Canas  F, Kramer  M,  et al.  Treatment of schizophrenia with paliperidone extended-release tablets: a 6-week placebo-controlled trial.  Schizophr Res. 2007;90(1-3):147-161. doi:10.1016/j.schres.2006.09.012PubMedGoogle ScholarCrossref
24.
Marder  SR, Kramer  M, Ford  L,  et al.  Efficacy and safety of paliperidone extended-release tablets: results of a 6-week, randomized, placebo-controlled study.  Biol Psychiatry. 2007;62(12):1363-1370. doi:10.1016/j.biopsych.2007.01.017PubMedGoogle ScholarCrossref
25.
Durgam  S, Starace  A, Li  D,  et al.  An evaluation of the safety and efficacy of cariprazine in patients with acute exacerbation of schizophrenia: a phase II, randomized clinical trial.  Schizophr Res. 2014;152(2-3):450-457. doi:10.1016/j.schres.2013.11.041PubMedGoogle ScholarCrossref
26.
Casey  DE, Sands  EE, Heisterberg  J, Yang  H-M.  Efficacy and safety of bifeprunox in patients with an acute exacerbation of schizophrenia: results from a randomized, double-blind, placebo-controlled, multicenter, dose-finding study.  Psychopharmacology (Berl). 2008;200(3):317-331. doi:10.1007/s00213-008-1207-7PubMedGoogle ScholarCrossref
27.
Potkin  SG, Cohen  M, Panagides  J.  Efficacy and tolerability of asenapine in acute schizophrenia: a placebo- and risperidone-controlled trial.  J Clin Psychiatry. 2007;68(10):1492-1500. doi:10.4088/JCP.v68n1004PubMedGoogle ScholarCrossref
28.
Durgam  S, Cutler  AJ, Lu  K,  et al.  Cariprazine in acute exacerbation of schizophrenia: a fixed-dose, phase 3, randomized, double-blind, placebo- and active-controlled trial.  J Clin Psychiatry. 2015;76(12):e1574-e1582. doi:10.4088/JCP.15m09997PubMedGoogle ScholarCrossref
29.
Hirayasu  Y, Tomioka  M, Iizumi  M, Kikuchi  H.  A double-blind, placebo-controlled, comparative study of paliperidone extended release (ER) tablets in patients with schizophrenia.  Jpn J Clin Psychopharmacol. 2010;13:2077-2103.Google Scholar
30.
Hera021, 041-021SH. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research. Medical review(s); 2009. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
31.
Ogasa  M, Kimura  T, Nakamura  M, Guarino  J.  Lurasidone in the treatment of schizophrenia: a 6-week, placebo-controlled study.  Psychopharmacology (Berl). 2013;225(3):519-530. doi:10.1007/s00213-012-2838-2PubMedGoogle ScholarCrossref
32.
McEvoy  JP, Daniel  DG, Carson  WH  Jr, McQuade  RD, Marcus  RN.  A randomized, double-blind, placebo-controlled, study of the efficacy and safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients with acute exacerbations of schizophrenia.  J Psychiatr Res. 2007;41(11):895-905. doi:10.1016/j.jpsychires.2007.05.002PubMedGoogle ScholarCrossref
33.
Tzimos  A, Samokhvalov  V, Kramer  M,  et al.  Safety and tolerability of oral paliperidone extended-release tablets in elderly patients with schizophrenia: a double-blind, placebo-controlled study with six-month open-label extension.  Am J Geriatr Psychiatry. 2008;16(1):31-43. doi:10.1097/JGP.0b013e31815a3e7aPubMedGoogle ScholarCrossref
34.
ClinicalTrials.gov. A study of the efficacy and safety of asenapine in participants with an acute exacerbation of schizophrenia. Identifier: NCT01617187. https://clinicaltrials.gov/ct2/show/NCT01617187?cond=A+study+of+the+efficacy+and+safety+of+asenapine+in+participants+with+an+acute+exacerbation+of+schizophrenia&rank=1. Accessed October 31, 2016.
35.
Nasrallah  HA, Silva  R, Phillips  D,  et al.  Lurasidone for the treatment of acutely psychotic patients with schizophrenia: a 6-week, randomized, placebo-controlled study.  J Psychiatr Res. 2013;47(5):670-677. doi:10.1016/j.jpsychires.2013.01.020PubMedGoogle ScholarCrossref
36.
Correll  CU, Skuban  A, Hobart  M,  et al.  Efficacy of brexpiprazole in patients with acute schizophrenia: review of three randomized, double-blind, placebo-controlled studies.  Schizophr Res. 2016;174(1-3):82-92. doi:10.1016/j.schres.2016.04.012PubMedGoogle ScholarCrossref
37.
Canuso  CM, Schooler  N, Carothers  J,  et al.  Paliperidone extended-release in schizoaffective disorder: a randomized, controlled study comparing a flexible dose with placebo in patients treated with and without antidepressants and/or mood stabilizers.  J Clin Psychopharmacol. 2010;30(5):487-495. doi:10.1097/JCP.0b013e3181eeb600PubMedGoogle ScholarCrossref
38.
Shen  JH, Zhao  Y, Rosenzweig-Lipson  S,  et al.  A 6-week randomized, double-blind, placebo-controlled, comparator referenced trial of vabicaserin in acute schizophrenia.  J Psychiatr Res. 2014;53:14-22. doi:10.1016/j.jpsychires.2014.02.012PubMedGoogle ScholarCrossref
39.
Kane  JM, Cohen  M, Zhao  J, Alphs  L, Panagides  J.  Efficacy and safety of asenapine in a placebo- and haloperidol-controlled trial in patients with acute exacerbation of schizophrenia.  J Clin Psychopharmacol. 2010;30(2):106-115. doi:10.1097/JCP.0b013e3181d35d6bPubMedGoogle ScholarCrossref
40.
Study93202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
41.
Lieberman  JA, Davis  RE, Correll  CU,  et al.  ITI-007 for the treatment of schizophrenia: A 4-week randomized, double-blind, controlled trial.  Biol Psychiatry. 2016;79(12):952-961. doi:10.1016/j.biopsych.2015.08.026PubMedGoogle ScholarCrossref
42.
Bugarski-Kirola  D, Wang  A, Abi-Saab  D, Blättler  T.  A phase II/III trial of bitopertin monotherapy compared with placebo in patients with an acute exacerbation of schizophrenia: results from the CandleLyte study.  Eur Neuropsychopharmacol. 2014;24(7):1024-1036. doi:10.1016/j.euroneuro.2014.03.007PubMedGoogle ScholarCrossref
43.
van Kammen  DP, McEvoy  JP, Targum  SD, Kardatzke  D, Sebree  TB.  A randomized, controlled, dose-ranging trial of sertindole in patients with schizophrenia.  Psychopharmacology (Berl). 1996;124(1-2):168-175. doi:10.1007/BF02245618PubMedGoogle ScholarCrossref
44.
Study94202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
45.
Potkin  SG, Saha  AR, Kujawa  MJ,  et al.  Aripiprazole, an antipsychotic with a novel mechanism of action, and risperidone vs placebo in patients with schizophrenia and schizoaffective disorder.  Arch Gen Psychiatry. 2003;60(7):681-690. doi:10.1001/archpsyc.60.7.681PubMedGoogle ScholarCrossref
46.
Litman  RE, Smith  MA, Desai  DG, Simpson  T, Sweitzer  D, Kanes  SJ.  The selective neurokinin 3 antagonist AZD2624 does not improve symptoms or cognition in schizophrenia: a proof-of-principle study.  J Clin Psychopharmacol. 2014;34(2):199-204. doi:10.1097/JCP.0000000000000071PubMedGoogle ScholarCrossref
47.
Kinon  BJ, Zhang  L, Millen  BA,  et al; HBBI Study Group.  A multicenter, inpatient, phase 2, double-blind, placebo-controlled dose-ranging study of LY2140023 monohydrate in patients with DSM-IV schizophrenia.  J Clin Psychopharmacol. 2011;31(3):349-355. doi:10.1097/JCP.0b013e318218dcd5PubMedGoogle ScholarCrossref
48.
Kane  JM, Carson  WH, Saha  AR,  et al.  Efficacy and safety of aripiprazole and haloperidol versus placebo in patients with schizophrenia and schizoaffective disorder.  J Clin Psychiatry. 2002;63(9):763-771. doi:10.4088/JCP.v63n0903PubMedGoogle ScholarCrossref
49.
Egan  MF, Zhao  X, Smith  A,  et al.  Randomized controlled study of the T-type calcium channel antagonist MK-8998 for the treatment of acute psychosis in patients with schizophrenia.  Hum Psychopharmacol. 2013;28(2):124-133. doi:10.1002/hup.2289PubMedGoogle ScholarCrossref
50.
Durgam  S, Litman  RE, Papadakis  K, Li  D, Németh  G, Laszlovszky  I.  Cariprazine in the treatment of schizophrenia: a proof-of-concept trial.  Int Clin Psychopharmacol. 2016;31(2):61-68. doi:10.1097/YIC.0000000000000110PubMedGoogle ScholarCrossref
51.
Barbato L, Newcomer J, Heisterberg J, Yeung P, Shapira N. Efficacy and metabolic profile of bifeprunox in patients with schizophrenia. Paper presented at: 11th International Congress on Schizophrenia Research; March 28 to April 1, 2007; Colorado Springs, CO.
52.
Study049. A 6-week, double-blind, randomized, fixed dose, parallel-group study of the efficacy and safety of three dose levels of SM-13496 (lurasidone) compared to placebo and haloperidol in patients with schizophrenia who are experiencing an acute exacerbation of symptoms. Center for Drug Evaluation and Research; Medical review(s). 2010. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
53.
Meltzer  HY, Cucchiaro  J, Silva  R,  et al.  Lurasidone in the treatment of schizophrenia: a randomized, double-blind, placebo-and olanzapine-controlled study. Am J Psychiatry. 2011;168(9):957-967. doi:10.1176/appi.ajp.2011.10060907PubMed
54.
Meltzer  HY, Barbato  L, Heisterberg  J, Yeung  P, Shapira  N.  A randomized, doubleblind, placebo-controlled efficacy and safety study of bifeprunox as treatment for patients with acutely exacerbated schizophrenia.  Schizophr Bull. 2007;33(2):446.Google Scholar
55.
Loebel  A, Cucchiaro  J, Sarma  K,  et al.  Efficacy and safety of lurasidone 80 mg/day and 160 mg/day in the treatment of schizophrenia: a randomized, double-blind, placebo- and active-controlled trial.  Schizophr Res. 2013;145(1-3):101-109. doi:10.1016/j.schres.2013.01.009PubMedGoogle ScholarCrossref
56.
Kane  JM, Zukin  S, Wang  Y,  et al.  Efficacy and safety of cariprazine in acute exacerbation of schizophrenia: results from an international, phase III clinical trial.  J Clin Psychopharmacol. 2015;35(4):367-373.PubMedGoogle Scholar
57.
Garcia  E, Robert  M, Peris  F, Nakamura  H, Sato  N, Terazawa  Y.  The efficacy and safety of blonanserin compared with haloperidol in acute-phase schizophrenia: a randomized, double-blind, placebo-controlled, multicentre study.  CNS Drugs. 2009;23(7):615-625. doi:10.2165/00023210-200923070-00006PubMedGoogle ScholarCrossref
58.
Davidson  M, Emsley  R, Kramer  M,  et al.  Efficacy, safety and early response of paliperidone extended-release tablets (paliperidone ER): results of a 6-week, randomized, placebo-controlled study.  Schizophr Res. 2007;93(1-3):117-130. doi:10.1016/j.schres.2007.03.003PubMedGoogle ScholarCrossref
59.
Canuso  CM, Lindenmayer  J-P, Kosik-Gonzalez  C,  et al.  A randomized, double-blind, placebo-controlled study of 2 dose ranges of paliperidone extended-release in the treatment of subjects with schizoaffective disorder.  J Clin Psychiatry. 2010;71(5):587-598. doi:10.4088/JCP.09m05564yelPubMedGoogle ScholarCrossref
60.
Hera022, SH H 041-022. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research; Medical review(s); 2009.Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
61.
Nakamura  M, Ogasa  M, Guarino  J,  et al.  Lurasidone in the treatment of acute schizophrenia: a double-blind, placebo-controlled trial.  J Clin Psychiatry. 2009;70(6):829-836. doi:10.4088/JCP.08m04905PubMedGoogle ScholarCrossref
62.
Study115. Center for Drug Evaluation and Research. Approval package for application number 20-825. Medical review(s); 2000. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
63.
Beasley  CM  Jr, Sanger  T, Satterlee  W, Tollefson  G, Tran  P, Hamilton  S.  Olanzapine versus placebo: results of a double-blind, fixed-dose olanzapine trial.  Psychopharmacology (Berl). 1996;124(1-2):159-167. doi:10.1007/BF02245617PubMedGoogle ScholarCrossref
64.
Lindenmayer  J-P, Brown  D, Liu  S, Brecher  M, Meulien  D.  The efficacy and tolerability of once-daily extended release quetiapine fumarate in hospitalized patients with acute schizophrenia: a 6-week randomized, double-blind, placebo-controlled study.  Psychopharmacol Bull. 2008;41(3):11-35.PubMedGoogle Scholar
65.
Schmidt  ME, Kent  JM, Daly  E,  et al.  A double-blind, randomized, placebo-controlled study with JNJ-37822681, a novel, highly selective, fast dissociating D2 receptor antagonist in the treatment of acute exacerbation of schizophrenia.  Eur Neuropsychopharmacol. 2012;22(10):721-733. doi:10.1016/j.euroneuro.2012.02.007PubMedGoogle ScholarCrossref
66.
Meltzer  HY, Arvanitis  L, Bauer  D, Rein  W; Meta-Trial Study Group.  Placebo-controlled evaluation of four novel compounds for the treatment of schizophrenia and schizoaffective disorder.  Am J Psychiatry. 2004;161(6):975-984. doi:10.1176/appi.ajp.161.6.975PubMedGoogle ScholarCrossref
67.
Coppola  D, Melkote  R, Lannie  C,  et al.  Efficacy and safety of paliperidone extended release 1.5 mg/day-a double-blind, placebo- and active-controlled, study in the treatment of patients with schizophrenia.  Psychopharmacol Bull. 2011;44(2):54-72.PubMedGoogle Scholar
68.
Chouinard  G, Jones  B, Remington  G,  et al.  A Canadian multicenter placebo-controlled study of fixed doses of risperidone and haloperidol in the treatment of chronic schizophrenic patients.  J Clin Psychopharmacol. 1993;13(1):25-40. doi:10.1097/00004714-199302000-00004PubMedGoogle ScholarCrossref
69.
Senn  S.  Seven myths of randomisation in clinical trials.  Stat Med. 2013;32(9):1439-1450. doi:10.1002/sim.5713PubMedGoogle ScholarCrossref
70.
Leucht  S, Davis  JM.  Enthusiasm and skepticism about using national registers to analyze psychotropic drug outcomes.  JAMA Psychiatry. 2018;75(4):314-315. doi:10.1001/jamapsychiatry.2017.4704PubMedGoogle ScholarCrossref
71.
Kubo  K, Fleischhacker  WW, Suzuki  T, Yasui-Furukori  N, Mimura  M, Uchida  H.  Placebo effects in adult and adolescent patients with schizophrenia: combined analysis of nine RCTs.  Acta Psychiatr Scand. 2019;139(2):108-116. doi:10.1111/acps.12960PubMedGoogle ScholarCrossref
72.
Hróbjartsson  A, Gøtzsche  PC.  Is the placebo powerless? an analysis of clinical trials comparing placebo with no treatment.  N Engl J Med. 2001;344(21):1594-1602. doi:10.1056/NEJM200105243442106PubMedGoogle ScholarCrossref
73.
Senn  S.  Applying results of randomised trials to patients. N of 1 trials are needed.  BMJ. 1998;317(7157):537-538. doi:10.1136/bmj.317.7157.537PubMedGoogle ScholarCrossref
74.
Wang  R, Lagakos  SW, Ware  JH, Hunter  DJ, Drazen  JM.  Statistics in medicine–reporting of subgroup analyses in clinical trials.  N Engl J Med. 2007;357(21):2189-2194. doi:10.1056/NEJMsr077003PubMedGoogle ScholarCrossref
75.
Araujo  A, Julious  S, Senn  S.  Understanding variation in sets of N-of-1 trials.  PLoS One. 2016;11(12):e0167167. doi:10.1371/journal.pone.0167167PubMedGoogle ScholarCrossref
76.
Joyce  DW, Tracy  DK, Shergill  SS.  Are we failing clinical trials? a case for strong aggregate outcomes.  Psychol Med. 2018;48(2):177-186. doi:10.1017/S0033291717001726PubMedGoogle ScholarCrossref
77.
González  AB, Cox  DR.  Interpretation of interaction: a review.  Ann Appl Stat. 2007;1(2):371-385. doi:10.1214/07-AOAS124Google ScholarCrossref
Original Investigation
June 3, 2019

Evaluation of Differences in Individual Treatment Response in Schizophrenia Spectrum Disorders: A Meta-analysis

Author Affiliations
  • 1Center for Psychiatric Neuroscience, Feinstein Institute for Medical Research, Manhasset, New York
  • 2Division of Psychiatry Research, Zucker Hillside Hospital, Northwell Health, New York, New York
  • 3Department of Psychiatry, Zucker School of Medicine at Northwell/Hofstra, Hempstead, New York
  • 4University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
  • 5Department of Psychiatry and Psychotherapy, Technische Universität München, Munich, Germany
JAMA Psychiatry. 2019;76(10):1063-1073. doi:10.1001/jamapsychiatry.2019.1530
Key Points

Question  Is there evidence from randomized clinical trials that patients respond differently to antipsychotic drugs?

Findings  In this meta-analysis of 52 randomized clinical trials involving 15 360 patients with a schizophrenia or schizoaffective diagnosis, the outcome variability in the antipsychotic drug treatment group was not higher but slightly lower than that in the placebo control group.

Meaning  This study cannot rule out that individual differences in drug response might still exist, but it does question the assumption of a personal element of response to antipsychotic treatment.

Abstract

Importance  An assumption among clinicians and researchers is that patients with schizophrenia vary considerably in their response to antipsychotic drugs in randomized clinical trials (RCTs).

Objective  To evaluate the overall variation in individual treatment response from random variation by comparing the variability between treatment and control groups.

Data Sources  Cochrane Schizophrenia, MEDLINE/PubMed, Embase, PsycINFO, Cochrane CENTRAL, BIOSIS Previews, ClinicalTrials.gov, and World Health Organization International Clinical Trials Registry Platform from January 1, 1955, to December 31, 2016.

Study Selection  Double-blind, placebo-controlled, RCTs of adults with a diagnosis of schizophrenia spectrum disorders and prescription for licensed antipsychotic drugs.

Data Extraction and Synthesis  Means and SDs of the Positive and Negative Syndrome Scale pretreatment and posttreatment outcome difference scores were extracted. Data quality and validity were ensured by following the PRISMA guidelines.

Main Outcomes and Measures  The outcome measure was the overall variability ratio of treatment to control in a meta-analysis across RCTs. Individual variability ratios were weighted by the inverse-variance method and entered into a random-effects model. A personal element of response was hypothesized to be reflected by a substantial overall increase in variability in the treatment group compared with the control group.

Results  An RCT was simulated, comprising 30 patients with schizophrenia randomized to either the treatment or the control group. The different components of variation in RCTs were illustrated with simulated data. In addition, we assessed the variability ratio in 52 RCTs involving 15 360 patients with a schizophrenia or schizoaffective diagnosis. The variability was slightly lower in the treatment compared with the control group (variability ratio = 0.97; 95% CI, 0.95-0.99; P = .01).

Conclusions and Relevance  In this study, no evidence was found in RCTs that antipsychotic drugs increased the outcome variance, suggesting no personal element of response to treatment but instead indicating that the variance was slightly lower in the treatment group than in the control group; although the study cannot rule out that subsets of patients respond differently to treatment, it suggests that the average treatment effect is a reasonable assumption for the individual patient.

Introduction

Personalized medicine is based on a widely held assumption that patients differ substantially in their response to treatments. The goal of personalized medicine is to find the right treatment for the right patient. Psychiatry is no exception. An assumption among clinicians and researchers alike is that the response to antipsychotic drugs by patients with psychosis differs considerably between individuals.1

We report that this assumption may be ungrounded. Although variation in the observed treatment responses obviously exists, it is crucial to distinguish between observed and true treatment response: observed response consists of true response plus regression to the mean, some placebo effects, and random terms such as (but not restricted to) measurement error. First, we exemplify why confusing observed with true treatment response is so common, and we use simulated data to show how variation that is purely random and unrelated to permanent differences in treatment response may suggest the need for personalized treatment. Next, we review the evidence of the differences in treatment response by conducting a meta-analysis of the variation in antipsychotic treatment trials. Although this issue is brought up by statisticians regularly,2 it deserves more attention from a general psychiatric audience.

Where does the assumption of individual differences in treatment response come from? In general, antipsychotic drugs are assessed in randomized clinical trials (RCTs), the criterion standard for identifying the efficacy of a treatment. In RCTs, patients are assessed at baseline (eg, with the Positive and Negative Syndrome Scale [PANSS]) and randomized to either a treatment or a control group. What RCTs can ultimately provide is an answer to whether a treatment works in general. This average treatment effect is derived from the direct comparison of the response between the treatment and the control groups, which is imperative in an RCT.3 Understandably, this answer may leave clinicians unsatisfied; after all, they are treating individual, and not typical, patients. From a clinical perspective, patients vary considerably in their response to antipsychotic drugs, and the general response may seem almost like an uninformed guess for the individual patient. Furthermore, clinicians seem to prefer categories such as normal or abnormal and responders or nonresponders to inform diagnostic and therapeutic decisions. A consequence is that many investigators now try to personalize medicine by aiming to tailor treatments to individual patients. They agree that response to treatment varies from patient to patient.

However, estimating individual response to treatment, known as the treatment-by-patient interaction, is more complex than often appreciated and depends on laborious study designs, such as repeated crossover trials.2 However, as we illustrate with simulated data, such study designs are needed to distinguish individual response to treatment from other components of variation that are unrelated to permanent differences in treatment response.4-7

By design, RCTs cannot estimate the treatment-by-patient interaction, the index of individual response. Although RCTs do not tell anything about individual response, they might indicate something about the presence of individual response. As recognized early by Fisher,8 an increase in variance in the treatment group compared with the control group could indicate the presence of variation in response to treatment.2 The strength of this increase would then quantify the size of the personal element of response and provide evidence for the presence of a treatment-by-patient interaction.9 A method has been developed to compare variances between groups across studies10 and has been adopted by a meta-analysis package.11 In psychiatry, this method has been applied to compare variances in brain structure12 and inflammatory parameters in psychosis.13 This method compares the variance of treatment and control by computing their ratio: a ratio of 1 means equal variances, a ratio greater than 1 means more variability in the treatment group, and a ratio smaller than 1 means less variability in the treatment group compared with the control group.10,12,13

This study is organized in 2 parts. The first part illustrates the different components of variation in RCTs with simulated data, showing the importance of recognizing the treatment-by-patient interaction (which reflects individual treatment response) as the component of interest. The second part shows the results of a meta-analysis, which tested for the presence of treatment-by-patient interaction in empirical data from antipsychotic drug RCTs. We compared the overall variability in the treatment group with the overall variability in the control group, using data from a recently published meta-analysis,14 summarizing 24 years of placebo-controlled, antipsychotic RCTs in schizophrenia. We hypothesized that compared with control, the often-highlighted heterogeneity in patients with schizophrenia would be reflected by a clinically relevant increase in overall variance of treatment, outcome, which is compatible with a personal element of response that deviates from the estimated average treatment effects.

Methods
Trial Simulation

To illustrate the different components of variation in RCTs, we simulated data from patients with schizophrenia who were randomized to either the antipsychotic treatment or control group and assessed with the PANSS and a positive effect of treatment (Cohen d = 1.32; t27.5 = 3.46; P = .002). First, we added a single crossover condition with either a constant or a varying treatment effect, and then we added a double crossover to this simulated trial. With these additions, we show how the variability between and within patients has to be distinguished from the treatment-by-patient interaction, the component reflecting the individual differences in treatment response.

Meta-analysis

To ensure data quality and validity, this meta-analysis was conducted in accordance with the PRISMA guidelines. We searched Cochrane Schizophrenia, MEDLINE/PubMed, Embase, PsycINFO, Cochrane CENTRAL, BIOSIS Previews, ClinicalTrials.gov, and World Health Organization International Clinical Trials Registry Platform from January 1, 1955, to December 31, 2016.

Using the meta-analysis of Leucht et al15 as a basis, we included published and unpublished double-blind, placebo-controlled RCTs of at least 3 weeks’ duration. These studies investigated adults with a diagnosis of schizophrenia spectrum disorders and prescription for licensed antipsychotic medications, except clozapine. Studies were excluded if they investigated relapse prevention, patients with predominant negative symptoms, patients with major concomitant somatic or psychiatric illness, or intramuscular formulations of antipsychotic treatment, or if they were Chinese research. We included only studies that reported the necessary information (mean, SD, and sample size) of the PANSS pretreatment and posttreatment outcome difference scores.

In studies that combined comparisons of multiple antipsychotic drugs with placebo, we calculated an aggregated SD across all comparisons, leaving only 1 SD per study. We extracted the PANSS means and SDs of the pretreatment and posttreatment outcome difference scores as well as the sample sizes for the treatment and the control groups. Further information on the search strategy is published elsewhere.15

Statistical Analysis

The SDs of the pretreatment and posttreatment outcome difference scores in the treatment and control groups consist of the same variance components, including the within-patient variation. The treatment group, however, may also include the additional treatment-by-patient interaction, which could indicate the presence of individual response differences. Thus, in the case of a variable treatment effect, an increase of the variance in the treatment group, compared with the control group, should be observable. To assess this variation, we calculated for each comparison between antipsychotic and placebo drugs the relative variability of treatment and control as the log variability ratio (log VR)16 with

Image description not available.,

in which SDTx was the reported sample SD for treatment, SDCt was the reported sample SD for control, nTx was the treatment sample size, and nCt the control sample size.10 The corresponding sampling variance (SD2logVR)for each comparison between antipsychotic and placebo drugs can be expressed as follows:

Image description not available..

We did not find an association between the pretreatment and posttreatment outcome difference scores and their respective SDs in the data for the control group (β = 0.16; P = .15; eFigure 1A in the Supplement) or the treatment group (β = –0.05; P = .63; eFigure 1B in the Supplement). For this reason, we did not consider the log coefficient of variation ratio (log CVR) as an additional index for comparing variabilities.10

We weighted each log VR with the inverse of this sampling variance11 and entered it into a random-effects model. This approach allows for the quantification of the true individual response, after adjusting for within-patient variability and regression to the mean.5,9 Results were back-transformed from the log scale for better interpretability, with a variability ratio higher than 1, indicating greater variability under treatment compared with control, and a ratio lower than 1, indicating less variability under treatment compared with control.

The analysis was performed from October 31, 2018, to March 29, 2019, with the R package metafor, version 2.0.0,11 and the manuscript was produced with the R package knitr, version 1.20, in RStudio (R Foundation for Statistical Computing). All the data and code we used are freely available online to ensure reproducibility (https://doi.org/10.17605/OSF.IO/QARVS).

Results
Simulation

We simulated an RCT of 30 patients with schizophrenia randomized to either the treatment or the control group. The individual pretreatment and posttreatment outcome differences (Figure 1A) might tempt us to infer that some patients in the treatment group responded better than others. We might then rank these patients according to their outcome and classify them as either responders or nonresponders. However, such ranking and classification can be misleading.

Although seemingly different (Figure 2A), adding a simulated crossover condition to the initial parallel trial may reveal that the apparent differences in improvement among patients in the treatment group vanish (Figure 2B) and the treatment effect may actually be constant across patients (Figure 2C). Such a scenario cannot be ruled out from the results of a parallel group trial. In addition, the same ranking (Figure 2A) may reflect yet another scenario, in which differences in improvement as calculated from a crossover condition may reverse the ranking (Figure 2D), such that patients who appeared to have improved the most had actually the smallest net improvement (Figure 2E). Apparent outcome differences among patients in an RCT may still be compatible with a constant treatment effect.

Next, outcome differences may also be found within patients. Assessing patients repeatedly over time might reveal that symptoms fluctuate randomly around the same mean score (Figure 3). This fluctuation shows that within-patient variability alone may suggest differences in treatment response that are a mere reflection of random fluctuation.

Again, we can add a simple crossover condition to the simulated parallel group trial (Figure 4A), in which each patient received both the treatment (antipsychotic drug) and control (placebo). Only by running the crossover trial once again (Figure 4B) can we determine whether the differences observed in the first crossover trial are indeed stable features of the patients. The net improvement from crossover trial 1 may not replicate in crossover trial 2, which indicates that the response differences are still not stable features of the patients (Figure 4C). For that stability to be the case, we would have to see a similar outcome in crossover trial 1 (Figure 4A) as in crossover trial 2 (Figure 4D), in which case we have identified a substantial treatment-by-patient interaction (Figure 4E).

A careful distinction of the sources of variation in a simulated RCT has shown that it is not trivial to distinguish the source of primary interest (treatment-by-patient interaction) from components that tell nothing about individual response. In the meta-analysis, we assessed whether evidence exists for such treatment-by-patient interaction across antipsychotic drug trials.

Meta-analysis

We investigated 75 comparisons of antipsychotic drug with placebo in 52 RCTs.17-68 None of these studies used a design such as repeated crossovers that would have allowed for a direct estimate of individual responses. Overall, a total of 15 360 patients with a schizophrenia or schizoaffective diagnosis were included, of whom 8550 (55.7%) had been randomized to the treatment group and 6810 (44.3%) to the control group (more details can be found in the eResults in the Supplement).

We found an overall lower variability in treatment compared with control (variability ratio = 0.97; 95% CI, 0.95-0.99; P = .01; Figure 517-68). This finding indicates that the overall variability across treatment groups was 3% lower compared with that in the control groups. Furthermore, we compared the variances in individual antipsychotic drug outcome and found the same pattern, with lower variability across treatment compared with control (variability ratio = 0.97; 95% CI, 0.95-1.00; P = .02; eFigure 2 in the Supplement).

No evidence was found that antipsychotic treatment increased the outcome variance compared with the control. Instead, the outcome variance was slightly lower in the treatment than in the control group.

Discussion

A widespread belief among clinicians and researchers is that patients differ substantially in their antipsychotic treatment response, but finding evidence for this assumption is complex. A likely explanation, supported by the simulations conducted for this study, is that taking an observed treatment response as the true treatment response is tempting, compelling us to ignore the components of variation most likely encountered: random variation within patients and differences between patients. The existing empirical evidence for such individual differences is weaker than expected: No evidence was found that the antipsychotic drug increased the outcome variance compared with the placebo. Instead, the outcome variance was slightly lower in the treatment group. With this finding, we still cannot rule out that subsets of patients responded differently to treatment, but the overall small difference in variances suggests that the average treatment effect is a reasonable assumption for the individual patient. By assuming heterogeneity in treatment outcomes, we might ultimately introduce noise into clinical practice by refusing to go with the best available evidence, the average treatment effect derived from RCTs.2

Although RCTs are questioned regularly, sometimes using questionable arguments,69 they remain the criterion standard in clinical research. They provide unbiased estimates of the relative efficacy of an intervention, which even the largest observational studies cannot provide.70 In addition, appreciating the role of randomization in RCTs is important. Randomization is not compatible with the notion that specific features, such as placebo response, increase in one but not the other group in an RCT. If evidence existed of an enhanced placebo response over time, as has been suggested repeatedly in the past years,14 this response would have been apparent in both the control and the treatment groups because of randomization and thus would have canceled out. Furthermore, the concept of placebo response, although regularly investigated,71 cannot be studied by looking at the observed responses in control groups,72 for the same reasons that this approach does not work for the treatment groups, as this study has shown.

Comparing the variabilities between treatment and control groups may provide valuable insight into the presence of individual response and the scope of personalized medicine. Recently, other groups have taken a similar approach to assess the presence of individual differences in brain structure12 and immunological parameters in psychosis.13 We assumed that, in the presence of a personal element of response to treatment, the variance in the treatment group should be higher compared with the control group, which in turn would require further investigation (eg, with n-of-1 trials).73-75 However, our results indicate that overall variability in the treatment groups was slightly lower, if only by a modest amount (1% to 5%). One explanation might be that the treatment had a stabilizing quality9 that reduced the variability in the treatment group. An example for such variance stabilization might be the floor effect, in which the assessment instrument is too coarse to capture patient improvements over a certain level.

Nevertheless, given the slightly lower variability under treatment compared with control found in this study, we cannot rule out that individual differences in response to antipsychotic drugs might still exist. A subset of ill patients may have responded well to treatment, whereas less affected patients may not have improved, resulting in an overall decreased variance under treatment than under control.9 Yet, the finding of a narrow CI around an overall only slightly lower variability suggests that substantial differences in drug response are rather unlikely. Thus, analyses aimed at estimating individual response might be premature until these differences have been shown to exist and to be clinically relevant.

As the simulations have shown, labeling patients as responders might be misleading. The label suggests that true response has been established as a permanent feature of the patient, even though the label is a mere reflection of the observed response, which includes true response plus regression to the mean, some placebo effect, and random terms such as measurement error. Thus, response rates that are calculated in RCTs reflect observed but not true response. We suggest that biomarker research aimed at identifying response to treatment of individuals or subgroups should consider the possibility that treatment outcome is less heterogeneous than anticipated and might even be close to constant across individuals.

Limitations

This meta-analysis has some limitations. First, the calculation of the pretreatment and posttreatment outcome difference scores varied between studies. Although some RCTs calculated the differences between outcome and baseline PANSS scores, others used analysis of covariances with the baseline PANSS scores and additional variables as covariates. Thus, some of the included SDs of change were adjusted for covariates but others were not. Second, the use of pretreatment and posttreatment outcome difference scores might lead to a loss of information and might not be sensitive enough to capture differences in response to treatment.76 Third, we assumed that individual responses to treatment were reflected by increased variance in the treatment group. Yet, this increased variance could have also indicated the presence of subgroups who responded differently to the treatment.9 Such a case would argue for stratified medicine rather than personalized medicine, in which subgroups of patients receive varying treatments. As any interaction, a treatment-by-patient interaction is ultimately scale dependent, which means it can be removed by transformation of the scale.77

Conclusions

Until the differences in individual response to treatment have been demonstrated with careful designs, the overall small differences in outcome variance suggest that the average treatment effect is a reasonable assumption for the individual patient.

Back to top
Article Information

Accepted for Publication: April 27, 2019.

Corresponding Authors: Stephanie Winkelbeiner, PhD, Center for Psychiatric Neuroscience, Feinstein Institute for Medical Research, 75-59 263rd St, New York, NY 11004 (swinkelbei@northwell.edu); Philipp Homan, MD, PhD, Center for Psychiatric Neuroscience, Feinstein Institute for Medical Research, 75-59 263rd St, New York, NY 11004 (phoman1@northwell.edu).

Published Online: June 3, 2019. doi:10.1001/jamapsychiatry.2019.1530

Author Contributions: Drs Homan and Winkelbeiner had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Winkelbeiner, Leucht, Homan.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Winkelbeiner, Homan.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Winkelbeiner, Homan.

Obtained funding: Winkelbeiner.

Administrative, technical, or material support: All authors.

Supervision: Homan.

Conflict of Interest Disclosures: Dr Winkelbeiner reported grants from Swiss National Science Foundation during the conduct of the study. Dr Leucht reported personal fees from LB Pharma International, H Lundbeck A/S, Otsuka Pharmaceutical, Teva Pharmaceutical Industries Ltd, LTS Lohmann Therapy Systems, Gedeon Richter, Recordati SpA, MSD, Boehringer Ingelheim and Sandoz, Janssen Pharmaceutica, Eli Lilly & Company, SanofiAventis, and Servier Laboratorie outside of the submitted work; reanalysis of a clinical trial together with Geodon Richter and the publication of its results; and honoraria from Johnson & Johnson, MSD, Angelini, and Sunovion. Dr Kane reported grants from Otsuka, Lundbeck, and Janssen, as well as other from Alkermes, Allergan, Forum, Genentech, Lundbeck, Intracellular Therapies, Janssen, Johnson & Johnson, Merck, Neurocrine, Otsuka, Pierre Fabre, Reviva, Roche, Sunovion, Takeda, Teva, Vanguard Research Group, and LB Pharmaceuticals outside of the submitted work. No other disclosures were reported.

Meeting Presentation: The results of this study were presented at the World Congress of Biological Psychiatry, June 3, 2019, Vancouver, British Columbia, Canada.

Additional Contributions: The authors thank Majnu John, PhD, Department of Mathematics, Zucker School of Medicine at Northwell/Hofstra, for advice on the analysis of the current study, as well as Stephen Senn, PhD, Methodology and Statistics, Luxembourg Institute of Health, and Daniel Guinart, MD, Department of Psychiatry, Zucker School of Medicine at Northwell/Hofstra, for helpful comments on the manuscript. These individuals received no additional compensation, outside of their usual salary, for their contributions.

Additional Information: All data and code are freely available online to ensure reproducibility (https://doi.org/10.17605/OSF.IO/QARVS).

References
1.
Garver  DL, Holcomb  JA, Christensen  JD.  Heterogeneity of response to antipsychotics from multiple disorders in the schizophrenia spectrum.  J Clin Psychiatry. 2000;61(12):964-972. doi:10.4088/JCP.v61n1213PubMedGoogle ScholarCrossref
2.
Senn  S.  Mastering variation: variance components and personalised medicine.  Stat Med. 2016;35(7):966-977. doi:10.1002/sim.6739PubMedGoogle ScholarCrossref
3.
Senn  S.  Trying to be precise about vagueness.  Stat Med. 2007;26(7):1417-1430. doi:10.1002/sim.2639PubMedGoogle ScholarCrossref
4.
Homan  P, Kane  JM.  Clozapine as an early-stage treatment.  Acta Psychiatr Scand. 2018;138(4):279-280. doi:10.1111/acps.12965PubMedGoogle ScholarCrossref
5.
Hecksteden  A, Kraushaar  J, Scharhag-Rosenberger  F, Theisen  D, Senn  S, Meyer  T.  Individual response to exercise training - a statistical perspective.  J Appl Physiol (1985). 2015;118(12):1450-1459. doi:10.1152/japplphysiol.00714.2014PubMedGoogle ScholarCrossref
6.
Hecksteden  A, Pitsch  W, Rosenberger  F, Meyer  T.  Repeated testing for the assessment of individual response to exercise training.  J Appl Physiol (1985). 2018;124(6):1567-1579. doi:10.1152/japplphysiol.00896.2017PubMedGoogle ScholarCrossref
7.
Dworkin  RH, McDermott  MP, Farrar  JT, O’Connor  AB, Senn  S.  Interpreting patient treatment response in analgesic clinical trials: implications for genotyping, phenotyping, and personalized pain treatment.  Pain. 2014;155(3):457-460. doi:10.1016/j.pain.2013.09.019PubMedGoogle ScholarCrossref
8.
Bennett  JH, ed.  Statistical Inference and Analysis: Selected Correspondence of RA Fisher. Oxford, UK: Clarendon Press; 1990. Cited by: Senn S. Seven myths of randomisation in clinical trials. Stat Med. 2013;32(9): 1439-1450. doi:10.1002/sim.5713
9.
Cortés  J, González  JA, Medina  MN,  et al.  Does evidence support the high expectations placed in precision medicine? A bibliographic review.  F1000 Res. 2019;7:30. doi:10.12688/f1000research.13490.4Google ScholarCrossref
10.
Nakagawa  S, Poulin  R, Mengersen  K,  et al.  Meta-analysis of variation: ecological and evolutionary applications and beyond.  Methods Ecol Evol. 2015;6(2):143-152. doi:10.1111/2041-210X.12309Google ScholarCrossref
11.
Viechtbauer  W.  Conducting meta-analyses in R with the metafor package.  J Stat Softw. 2010;36(3):1-48. doi:10.18637/jss.v036.i03Google ScholarCrossref
12.
Brugger  SP, Howes  OD.  Heterogeneity and homogeneity of regional brain structure in schizophrenia: a meta-analysis.  JAMA Psychiatry. 2017;74(11):1104-1111. doi:10.1001/jamapsychiatry.2017.2663PubMedGoogle ScholarCrossref
13.
Pillinger  T, Osimo  EF, Brugger  S, Mondelli  V, McCutcheon  RA, Howes  OD.  A meta-analysis of immune parameters, variability, and assessment of modal distribution in psychosis and test of the immune subgroup hypothesis [published November 8, 2018].  Schizophr Bull. 2018. doi:10.1093/schbul/sby160PubMedGoogle Scholar
14.
Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle ScholarCrossref
15.
Leucht  S, Davis  JM.  Do antipsychotic drugs lose their efficacy for relapse prevention over time?  Br J Psychiatry. 2017;211(3):127-129. doi:10.1192/bjp.bp.117.201103PubMedGoogle ScholarCrossref
16.
Hedges  LV, Nowell  A.  Sex differences in mental test scores, variability, and numbers of high-scoring individuals.  Science. 1995;269(5220):41-45. doi:10.1126/science.7604277PubMedGoogle ScholarCrossref
17.
Litman  RE, Smith  MA, Doherty  JJ,  et al.  AZD8529, a positive allosteric modulator at the mGluR2 receptor, does not improve symptoms in schizophrenia: a proof of principle study.  Schizophr Res. 2016;172(1-3):152-157. doi:10.1016/j.schres.2016.02.001PubMedGoogle ScholarCrossref
18.
StudyRIS. Office of Clinical Pharmacology and Biopharmacy Review. NDA number: 20272. Janssen-Cilag; data on file 1996. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
19.
Daniel  DG, Zimbroff  DL, Potkin  SG, Reeves  KR, Harrigan  EP, Lakshminarayanan  M; Ziprasidone Study Group.  Ziprasidone 80 mg/day and 160 mg/day in the acute exacerbation of schizophrenia and schizoaffective disorder: a 6-week placebo-controlled trial.  Neuropsychopharmacology. 1999;20(5):491-505. doi:10.1016/S0893-133X(98)00090-6PubMedGoogle ScholarCrossref
20.
Potkin  SG, Litman  RE, Torres  R, Wolfgang  CD.  Efficacy of iloperidone in the treatment of schizophrenia: initial phase 3 studies.  J Clin Psychopharmacol. 2008;28(2)(suppl 1):S4-S11. doi:10.1097/JCP.0b013e3181692787PubMedGoogle ScholarCrossref
21.
Zborowski  J, Schmitz  P, Staser  J,  et al.  Efficacy and safety of sertindole in a trial of schizophrenic patients.  Biol Psychiatry. 1995;9(37):661-662. doi:10.1016/0006-3223(95)94656-HGoogle ScholarCrossref
22.
Marder  SR, Meibach  RC.  Risperidone in the treatment of schizophrenia.  Am J Psychiatry. 1994;151(6):825-835. doi:10.1176/ajp.151.6.825PubMedGoogle ScholarCrossref
23.
Kane  J, Canas  F, Kramer  M,  et al.  Treatment of schizophrenia with paliperidone extended-release tablets: a 6-week placebo-controlled trial.  Schizophr Res. 2007;90(1-3):147-161. doi:10.1016/j.schres.2006.09.012PubMedGoogle ScholarCrossref
24.
Marder  SR, Kramer  M, Ford  L,  et al.  Efficacy and safety of paliperidone extended-release tablets: results of a 6-week, randomized, placebo-controlled study.  Biol Psychiatry. 2007;62(12):1363-1370. doi:10.1016/j.biopsych.2007.01.017PubMedGoogle ScholarCrossref
25.
Durgam  S, Starace  A, Li  D,  et al.  An evaluation of the safety and efficacy of cariprazine in patients with acute exacerbation of schizophrenia: a phase II, randomized clinical trial.  Schizophr Res. 2014;152(2-3):450-457. doi:10.1016/j.schres.2013.11.041PubMedGoogle ScholarCrossref
26.
Casey  DE, Sands  EE, Heisterberg  J, Yang  H-M.  Efficacy and safety of bifeprunox in patients with an acute exacerbation of schizophrenia: results from a randomized, double-blind, placebo-controlled, multicenter, dose-finding study.  Psychopharmacology (Berl). 2008;200(3):317-331. doi:10.1007/s00213-008-1207-7PubMedGoogle ScholarCrossref
27.
Potkin  SG, Cohen  M, Panagides  J.  Efficacy and tolerability of asenapine in acute schizophrenia: a placebo- and risperidone-controlled trial.  J Clin Psychiatry. 2007;68(10):1492-1500. doi:10.4088/JCP.v68n1004PubMedGoogle ScholarCrossref
28.
Durgam  S, Cutler  AJ, Lu  K,  et al.  Cariprazine in acute exacerbation of schizophrenia: a fixed-dose, phase 3, randomized, double-blind, placebo- and active-controlled trial.  J Clin Psychiatry. 2015;76(12):e1574-e1582. doi:10.4088/JCP.15m09997PubMedGoogle ScholarCrossref
29.
Hirayasu  Y, Tomioka  M, Iizumi  M, Kikuchi  H.  A double-blind, placebo-controlled, comparative study of paliperidone extended release (ER) tablets in patients with schizophrenia.  Jpn J Clin Psychopharmacol. 2010;13:2077-2103.Google Scholar
30.
Hera021, 041-021SH. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research. Medical review(s); 2009. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
31.
Ogasa  M, Kimura  T, Nakamura  M, Guarino  J.  Lurasidone in the treatment of schizophrenia: a 6-week, placebo-controlled study.  Psychopharmacology (Berl). 2013;225(3):519-530. doi:10.1007/s00213-012-2838-2PubMedGoogle ScholarCrossref
32.
McEvoy  JP, Daniel  DG, Carson  WH  Jr, McQuade  RD, Marcus  RN.  A randomized, double-blind, placebo-controlled, study of the efficacy and safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients with acute exacerbations of schizophrenia.  J Psychiatr Res. 2007;41(11):895-905. doi:10.1016/j.jpsychires.2007.05.002PubMedGoogle ScholarCrossref
33.
Tzimos  A, Samokhvalov  V, Kramer  M,  et al.  Safety and tolerability of oral paliperidone extended-release tablets in elderly patients with schizophrenia: a double-blind, placebo-controlled study with six-month open-label extension.  Am J Geriatr Psychiatry. 2008;16(1):31-43. doi:10.1097/JGP.0b013e31815a3e7aPubMedGoogle ScholarCrossref
34.
ClinicalTrials.gov. A study of the efficacy and safety of asenapine in participants with an acute exacerbation of schizophrenia. Identifier: NCT01617187. https://clinicaltrials.gov/ct2/show/NCT01617187?cond=A+study+of+the+efficacy+and+safety+of+asenapine+in+participants+with+an+acute+exacerbation+of+schizophrenia&rank=1. Accessed October 31, 2016.
35.
Nasrallah  HA, Silva  R, Phillips  D,  et al.  Lurasidone for the treatment of acutely psychotic patients with schizophrenia: a 6-week, randomized, placebo-controlled study.  J Psychiatr Res. 2013;47(5):670-677. doi:10.1016/j.jpsychires.2013.01.020PubMedGoogle ScholarCrossref
36.
Correll  CU, Skuban  A, Hobart  M,  et al.  Efficacy of brexpiprazole in patients with acute schizophrenia: review of three randomized, double-blind, placebo-controlled studies.  Schizophr Res. 2016;174(1-3):82-92. doi:10.1016/j.schres.2016.04.012PubMedGoogle ScholarCrossref
37.
Canuso  CM, Schooler  N, Carothers  J,  et al.  Paliperidone extended-release in schizoaffective disorder: a randomized, controlled study comparing a flexible dose with placebo in patients treated with and without antidepressants and/or mood stabilizers.  J Clin Psychopharmacol. 2010;30(5):487-495. doi:10.1097/JCP.0b013e3181eeb600PubMedGoogle ScholarCrossref
38.
Shen  JH, Zhao  Y, Rosenzweig-Lipson  S,  et al.  A 6-week randomized, double-blind, placebo-controlled, comparator referenced trial of vabicaserin in acute schizophrenia.  J Psychiatr Res. 2014;53:14-22. doi:10.1016/j.jpsychires.2014.02.012PubMedGoogle ScholarCrossref
39.
Kane  JM, Cohen  M, Zhao  J, Alphs  L, Panagides  J.  Efficacy and safety of asenapine in a placebo- and haloperidol-controlled trial in patients with acute exacerbation of schizophrenia.  J Clin Psychopharmacol. 2010;30(2):106-115. doi:10.1097/JCP.0b013e3181d35d6bPubMedGoogle ScholarCrossref
40.
Study93202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
41.
Lieberman  JA, Davis  RE, Correll  CU,  et al.  ITI-007 for the treatment of schizophrenia: A 4-week randomized, double-blind, controlled trial.  Biol Psychiatry. 2016;79(12):952-961. doi:10.1016/j.biopsych.2015.08.026PubMedGoogle ScholarCrossref
42.
Bugarski-Kirola  D, Wang  A, Abi-Saab  D, Blättler  T.  A phase II/III trial of bitopertin monotherapy compared with placebo in patients with an acute exacerbation of schizophrenia: results from the CandleLyte study.  Eur Neuropsychopharmacol. 2014;24(7):1024-1036. doi:10.1016/j.euroneuro.2014.03.007PubMedGoogle ScholarCrossref
43.
van Kammen  DP, McEvoy  JP, Targum  SD, Kardatzke  D, Sebree  TB.  A randomized, controlled, dose-ranging trial of sertindole in patients with schizophrenia.  Psychopharmacology (Berl). 1996;124(1-2):168-175. doi:10.1007/BF02245618PubMedGoogle ScholarCrossref
44.
Study94202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
45.
Potkin  SG, Saha  AR, Kujawa  MJ,  et al.  Aripiprazole, an antipsychotic with a novel mechanism of action, and risperidone vs placebo in patients with schizophrenia and schizoaffective disorder.  Arch Gen Psychiatry. 2003;60(7):681-690. doi:10.1001/archpsyc.60.7.681PubMedGoogle ScholarCrossref
46.
Litman  RE, Smith  MA, Desai  DG, Simpson  T, Sweitzer  D, Kanes  SJ.  The selective neurokinin 3 antagonist AZD2624 does not improve symptoms or cognition in schizophrenia: a proof-of-principle study.  J Clin Psychopharmacol. 2014;34(2):199-204. doi:10.1097/JCP.0000000000000071PubMedGoogle ScholarCrossref
47.
Kinon  BJ, Zhang  L, Millen  BA,  et al; HBBI Study Group.  A multicenter, inpatient, phase 2, double-blind, placebo-controlled dose-ranging study of LY2140023 monohydrate in patients with DSM-IV schizophrenia.  J Clin Psychopharmacol. 2011;31(3):349-355. doi:10.1097/JCP.0b013e318218dcd5PubMedGoogle ScholarCrossref
48.
Kane  JM, Carson  WH, Saha  AR,  et al.  Efficacy and safety of aripiprazole and haloperidol versus placebo in patients with schizophrenia and schizoaffective disorder.  J Clin Psychiatry. 2002;63(9):763-771. doi:10.4088/JCP.v63n0903PubMedGoogle ScholarCrossref
49.
Egan  MF, Zhao  X, Smith  A,  et al.  Randomized controlled study of the T-type calcium channel antagonist MK-8998 for the treatment of acute psychosis in patients with schizophrenia.  Hum Psychopharmacol. 2013;28(2):124-133. doi:10.1002/hup.2289PubMedGoogle ScholarCrossref
50.
Durgam  S, Litman  RE, Papadakis  K, Li  D, Németh  G, Laszlovszky  I.  Cariprazine in the treatment of schizophrenia: a proof-of-concept trial.  Int Clin Psychopharmacol. 2016;31(2):61-68. doi:10.1097/YIC.0000000000000110PubMedGoogle ScholarCrossref
51.
Barbato L, Newcomer J, Heisterberg J, Yeung P, Shapira N. Efficacy and metabolic profile of bifeprunox in patients with schizophrenia. Paper presented at: 11th International Congress on Schizophrenia Research; March 28 to April 1, 2007; Colorado Springs, CO.
52.
Study049. A 6-week, double-blind, randomized, fixed dose, parallel-group study of the efficacy and safety of three dose levels of SM-13496 (lurasidone) compared to placebo and haloperidol in patients with schizophrenia who are experiencing an acute exacerbation of symptoms. Center for Drug Evaluation and Research; Medical review(s). 2010. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
53.
Meltzer  HY, Cucchiaro  J, Silva  R,  et al.  Lurasidone in the treatment of schizophrenia: a randomized, double-blind, placebo-and olanzapine-controlled study. Am J Psychiatry. 2011;168(9):957-967. doi:10.1176/appi.ajp.2011.10060907PubMed
54.
Meltzer  HY, Barbato  L, Heisterberg  J, Yeung  P, Shapira  N.  A randomized, doubleblind, placebo-controlled efficacy and safety study of bifeprunox as treatment for patients with acutely exacerbated schizophrenia.  Schizophr Bull. 2007;33(2):446.Google Scholar
55.
Loebel  A, Cucchiaro  J, Sarma  K,  et al.  Efficacy and safety of lurasidone 80 mg/day and 160 mg/day in the treatment of schizophrenia: a randomized, double-blind, placebo- and active-controlled trial.  Schizophr Res. 2013;145(1-3):101-109. doi:10.1016/j.schres.2013.01.009PubMedGoogle ScholarCrossref
56.
Kane  JM, Zukin  S, Wang  Y,  et al.  Efficacy and safety of cariprazine in acute exacerbation of schizophrenia: results from an international, phase III clinical trial.  J Clin Psychopharmacol. 2015;35(4):367-373.PubMedGoogle Scholar
57.
Garcia  E, Robert  M, Peris  F, Nakamura  H, Sato  N, Terazawa  Y.  The efficacy and safety of blonanserin compared with haloperidol in acute-phase schizophrenia: a randomized, double-blind, placebo-controlled, multicentre study.  CNS Drugs. 2009;23(7):615-625. doi:10.2165/00023210-200923070-00006PubMedGoogle ScholarCrossref
58.
Davidson  M, Emsley  R, Kramer  M,  et al.  Efficacy, safety and early response of paliperidone extended-release tablets (paliperidone ER): results of a 6-week, randomized, placebo-controlled study.  Schizophr Res. 2007;93(1-3):117-130. doi:10.1016/j.schres.2007.03.003PubMedGoogle ScholarCrossref
59.
Canuso  CM, Lindenmayer  J-P, Kosik-Gonzalez  C,  et al.  A randomized, double-blind, placebo-controlled study of 2 dose ranges of paliperidone extended-release in the treatment of subjects with schizoaffective disorder.  J Clin Psychiatry. 2010;71(5):587-598. doi:10.4088/JCP.09m05564yelPubMedGoogle ScholarCrossref
60.
Hera022, SH H 041-022. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research; Medical review(s); 2009.Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
61.
Nakamura  M, Ogasa  M, Guarino  J,  et al.  Lurasidone in the treatment of acute schizophrenia: a double-blind, placebo-controlled trial.  J Clin Psychiatry. 2009;70(6):829-836. doi:10.4088/JCP.08m04905PubMedGoogle ScholarCrossref
62.
Study115. Center for Drug Evaluation and Research. Approval package for application number 20-825. Medical review(s); 2000. Cited by: Leucht  S, Leucht  C, Huhn  M,  et al.  Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors.  Am J Psychiatry. 2017;174(10):927-942. doi:10.1176/appi.ajp.2017.16121358PubMedGoogle Scholar
63.
Beasley  CM  Jr, Sanger  T, Satterlee  W, Tollefson  G, Tran  P, Hamilton  S.  Olanzapine versus placebo: results of a double-blind, fixed-dose olanzapine trial.  Psychopharmacology (Berl). 1996;124(1-2):159-167. doi:10.1007/BF02245617PubMedGoogle ScholarCrossref
64.
Lindenmayer  J-P, Brown  D, Liu  S, Brecher  M, Meulien  D.  The efficacy and tolerability of once-daily extended release quetiapine fumarate in hospitalized patients with acute schizophrenia: a 6-week randomized, double-blind, placebo-controlled study.  Psychopharmacol Bull. 2008;41(3):11-35.PubMedGoogle Scholar
65.
Schmidt  ME, Kent  JM, Daly  E,  et al.  A double-blind, randomized, placebo-controlled study with JNJ-37822681, a novel, highly selective, fast dissociating D2 receptor antagonist in the treatment of acute exacerbation of schizophrenia.  Eur Neuropsychopharmacol. 2012;22(10):721-733. doi:10.1016/j.euroneuro.2012.02.007PubMedGoogle ScholarCrossref
66.
Meltzer  HY, Arvanitis  L, Bauer  D, Rein  W; Meta-Trial Study Group.  Placebo-controlled evaluation of four novel compounds for the treatment of schizophrenia and schizoaffective disorder.  Am J Psychiatry. 2004;161(6):975-984. doi:10.1176/appi.ajp.161.6.975PubMedGoogle ScholarCrossref
67.
Coppola  D, Melkote  R, Lannie  C,  et al.  Efficacy and safety of paliperidone extended release 1.5 mg/day-a double-blind, placebo- and active-controlled, study in the treatment of patients with schizophrenia.  Psychopharmacol Bull. 2011;44(2):54-72.PubMedGoogle Scholar
68.
Chouinard  G, Jones  B, Remington  G,  et al.  A Canadian multicenter placebo-controlled study of fixed doses of risperidone and haloperidol in the treatment of chronic schizophrenic patients.  J Clin Psychopharmacol. 1993;13(1):25-40. doi:10.1097/00004714-199302000-00004PubMedGoogle ScholarCrossref
69.
Senn  S.  Seven myths of randomisation in clinical trials.  Stat Med. 2013;32(9):1439-1450. doi:10.1002/sim.5713PubMedGoogle ScholarCrossref
70.
Leucht  S, Davis  JM.  Enthusiasm and skepticism about using national registers to analyze psychotropic drug outcomes.  JAMA Psychiatry. 2018;75(4):314-315. doi:10.1001/jamapsychiatry.2017.4704PubMedGoogle ScholarCrossref
71.
Kubo  K, Fleischhacker  WW, Suzuki  T, Yasui-Furukori  N, Mimura  M, Uchida  H.  Placebo effects in adult and adolescent patients with schizophrenia: combined analysis of nine RCTs.  Acta Psychiatr Scand. 2019;139(2):108-116. doi:10.1111/acps.12960PubMedGoogle ScholarCrossref
72.
Hróbjartsson  A, Gøtzsche  PC.  Is the placebo powerless? an analysis of clinical trials comparing placebo with no treatment.  N Engl J Med. 2001;344(21):1594-1602. doi:10.1056/NEJM200105243442106PubMedGoogle ScholarCrossref
73.
Senn  S.  Applying results of randomised trials to patients. N of 1 trials are needed.  BMJ. 1998;317(7157):537-538. doi:10.1136/bmj.317.7157.537PubMedGoogle ScholarCrossref
74.
Wang  R, Lagakos  SW, Ware  JH, Hunter  DJ, Drazen  JM.  Statistics in medicine–reporting of subgroup analyses in clinical trials.  N Engl J Med. 2007;357(21):2189-2194. doi:10.1056/NEJMsr077003PubMedGoogle ScholarCrossref
75.
Araujo  A, Julious  S, Senn  S.  Understanding variation in sets of N-of-1 trials.  PLoS One. 2016;11(12):e0167167. doi:10.1371/journal.pone.0167167PubMedGoogle ScholarCrossref
76.
Joyce  DW, Tracy  DK, Shergill  SS.  Are we failing clinical trials? a case for strong aggregate outcomes.  Psychol Med. 2018;48(2):177-186. doi:10.1017/S0033291717001726PubMedGoogle ScholarCrossref
77.
González  AB, Cox  DR.  Interpretation of interaction: a review.  Ann Appl Stat. 2007;1(2):371-385. doi:10.1214/07-AOAS124Google ScholarCrossref
×