Placebo Response and Practice Effects in Schizophrenia Cognition Trials | Dementia and Cognitive Impairment | JAMA Psychiatry | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Mean Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) Cognitive Composite T Scores by Study and Visit for Patients Receiving Placebo
Mean Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) Cognitive Composite T Scores by Study and Visit for Patients Receiving Placebo

Unadjusted mean MCCB cognitive composite scores are plotted by study and week of assessment through week 24. The line color denotes studies that assessed cognition at a screening visit (“Yes”) and those that began assessments at baseline (“No”). The line thickness is proportional to the study sample size, and assessment time points for each study are denoted by markers.

Figure 2.  Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Study for Patients Receiving Placebo
Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Study for Patients Receiving Placebo

Least squares mean change, 95% CIs, sample size, and number of postbaseline visits are presented for each study and overall. Individual study results are from separate mixed-effects repeated measures models of the change from baseline through week 24, with a fixed categorical effect for visit and a continuous fixed covariate for the baseline score, assuming an unstructured covariance matrix. In the overall model, visit was nested within a random study effect.

Figure 3.  Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Level of Practice Effect in Studies With a Screening Assessment
Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Level of Practice Effect in Studies With a Screening Assessment

Least squares mean change from baseline through week 24 is plotted by level of practice effect as calculated by the T-score change in the cognitive composite from screening to baseline. Results are from a mixed-effect repeated measures model with a continuous fixed covariate for the baseline score and a categorical effect for visit nested within a random study effect, assuming an unstructured covariance matrix. This analysis was limited to the 7 studies2,3,24-26 that assessed cognition at screening (N = 558) and revealed a significantly increasing placebo effect with decreased practice effect between screening and baseline (P < .001 overall and for all pairwise comparisons). Patients were classified into 3 groups based on the magnitude of the change in their screening to baseline practice effect: negative (change of ≤–5 points), small or neutral (change of >–5 to 5 points), and positive (>5-point change). Forty-two patients without composite scores at screening, baseline, and at least 1 follow-up were excluded from the analysis (n = 516). Test for practice effect-by-week interaction was nonsignificant (P = .67).

Table.  Demographic and Baseline Characteristics
Demographic and Baseline Characteristics
1.
Keefe  RS, Meltzer  HA, Dgetluck  N,  et al.  Randomized, double-blind, placebo-controlled study of encenicline, an α7 nicotinic acetylcholine receptor agonist, as a treatment for cognitive impairment in schizophrenia.  Neuropsychopharmacology. 2015;40(13):3053-3060.PubMedGoogle ScholarCrossref
2.
Haig  GM, Bain  EE, Robieson  WZ, Baker  JD, Othman  AA.  A randomized trial to assess the efficacy and safety of ABT-126, a selective α7 nicotinic acetylcholine receptor agonist, in the treatment of cognitive impairment in schizophrenia.  Am J Psychiatry. 2016;173(8):827-835.PubMedGoogle ScholarCrossref
3.
Bugarski-Kirola  D, Blaettler  T, Arango  C,  et al.  Bitopertin in negative symptoms of schizophrenia—results from the phase III FlashLyte and DayLyte studies  [published online December 15, 2016].  Biol Psychiatry. 2016. doi:10.1016/j.biopsych.2016.11.014PubMedGoogle Scholar
4.
Umbricht  D, Alberati  D, Martin-Facklam  M,  et al.  Effect of bitopertin, a glycine reuptake inhibitor, on negative symptoms of schizophrenia: a randomized, double-blind, proof-of-concept study.  JAMA Psychiatry. 2014;71(6):637-646.PubMedGoogle ScholarCrossref
5.
Keefe  RS, Buchanan  RW, Marder  SR,  et al.  Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far?  Schizophr Bull. 2013;39(2):417-435. PubMedGoogle ScholarCrossref
6.
Schooler  NR, Buchanan  RW, Laughren  T,  et al.  Defining therapeutic benefit for people with schizophrenia: focus on negative symptoms.  Schizophr Res. 2015;162(1-3):169-174.PubMedGoogle ScholarCrossref
7.
Hogarty  GE, Greenwald  D, Ulrich  RF,  et al.  Three-year trials of personal therapy among schizophrenic patients living with or independent of family, II: effects on adjustment of patients.  Am J Psychiatry. 1997;154(11):1514-1524.PubMedGoogle ScholarCrossref
8.
Eack  SM, Hogarty  GE, Greenwald  DP, Hogarty  SS, Keshavan  MS.  Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: results from a two-year randomized trial.  Res Soc Work Pract. 2011;21(1):32-42.PubMedGoogle ScholarCrossref
9.
Fisher  M, Holland  C, Merzenich  MM, Vinogradov  S.  Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia.  Am J Psychiatry. 2009;166(7):805-811.PubMedGoogle ScholarCrossref
10.
Fisher  M, Mellon  SH, Wolkowitz  O, Vinogradov  S.  Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor.  Schizophr Res Cogn. 2016;3:1-7.PubMedGoogle ScholarCrossref
11.
Mahncke  H, Kim  SJ, Stasio  C,  et al. Results of an FDA device clearance trial for Plasticity-Based Adaptive Cognitive Remediation (PACR). Paper presented at: 5th Schizophrenia International Research Society Conference; April 5, 2016; Florence, Italy.
12.
Kantrowitz  JT, Sharif  Z, Medalia  A,  et al.  A multicenter, rater-blinded, randomized controlled study of auditory processing–focused cognitive remediation combined with open-label lurasidone in patients with schizophrenia and schizoaffective disorder.  J Clin Psychiatry. 2016;77(6):799-806.PubMedGoogle ScholarCrossref
13.
Wykes  T, Huddy  V, Cellard  C, McGurk  SR, Czobor  P.  A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes.  Am J Psychiatry. 2011;168(5):472-485.PubMedGoogle ScholarCrossref
14.
Buchanan  RW, Davis  M, Goff  D,  et al.  A summary of the FDA-NIMH-MATRICS workshop on clinical trial design for neurocognitive drugs for schizophrenia.  Schizophr Bull. 2005;31(1):5-19.PubMedGoogle ScholarCrossref
15.
Buchanan  RW, Keefe  RSE, Umbricht  D, Green  MF, Laughren  T, Marder  SR.  The FDA-NIMH-MATRICS guidelines for clinical trial design of cognitive-enhancing drugs: what do we know 5 years later?  Schizophr Bull. 2011;37(6):1209-1217.PubMedGoogle ScholarCrossref
16.
Green  MF, Nuechterlein  KH, Kern  RS,  et al.  Functional co-primary measures for clinical trials in schizophrenia: results from the MATRICS Psychometric and Standardization Study.  Am J Psychiatry. 2008;165(2):221-228.PubMedGoogle ScholarCrossref
17.
Nuechterlein  KH, Green  MF, Kern  RS,  et al.  The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity.  Am J Psychiatry. 2008;165(2):203-213.PubMedGoogle ScholarCrossref
18.
Kern  RS, Nuechterlein  KH, Green  MF,  et al.  The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization.  Am J Psychiatry. 2008;165(2):214-220.PubMedGoogle ScholarCrossref
19.
Umbricht  D, Keefe  RS, Murray  S,  et al.  A randomized, placebo-controlled study investigating the nicotinic α7 agonist, RG3487, for cognitive deficits in schizophrenia.  Neuropsychopharmacology. 2014;39(7):1568-1577.PubMedGoogle ScholarCrossref
20.
Javitt  DC, Buchanan  RW, Keefe  RSE,  et al.  Effect of the neuroprotective peptide davunetide (AL-108) on cognition and functional capacity in schizophrenia.  Schizophr Res. 2012;136(1-3):25-31.PubMedGoogle ScholarCrossref
21.
Goldberg  TE, Keefe  RS, Goldman  RS, Robinson  DG, Harvey  PD.  Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies.  Neuropsychopharmacology. 2010;35(5):1053-1062.PubMedGoogle ScholarCrossref
22.
Keefe  RS, Goldberg  TE, Harvey  PD, Gold  JM, Poe  MP, Coughenour  L.  The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery.  Schizophr Res. 2004;68(2-3):283-297.PubMedGoogle ScholarCrossref
23.
Papakostas  GI, Østergaard  SD, Iovieno  N.  The nature of placebo response in clinical studies of major depressive disorder.  J Clin Psychiatry. 2015;76(4):456-466.PubMedGoogle ScholarCrossref
24.
Haig  GM, Bain  E, Robieson  W, Othman  AA, Baker  J, Lenz  RA.  A randomized trial of the efficacy and safety of the H3 antagonist ABT-288 in cognitive impairment associated with schizophrenia.  Schizophr Bull. 2014;40(6):1433-1442.PubMedGoogle ScholarCrossref
25.
Clinicaltrials.gov. A phase 2 study to evaluate ABT-126 for the treatment of cognitive deficits in schizophrenia. NCT01678755. https://clinicaltrials.gov/ct2/show/NCT01678755. Accessed April 4, 2015.
26.
Haig  G, Wang  D, Othman  AA, Zhao  J.  The α7 nicotinic agonist ABT-126 in the treatment of cognitive impairment associated with schizophrenia in nonsmokers: results from a randomized controlled phase 2b study.  Neuropsychopharmacology. 2016;41(12):2893-2902.PubMedGoogle ScholarCrossref
27.
Buchanan  RW, Keefe  RS, Lieberman  JA,  et al.  A randomized clinical trial of MK-0777 for the treatment of cognitive impairments in people with schizophrenia.  Biol Psychiatry. 2011;69(5):442-449.PubMedGoogle ScholarCrossref
28.
Clinicaltrials.gov. Efficacy and safety of AVE1625 as a co-treatment with antipsychotic therapy in schizophrenia (CONNECT). NCT00439634. https://clinicaltrials.gov/ct2/show/NCT00439634. Accessed January 10, 2015.
29.
Clinicaltrials.gov. A study of LY2140023 in schizophrenia patients with prominent negative symptoms. NCT01052103. https://clinicaltrials.gov/ct2/show/NCT01052103. Accessed June 21, 2015.
30.
Patterson  TL, Goldman  S, McKibbin  CL, Hughs  T, Jeste  DV.  UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults.  Schizophr Bull. 2001;27(2):235-245. PubMedGoogle ScholarCrossref
31.
Georgiades  A, Davis  VG, Atkins  A,  et al.  Psychometric characteristics of the MATRICS Consensus Cognitive Battery in a large pooled cohort of stable schizophrenia patients  [published online April 19, 2017].  Schizophr Res. doi:10.1016/j.schres.2017.03.040PubMedGoogle Scholar
32.
Green  MF, Harris  JG, Nuechterlein  KH.  The MATRICS consensus cognitive battery: what we know 6 years later.  Am J Psychiatry. 2014;171(11):1151-1154.PubMedGoogle ScholarCrossref
33.
Kay  SR, Fiszbein  A, Opler  LA.  The Positive and Negative Syndrome Scale (PANSS) for schizophrenia.  Schizophr Bull. 1987;13(2):261-276.PubMedGoogle ScholarCrossref
34.
Marder  SR, Davis  JM, Chouinard  G.  The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: combined results of the North American trials.  J Clin Psychiatry. 1997;58(12):538-546.PubMedGoogle ScholarCrossref
35.
Alphs  LD, Summerfelt  A, Lann  H, Muller  RJ.  The negative symptom assessment: a new instrument to assess negative symptoms of schizophrenia.  Psychopharmacol Bull. 1989;25(2):159-163.PubMedGoogle Scholar
36.
Axelrod  BN, Goldman  RS, Alphs  LD.  Validation of the 16-item negative symptom assessment.  J Psychiatr Res. 1993;27(3):253-258.PubMedGoogle ScholarCrossref
37.
Cohen  J, ed.  Statistical Power Analysis for the Behavioral Sciences. Orlando, FL: Academic Press, Inc; 1977.
38.
Ivanova  A, Qaqish  B, Schoenfeld  DA.  Optimality, sample size, and power calculations for the sequential parallel comparison design.  Stat Med. 2011;30(23):2793-2803.PubMedGoogle ScholarCrossref
39.
Doros  G, Pencina  M, Rybin  D, Meisner  A, Fava  M.  A repeated measures model for analysis of continuous outcomes in sequential parallel comparison design studies.  Stat Med. 2013;32(16):2767-2789.PubMedGoogle ScholarCrossref
40.
Brannan  S. The difficulties in going from P2 to P3 in CNS trials: red flags from a recent CIS program. Paper presented at: 13th Annual Scientific Meeting of the International Society of CNS Clinical Trials; February 23, 2017; Washington, DC.
41.
Peterson  JB, Rothfleisch  J, Zelazo  PD, Pihl  RO.  Acute alcohol intoxication and cognitive functioning.  J Stud Alcohol. 1990;51(2):114-122.PubMedGoogle ScholarCrossref
42.
Smith  RC, Warner-Cohen  J, Matute  M,  et al.  Effects of nicotine nasal spray on cognitive function in schizophrenia.  Neuropsychopharmacology. 2006;31(3):637-643.PubMedGoogle ScholarCrossref
43.
Keefe  RS, Bilder  RM, Davis  SM,  et al; CATIE Investigators; Neurocognitive Working Group.  Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE Trial.  Arch Gen Psychiatry. 2007;64(6):633-647.PubMedGoogle ScholarCrossref
44.
Szeszko  PR, Bilder  RM, Dunlop  JA, Walder  DJ, Lieberman  JA.  Longitudinal assessment of methylphenidate effects on oral word production and symptoms in first-episode schizophrenia at acute and stabilized phases.  Biol Psychiatry. 1999;45(6):680-686.PubMedGoogle ScholarCrossref
45.
Kantrowitz  JT, Epstein  ML, Beggel  O,  et al.  Neurophysiological mechanisms of cortical plasticity impairments in schizophrenia and modulation by the NMDA receptor agonist D-serine.  Brain. 2016;139(pt 12):3281-3295.PubMedGoogle ScholarCrossref
Original Investigation
August 2017

Placebo Response and Practice Effects in Schizophrenia Cognition Trials

Author Affiliations
  • 1Department of Psychiatry and Behavioral Sciences, Duke University, Duke University Medical Center, Durham, North Carolina
  • 2NeuroCog Trials, Durham, North Carolina
  • 3Department of Psychiatry and Behavioral Sciences, University of Miami, Miami, Florida
  • 4Neuroscience Development, Abbvie, North Chicago, Illinois
  • 5Research and Development, Immunology and Inflammation, Sanofi, Bridgewater, New Jersey
  • 6Semel Institute for Neuroscience at the University of California, Los Angeles
  • 7Drug Development, FORUM Pharmaceuticals, Waltham, Massachusetts
  • 8Neuroscience, Ophthalmology, Rare Diseases, Roche Pharma Research and Early Development, Roche Innovation Center Basel, F. Hoffmann–La Roche Ltd, Basel, Switzerland
JAMA Psychiatry. 2017;74(8):807-814. doi:10.1001/jamapsychiatry.2017.1574
Key Points

Questions  What is the magnitude of learning effects in patients with schizophrenia receiving placebo on the criterion standard Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB), and can these learning effects be anticipated in individual patients?

Findings  This blinded review of 12 trials including 813 patients with schizophrenia suggested that the mean change in the MCCB during receipt of placebo was less than 0.2 SDs, a small effect. Factors associated with greater improvements on the MCCB included more motivation, more depression/anxiety, and less improvement from screening to baseline.

Meaning  These changes in the MCCB are too small to be responsible for effective active treatments failing to separate from placebo.

Abstract

Importance  Patients’ previous experience with performance-based cognitive tests in clinical trials for cognitive impairment associated with schizophrenia can create practice-related improvements. Placebo-controlled trials for cognitive impairment associated with schizophrenia are at risk for these practice effects, which can be difficult to distinguish from placebo effects.

Objectives  To conduct a systematic evaluation of the magnitude of practice effects on the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) in cognitive impairment associated with schizophrenia and to examine which demographic, clinical, and cognitive characteristics were associated with improvement in placebo conditions.

Design, Setting, and Participants  A blinded review was conducted of data from 813 patients with schizophrenia who were treated with placebo in 12 randomized placebo-controlled clinical trials conducted mostly in outpatient clinics in North America, Europe, Asia, and Latin America from February 22, 2007, to March 1, 2014. A total of 779 patients provided data for the primary outcome measure at baseline and at least 1 follow-up. Seven trials had prebaseline assessments wherein the patients knew that they were not receiving treatment, allowing a comparison of practice and placebo effects in the same patients.

Interventions  Placebo compared with various experimental drug treatments.

Main Outcomes and Measures  Composite score on the MCCB.

Results  Of the 813 patients in the study (260 women and 553 men; mean [SD] age, 41.2 [11.5] years), the mean MCCB composite score at baseline was 22.8 points below the normative mean, and the mean (SEM) total change in the MCCB during receipt of placebo was 1.8 (0.2) T-score points (95% CI, 1.40-2.18), equivalent to a change of 0.18 SD. Practice effects in the 7 studies in which there was a prebaseline assessment were essentially identical to the postbaseline placebo changes. Baseline factors associated with greater improvements in the MCCB during receipt of placebo included more depression/anxiety (F1,438 = 5.41; P = .02), more motivation (F1,272 = 4.63; P = .03), and less improvement from screening to baseline (F1,421 = 59.32; P < .001).

Conclusions and Relevance  Placebo effects were minimal and associated with the number of postbaseline assessments and several patient characteristics. Given that the patients performed 2.28 SDs below normative standards on average at baseline, a mean placebo-associated improvement of less than 0.2 SD provides evidence that ceiling effects do not occur in these trials. These minimal changes in the MCCB could not be responsible for effective active treatments failing to separate from placebo.

Introduction

Cognitive impairment associated with schizophrenia is severe and a primary cause of poor functional outcomes. Extensive resources have been devoted to developing pharmacologic,1-6 psychosocial,7,8 and cognitive remediation9-13 treatments to reduce cognitive and functional impairment in patients with schizophrenia. To our knowledge, no treatments to date have had sufficient benefit to warrant regulatory approval. Because there are no effective medical treatments for cognitive impairment associated with schizophrenia, it is challenging to determine whether the lack of success of previous treatment development programs has been due to inadequacies of the treatments being tested or to weaknesses in the methods used.

Cognitive enhancement studies have typically used a regulatory pathway that was developed during the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative.14,15 Central to this method has been the assessment of cognition with a set of performance-based tests referred to as the MATRICS Consensus Cognitive Battery (MCCB),16-18 which was selected on the basis of several criteria, including usefulness as a repeated measure, defined as high test-retest reliability combined with minimal practice effects. However, the MCCB validation study used a single test-retest assessment,17 which does not reflect the complexity of standard clinical trials that have more than 2 assessments performed with the MCCB1,2,19,20 and multiple treatment conditions.1-4

Patients with schizophrenia manifest consistent cognitive performance over time with a variety of different tests17,21,22 and may generate practice effects when they are reassessed.21 However, most clinical trials have used parallel group designs with a placebo control; thus, it is difficult to determine if improved performance over time in patients receiving placebo is due to simple learning or practice effects or to the expectation bias inherent in a placebo-controlled design. A comparison of the improvement in patients receiving placebo with those who were aware that they were not receiving any treatment would help to determine if expectation bias increases the known improvement due to practice effects.

Regardless of their source, improvements on cognitive tests in patients receiving placebo may be so large that treatment effects are impossible to detect. The opposite speculation has equivalent face validity and has been supported by single studies1,19,20; if an individual is familiar with being assessed and with the assessment materials, additional assessments are less likely to lead to gains in performance based solely on test exposure. Furthermore, the magnitude of expected practice and placebo response of the MCCB, which is used in most large multisite clinical trials for cognitive impairment associated with schizophrenia, is unknown. Finally, while certain patients with schizophrenia and depression who are receiving placebo demonstrate large symptom responses that can be anticipated based on their baseline characteristics,23 it has not been determined if patients receiving placebo who respond in schizophrenia cognition trials can be similarly identified.

There are clear implications in these questions for the interpretation of previous results and the design of future clinical trials. If individual differences among patients, including differences in their performance change from screening to baseline, are associated with postbaseline retest changes, these measures could be used as factors to be considered in statistical analysis or in patient randomization and stratification.

In this study, we merged data from 12 separate clinical trials,1-3,20,24-29 including 813 patients who were randomized to receive placebo during the course of a double-blind pharmacologic intervention study, and investigated the characteristics of retest-associated improvements in performance on the MCCB. We compared prerandomization improvements with postrandomization improvements in patients receiving placebo, and we assessed the effect of a number of measures on the magnitude of postbaseline improvement, including the number of retest assessments, the spacing of the retest assessments, the time elapsed since baseline while receiving placebo, and the number of treatment arms of the study. Finally, we examined a variety of patient demographic and clinical characteristics to determine if they are associated with cognitive placebo response.

Methods
Sample Characteristics

This review included 813 patients with schizophrenia who were treated with placebo in randomized clinical trials from February 22, 2007, to March 1, 2014. The Table describes the demographic and baseline characteristics of the merged data set, with mean (SD) values for continuous variables and numbers (percentages) for categorical variables.30 As is typical of clinical trial data on patients with schizophrenia, approximately two-thirds of the patients (553 [68.0%]) were men. Most of the patients were evaluated in North America, with samples also collected in Europe, Asia, and Latin America. eTable 1 in the Supplement describes the characteristics of the 12 trials from which the data were merged, including ClinicalTrials.gov identification number, total number of individuals randomized and assigned to receive placebo, dates of the trial, the number of treatment groups, and the number of MCCB assessments performed. Each clinical trial was approved by central and/or local institutional review boards, and all study participants provided written informed consent.

Thirty-one patients had 1 or more missing MCCB domain values at the baseline visit, resulting in 782 patients with a cognitive composite score derived from all 7 MCCB domains at baseline. Three patients did not have at least 1 follow-up score, resulting in a final sample size of 779. An additional 54 patients had test results missing in a domain that had additional tests, so a domain score was able to be calculated. For all of the trials, patients were treated with a stable dose of antipsychotic medication, and most patients were receiving only 1 antipsychotic. The mean (SD) MCCB cognitive composite score at first assessment was 27.2 (12.78), consistent with similarly timed assessment scores from a larger data set of more than 4000 patients.31 This score reflects performance that is more than 2 SDs below normative standards. An improvement of 1 point on the MCCB cognitive composite score reflects a 0.1-SD improvement in normative scores, which means that the patients were a mean (SD) 23.0 (2.3) points below the normative population mean and at least a mean (SD) of 73.0 (7.3) points below a maximum score on the MCCB composite.

MCCB Assessments

The MCCB subtests are organized into the following 7 domains: (1) Speed of Processing: Trailmaking Test, Brief Assessment of Cognition Symbol Coding, and Category Fluency; (2) Attention-Vigilance: Continuous Performance Test-Identical Pairs; (3) Working memory: Wechsler Memory Scale–III Spatial Span and Letter-Number Span; (4) Verbal learning: Hopkins Verbal Learning Test-Revised; (5) Visual learning: Brief Visuospatial Memory Test-Revised; (6) Reasoning and Problem Solving: Neuropsychological Assessment Battery Mazes; and (7) Social Cognition: Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) Managing Emotions.

Data Quality Assurance

All testers who interacted with patients were first trained on the administration and scoring of the MCCB using video and group training sessions and were individually certified by an MCCB expert. All MCCB data were scored locally or sent to a central site, where they were scored or rescored. Training, data collection, and data quality assurance were implemented or supervised by an experienced psychologist (R.S.E.K. or A.S.A.) as per the guidelines outlined in the MCCB manual.17

MCCB Composite Scores

For the 10 subtests, 7 domains, and 2 composite scores of the MCCB, the scoring program yields T scores that are standardized and corrected for age and sex, based on previously collected normative data matched for age, sex, and race/ethnicity to the US Census. Each standardized measure has a mean (SD) of 50 (10).17 The cognitive composite is the standardized total of the 7 domains. The neurocognitive composite is calculated similarly but does not include Social Cognition.32

Symptom Assessments
Positive and Negative Syndrome Scale

Data from the Positive and Negative Syndrome Scale (PANSS)33 were available in 7 studies.2,3,24-26 The PANSS symptoms were grouped into 5 factors (positive symptoms, negative symptoms, disorganized thought, hostility/excitement, and depression/anxiety), based on a previously established model.34

Negative Symptom Assessment

In 5 studies,3,25,26 the Negative Symptom Assessment35,36 was used to examine negative symptoms in 4 factors: communication, emotion, motivation, and sociality.

Statistical Analysis

Data were combined from the intent-to-treat populations of 12 MCCB clinical trials. Visit schedules varied considerably among the studies, ranging from 2 to 6 MCCB assessments (1-4 postbaseline reassessments) during a period of 4 to 24 weeks. Change from baseline in the MCCB cognitive composite score was investigated using a basic linear mixed model for repeated measures, controlling for baseline value and week of assessment nested within a random study effect. The ability of patient-level and study-level variables to estimate the magnitude of the placebo effect was analyzed with separate mixed models for each predictor. Change in MCCB composite score between screening and baseline (“pure” practice effect) was compared between studies with analysis of covariance adjusting for the screening value. This analysis included the 7 studies2,3,24-26 (n = 558) that administered the MCCB prior to and again at patients’ baseline assessment, during a period in which patients knew that they were not receiving treatment. P < .05 was considered significant. Complete information on the statistical plan is in the eAppendix in the Supplement.

Results

The mean MCCB composite score at baseline was 22.8 points below the normative mean (Table). Figure 1 describes the mean MCCB cognitive composite score over time in all 12 studies. The full 56-week data for 3 studies are available in the eFigure in the Supplement. The overall mean (SEM) change in the MCCB cognitive composite during receipt of placebo, after adjusting for baseline score, week of assessment, and study, was 1.8 (0.2) T-score points (95% CI, 1.40-2.18), with a range across studies of 0.6 to 3.3 points (Figure 2).1-3,20,24-29 The effects of baseline score (F1,642 = 7.52), week of assessment (F11,181 = 3.43), and study (F11,184 = 2.82) were all statistically significant (P < .01). Mean (SEM) change in scores for the 10 subtests comprising the MCCB were all small, ranging from 0.2 (0.3) (MSCEIT managing emotions) to 2.2 (0.3) (Trail Making A) T-score points (eTable 2 in the Supplement).

Prerandomization Practice Effects

Seven of the studies had a prebaseline screening assessment (n = 558).2,3,24-26 The changes from screening to baseline assessment for the cognitive composite score varied from 0.7 to 3.1 MCCB T-score points (mean [SEM], 2.2 [0.25] T-score points). The analysis of covariance for the practice effects found a significant effect of retesting (F1,502 = 10.98; P = .001). The effect of the study was nonsignificant (F6,502 = 1.69; P = .12), indicating that the retest effects were not significantly different across studies.

Postbaseline Changes in Patients Receiving Placebo

To examine the effects of a prebaseline screening assessment on postbaseline placebo effects, the postbaseline changes in the 7 studies2,3,24-26 with a screening assessment were compared with the changes for the 5 studies1,20,27-29 without such an assessment. When a dichotomous variable indicating the presence or absence of a screening MCCB assessment was added to the model as a covariate for the postbaseline analyses, its effect was not statistically significant (F1,695 = 1.64; P = .20), indicating that performing prebaseline screening assessments was not associated with differences in postbaseline changes while the patients were receiving placebo.

We also explored the association between postbaseline changes and the numbers of weeks and assessments that had occurred since baseline, adjusting for baseline scores and whether the study had a screening visit. The effect of time since baseline was statistically significant (F1,507 = 15.28; P < .001), as was the number of retest assessments (F1,463 = 16.39; P < .001). A screening visit did not exert a statistically significant influence on postbaseline change (F1,649 = 3.03; P = .08). The mean (SEM) change in MCCB composite scores per week, after controlling for the other factors, was 0.08 (0.02) T-score points per week of placebo exposure and 0.73 (0.18) T-score points per follow-up assessment. The increases by assessment were 1.2 MCCB T-score points for 1 postbaseline assessments, 1.9 points for 2 postbaseline assessments, 2.2 points for 3 postbaseline assessments, and 3.6 points for 4 postbaseline assessments. Thus, the points gained per retest assessment during receipt of placebo generally decreased with an increased number of assessments, and the retesting effects were larger than the effects of time of placebo exposure alone.

Factors Associated With Placebo Response

The following variables were associated with placebo response: the Negative Symptoms Assessment motivation factor score (F1,272 = 4.63; P = .03), the PANSS Depression/Anxiety Marder Factor score (F1,438 = 5.41; P = .02), the practice effect between screening and baseline MCCB (F1,421 = 59.32; P < .001), 4 vs 3 treatment groups (F1,652 = 5.14; P = .02), and the study sample size (F1,725 = 4.65; P = .03). A larger placebo response was associated with more motivation, greater depression/anxiety, lower prebaseline practice effects, 4 vs 3 treatment groups, and a larger study sample.

Given the robustness of the screening to baseline MCCB composite score change as a factor associated with response, we further investigated the effect of classifying patients into the following 3 groups based on the magnitude of the change in their screening to baseline practice effect: negative (≤−5-point change), small or neutral (>−5- to 5-point change), and positive (>5-point change). As demonstrated in Figure 3, there were robust differences between the groups. Patients who manifested negative practice effects (mean [SEM], 4.8 [0.68] T-score points) demonstrated larger placebo effects (t425 = 3.43; P < .001) than those with small practice effects (mean [SEM], 2.2 [0.26] T-score points), who in turn demonstrated larger placebo effects (t410 = 4.13; P < .001) than those who had positive practice effects (mean [SEM], −0.1 [0.51] T-score points).

Discussion

The data from this study were generated from the placebo condition in 12 separate randomized, placebo-controlled clinical trials using the MCCB as the primary end point to assess response to treatment with a pharmacologic intervention in patients with schizophrenia. The mean amount of total improvement during treatment in patients receiving placebo was approximately 2 T-score points on the MCCB composite score, consistent with a Cohen d effect size of 0.18, which is slightly smaller than the Cohen definition of small.37 The amount of change due to the practice effect from prebaseline to baseline was almost identical to the mean amount of change reported in placebo conditions. Studies using a prebaseline assessment did not have smaller postbaseline placebo treatment effects than did studies with the first MCCB assessment at baseline. The size of the placebo effect was more influenced by the number of postbaseline retest assessments than by the duration of placebo treatment, with the number of reassessments associated with an increase in scores. Key additional factors associated with the magnitude of placebo response in these trials included smaller improvement from prebaseline to baseline assessment with the MCCB, less severe negative symptoms, more severe depression/anxiety symptoms, 4 vs 3 treatment groups in the trial, and larger sample size.

There are several important implications of these findings. The most obvious conclusion from these data is that the concept of a substantial placebo effect in schizophrenia clinical trials using the MCCB—one that may obscure the detection of meaningful clinical change—is clearly not substantiated. The data in this report suggest that cognitive performance assessed with the MCCB is 2.28 SDs below the mean of the healthy general population at baseline. Thus, the 0.18-SD change reported with placebo is exceeded by the size of the baseline cognitive impairment in patients with schizophrenia by a factor of 12. This amount of postbaseline improvement is almost identical to the amount of prebaseline improvement in these trials and is consistent with the amount of prebaseline improvement reported in a larger sample of 1814 patients from 10 trials.31 Thus, the amount of improvement in the placebo group in cognition clinical trials is consistent with the expected amount of improvement due to learning effects and is small enough to be easily overwhelmed by a legitimate treatment effect. The notion that this magnitude of retest improvement accounts for the negative results in previous cognitive enhancement trials seems unlikely. For the cognitive changes associated with a pharmacologic agent to be of real benefit to patients with schizophrenia, they must exceed the small learning effect represented in the placebo condition in these trials, which is also a part of the cognitive response in patients receiving active treatment.

This amount of improvement with placebo in performance-based cognitive assessments is clearly less than that seen in clinical trials across different neuropsychiatric conditions, including symptom rating scales such as the PANSS and the Hamilton Depression Rating Scale.23 Because the amount of cognitive improvement in patients with cognitive impairment associated with schizophrenia receiving placebo is almost identical to the learning effects that are seen prior to baseline, the increases are almost certainly due to practice, comfort, or learning effects and not to the expectation biases that can have a tremendous effect on rating scales.

Our results can be interpreted to provide recommendations regarding the strategy of testing patients once prior to baseline to expose them to the testing process or to minimize a learning effect. A simple analysis comparing patients who were tested prior to baseline with patients who were not suggests that they had similar improvements in placebo conditions and that this strategy does not systematically lead to reduced postbaseline changes. Also, a larger number of assessments after baseline was associated with slightly larger MCCB improvements, although the magnitude of between-assessment improvement may diminish as the number of postbaseline assessments increases.

These results have implications for study design. First, researchers will need to weigh the value of additional assessments during clinical trials against the potential for an additional learning effect in patients receiving placebo and active treatment, although this effect is small. Second, important information may be gained from a prebaseline assessment. Patients whose cognitive performance worsened from screening to baseline had the largest placebo effect, and those with the largest improvements from screening to baseline had no placebo effect at all. These results, consistent with the principle of regression to the mean, suggest that, for schizophrenia cognition studies, the concept of eliminating “placebo responders” who would then go on to have large placebo effects during a trial38,39 is not a viable approach. However, a screening assessment may produce information that can facilitate study design, such as stratification of patients based on potential placebo effects. It may also allow statistical approaches that adjust for screening values (along with the customary baseline values) in the primary efficacy model as a method for reducing the noise associated with large fluctuations in patients who may have had exceptionally good or bad days at the baseline visit. Preliminary analyses using this method suggest that it may increase precision of assessing treatment effects in cognitive treatment trials.40 Other individual difference characteristics found in our analyses to be associated with placebo group improvements, including more severe depression/anxiety and more motivation, could be combined with prebaseline changes to equalize factors that appear to increase the placebo response.

The present results raise a more general question of whether cognitive impairments measured by neuropsychological tests are indeed mutable with pharmacologic intervention in schizophrenia, and whether other measures of brain function are better candidates to serve this purpose. First, neuropsychological tests have demonstrated sensitivity to changes under a variety of different conditions in schizophrenia. They are highly sensitive to the effects of alcohol,41 nicotine,42 anticholinergic medications,43 and stimulants,44 as well as other drugs. Second, because they have consistently demonstrated a clinically significant magnitude of improvement in response to cognitive remediation strategies in schizophrenia,13 the possibility that neuropsychological tests could be a plastic attribute in response to behavioral intervention but a stable trait in response to pharmacologic intervention does not seem reasonable. Third, although it is often assumed that the effects of drugs on other, more biologically proximal measures of brain function, such as brain imaging or electrophysiology, are more powerful than on neuropsychological tests, there is little evidence to support this assumption.45 Finally, several early-phase studies have demonstrated improvements on the MCCB with pharmacologic intervention1,2 that were not replicated in larger programs. It is not at all clear whether the inability of later studies to replicate these findings were due to inefficacy of the pharmacologic intervention, insensitivity of the cognitive tests, or other weaknesses of study design and implementation.2,40 Only one phase 3 program has used the MCCB as a primary end point, and the results of that program were inconclusive.40

Conclusions

In 12 double-blind, placebo-controlled, clinical trials for cognitive impairment associated with schizophrenia, the magnitude of placebo effects on the MCCB was small and not different from the retesting effects during the pretreatment screening period when patients knew that they were not receiving treatment. The magnitude of the placebo effect was not large enough to obscure a treatment effect with a small to medium effect size. There are patient and study characteristics that have a modest influence on postbaseline retest changes, and the number of reassessments is minimally associated with changes in performance on the MCCB.

Back to top
Article Information

Accepted for Publication: May 3, 2017.

Corresponding Author: Richard S. E. Keefe, PhD, Department of Psychiatry and Behavioral Sciences, Duke University Medical Center, Box 3270, Durham, NC 27710 (richard.keefe@duke.edu).

Published Online: June 21, 2017. doi:10.1001/jamapsychiatry.2017.1574

Author Contributions: Drs Keefe and Davis had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Keefe, Davis, Harvey, Atkins, Marder.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Keefe, Davis, Harvey, Marder.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Davis.

Obtained funding: Keefe, Haig.

Administrative, technical, or material support: Davis, Harvey, Atkins, Haig, Hagino, Hilt.

Study supervision: Keefe.

Conflict of Interest Disclosures: Dr Keefe reported receiving investigator-initiated research funding support from the Department of Veterans Affairs, the National Institute of Mental Health, and the Singapore National Medical Research Council; receiving honoraria and serving as a consultant, speaker, or advisory board member for Abbvie, Acadia, Akebia, Akili, Alkermes, Astellas, Asubio, Avanir, AviNeuro/ChemRar, Axovant, Biogen, Boehringer Ingelehim, Cerecor, CoMentis, FORUM, Global Medical Education, GW Pharmaceuticals, Intracellular Therapeutics, Janssen, Lundbeck, MedScape, Merck, Minerva Neurosciences Inc, Mitsubishi, the Moscow Research Institute of Psychiatry, Neuralstem, Neuronix, Novartis, the New York State Office of Mental Health, Otsuka, Pfizer, Reviva, Roche, Sanofi, Sunovion, Takeda, Targacept, the University of Moscow, the University of Texas Southwest Medical Center, and WebMD; receiving royalties from the Brief Assessment of Cognition in Schizophrenia (BACS) testing battery and the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Battery (BACS Symbol Coding); and being a shareholder in NeuroCog Trials and Sengenix. Dr Harvey reported serving as a consultant to AbbVie, Allergan, Boehringer Ingelheim, Forum Pharma, Genentech, Lundbeck Pharma, Otsuka Digital Health, Roche Pharma, Sanofi, Sunovion, and Takeda Pharma; receiving royalties for the BACS; and receiving research grants from Lundbeck and the Stanley Medical Research Foundation.

Additional Contributions: We acknowledge data contributions from Eli Lilly, Abbvie, Sanofi, the National Institute of Mental Health Treatment Units for Research on Neurocognition in Schizophrenia, FORUM Pharmaceuticals, and F. Hoffmann–La Roche Ltd.

References
1.
Keefe  RS, Meltzer  HA, Dgetluck  N,  et al.  Randomized, double-blind, placebo-controlled study of encenicline, an α7 nicotinic acetylcholine receptor agonist, as a treatment for cognitive impairment in schizophrenia.  Neuropsychopharmacology. 2015;40(13):3053-3060.PubMedGoogle ScholarCrossref
2.
Haig  GM, Bain  EE, Robieson  WZ, Baker  JD, Othman  AA.  A randomized trial to assess the efficacy and safety of ABT-126, a selective α7 nicotinic acetylcholine receptor agonist, in the treatment of cognitive impairment in schizophrenia.  Am J Psychiatry. 2016;173(8):827-835.PubMedGoogle ScholarCrossref
3.
Bugarski-Kirola  D, Blaettler  T, Arango  C,  et al.  Bitopertin in negative symptoms of schizophrenia—results from the phase III FlashLyte and DayLyte studies  [published online December 15, 2016].  Biol Psychiatry. 2016. doi:10.1016/j.biopsych.2016.11.014PubMedGoogle Scholar
4.
Umbricht  D, Alberati  D, Martin-Facklam  M,  et al.  Effect of bitopertin, a glycine reuptake inhibitor, on negative symptoms of schizophrenia: a randomized, double-blind, proof-of-concept study.  JAMA Psychiatry. 2014;71(6):637-646.PubMedGoogle ScholarCrossref
5.
Keefe  RS, Buchanan  RW, Marder  SR,  et al.  Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far?  Schizophr Bull. 2013;39(2):417-435. PubMedGoogle ScholarCrossref
6.
Schooler  NR, Buchanan  RW, Laughren  T,  et al.  Defining therapeutic benefit for people with schizophrenia: focus on negative symptoms.  Schizophr Res. 2015;162(1-3):169-174.PubMedGoogle ScholarCrossref
7.
Hogarty  GE, Greenwald  D, Ulrich  RF,  et al.  Three-year trials of personal therapy among schizophrenic patients living with or independent of family, II: effects on adjustment of patients.  Am J Psychiatry. 1997;154(11):1514-1524.PubMedGoogle ScholarCrossref
8.
Eack  SM, Hogarty  GE, Greenwald  DP, Hogarty  SS, Keshavan  MS.  Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: results from a two-year randomized trial.  Res Soc Work Pract. 2011;21(1):32-42.PubMedGoogle ScholarCrossref
9.
Fisher  M, Holland  C, Merzenich  MM, Vinogradov  S.  Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia.  Am J Psychiatry. 2009;166(7):805-811.PubMedGoogle ScholarCrossref
10.
Fisher  M, Mellon  SH, Wolkowitz  O, Vinogradov  S.  Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor.  Schizophr Res Cogn. 2016;3:1-7.PubMedGoogle ScholarCrossref
11.
Mahncke  H, Kim  SJ, Stasio  C,  et al. Results of an FDA device clearance trial for Plasticity-Based Adaptive Cognitive Remediation (PACR). Paper presented at: 5th Schizophrenia International Research Society Conference; April 5, 2016; Florence, Italy.
12.
Kantrowitz  JT, Sharif  Z, Medalia  A,  et al.  A multicenter, rater-blinded, randomized controlled study of auditory processing–focused cognitive remediation combined with open-label lurasidone in patients with schizophrenia and schizoaffective disorder.  J Clin Psychiatry. 2016;77(6):799-806.PubMedGoogle ScholarCrossref
13.
Wykes  T, Huddy  V, Cellard  C, McGurk  SR, Czobor  P.  A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes.  Am J Psychiatry. 2011;168(5):472-485.PubMedGoogle ScholarCrossref
14.
Buchanan  RW, Davis  M, Goff  D,  et al.  A summary of the FDA-NIMH-MATRICS workshop on clinical trial design for neurocognitive drugs for schizophrenia.  Schizophr Bull. 2005;31(1):5-19.PubMedGoogle ScholarCrossref
15.
Buchanan  RW, Keefe  RSE, Umbricht  D, Green  MF, Laughren  T, Marder  SR.  The FDA-NIMH-MATRICS guidelines for clinical trial design of cognitive-enhancing drugs: what do we know 5 years later?  Schizophr Bull. 2011;37(6):1209-1217.PubMedGoogle ScholarCrossref
16.
Green  MF, Nuechterlein  KH, Kern  RS,  et al.  Functional co-primary measures for clinical trials in schizophrenia: results from the MATRICS Psychometric and Standardization Study.  Am J Psychiatry. 2008;165(2):221-228.PubMedGoogle ScholarCrossref
17.
Nuechterlein  KH, Green  MF, Kern  RS,  et al.  The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity.  Am J Psychiatry. 2008;165(2):203-213.PubMedGoogle ScholarCrossref
18.
Kern  RS, Nuechterlein  KH, Green  MF,  et al.  The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization.  Am J Psychiatry. 2008;165(2):214-220.PubMedGoogle ScholarCrossref
19.
Umbricht  D, Keefe  RS, Murray  S,  et al.  A randomized, placebo-controlled study investigating the nicotinic α7 agonist, RG3487, for cognitive deficits in schizophrenia.  Neuropsychopharmacology. 2014;39(7):1568-1577.PubMedGoogle ScholarCrossref
20.
Javitt  DC, Buchanan  RW, Keefe  RSE,  et al.  Effect of the neuroprotective peptide davunetide (AL-108) on cognition and functional capacity in schizophrenia.  Schizophr Res. 2012;136(1-3):25-31.PubMedGoogle ScholarCrossref
21.
Goldberg  TE, Keefe  RS, Goldman  RS, Robinson  DG, Harvey  PD.  Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies.  Neuropsychopharmacology. 2010;35(5):1053-1062.PubMedGoogle ScholarCrossref
22.
Keefe  RS, Goldberg  TE, Harvey  PD, Gold  JM, Poe  MP, Coughenour  L.  The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery.  Schizophr Res. 2004;68(2-3):283-297.PubMedGoogle ScholarCrossref
23.
Papakostas  GI, Østergaard  SD, Iovieno  N.  The nature of placebo response in clinical studies of major depressive disorder.  J Clin Psychiatry. 2015;76(4):456-466.PubMedGoogle ScholarCrossref
24.
Haig  GM, Bain  E, Robieson  W, Othman  AA, Baker  J, Lenz  RA.  A randomized trial of the efficacy and safety of the H3 antagonist ABT-288 in cognitive impairment associated with schizophrenia.  Schizophr Bull. 2014;40(6):1433-1442.PubMedGoogle ScholarCrossref
25.
Clinicaltrials.gov. A phase 2 study to evaluate ABT-126 for the treatment of cognitive deficits in schizophrenia. NCT01678755. https://clinicaltrials.gov/ct2/show/NCT01678755. Accessed April 4, 2015.
26.
Haig  G, Wang  D, Othman  AA, Zhao  J.  The α7 nicotinic agonist ABT-126 in the treatment of cognitive impairment associated with schizophrenia in nonsmokers: results from a randomized controlled phase 2b study.  Neuropsychopharmacology. 2016;41(12):2893-2902.PubMedGoogle ScholarCrossref
27.
Buchanan  RW, Keefe  RS, Lieberman  JA,  et al.  A randomized clinical trial of MK-0777 for the treatment of cognitive impairments in people with schizophrenia.  Biol Psychiatry. 2011;69(5):442-449.PubMedGoogle ScholarCrossref
28.
Clinicaltrials.gov. Efficacy and safety of AVE1625 as a co-treatment with antipsychotic therapy in schizophrenia (CONNECT). NCT00439634. https://clinicaltrials.gov/ct2/show/NCT00439634. Accessed January 10, 2015.
29.
Clinicaltrials.gov. A study of LY2140023 in schizophrenia patients with prominent negative symptoms. NCT01052103. https://clinicaltrials.gov/ct2/show/NCT01052103. Accessed June 21, 2015.
30.
Patterson  TL, Goldman  S, McKibbin  CL, Hughs  T, Jeste  DV.  UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults.  Schizophr Bull. 2001;27(2):235-245. PubMedGoogle ScholarCrossref
31.
Georgiades  A, Davis  VG, Atkins  A,  et al.  Psychometric characteristics of the MATRICS Consensus Cognitive Battery in a large pooled cohort of stable schizophrenia patients  [published online April 19, 2017].  Schizophr Res. doi:10.1016/j.schres.2017.03.040PubMedGoogle Scholar
32.
Green  MF, Harris  JG, Nuechterlein  KH.  The MATRICS consensus cognitive battery: what we know 6 years later.  Am J Psychiatry. 2014;171(11):1151-1154.PubMedGoogle ScholarCrossref
33.
Kay  SR, Fiszbein  A, Opler  LA.  The Positive and Negative Syndrome Scale (PANSS) for schizophrenia.  Schizophr Bull. 1987;13(2):261-276.PubMedGoogle ScholarCrossref
34.
Marder  SR, Davis  JM, Chouinard  G.  The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: combined results of the North American trials.  J Clin Psychiatry. 1997;58(12):538-546.PubMedGoogle ScholarCrossref
35.
Alphs  LD, Summerfelt  A, Lann  H, Muller  RJ.  The negative symptom assessment: a new instrument to assess negative symptoms of schizophrenia.  Psychopharmacol Bull. 1989;25(2):159-163.PubMedGoogle Scholar
36.
Axelrod  BN, Goldman  RS, Alphs  LD.  Validation of the 16-item negative symptom assessment.  J Psychiatr Res. 1993;27(3):253-258.PubMedGoogle ScholarCrossref
37.
Cohen  J, ed.  Statistical Power Analysis for the Behavioral Sciences. Orlando, FL: Academic Press, Inc; 1977.
38.
Ivanova  A, Qaqish  B, Schoenfeld  DA.  Optimality, sample size, and power calculations for the sequential parallel comparison design.  Stat Med. 2011;30(23):2793-2803.PubMedGoogle ScholarCrossref
39.
Doros  G, Pencina  M, Rybin  D, Meisner  A, Fava  M.  A repeated measures model for analysis of continuous outcomes in sequential parallel comparison design studies.  Stat Med. 2013;32(16):2767-2789.PubMedGoogle ScholarCrossref
40.
Brannan  S. The difficulties in going from P2 to P3 in CNS trials: red flags from a recent CIS program. Paper presented at: 13th Annual Scientific Meeting of the International Society of CNS Clinical Trials; February 23, 2017; Washington, DC.
41.
Peterson  JB, Rothfleisch  J, Zelazo  PD, Pihl  RO.  Acute alcohol intoxication and cognitive functioning.  J Stud Alcohol. 1990;51(2):114-122.PubMedGoogle ScholarCrossref
42.
Smith  RC, Warner-Cohen  J, Matute  M,  et al.  Effects of nicotine nasal spray on cognitive function in schizophrenia.  Neuropsychopharmacology. 2006;31(3):637-643.PubMedGoogle ScholarCrossref
43.
Keefe  RS, Bilder  RM, Davis  SM,  et al; CATIE Investigators; Neurocognitive Working Group.  Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE Trial.  Arch Gen Psychiatry. 2007;64(6):633-647.PubMedGoogle ScholarCrossref
44.
Szeszko  PR, Bilder  RM, Dunlop  JA, Walder  DJ, Lieberman  JA.  Longitudinal assessment of methylphenidate effects on oral word production and symptoms in first-episode schizophrenia at acute and stabilized phases.  Biol Psychiatry. 1999;45(6):680-686.PubMedGoogle ScholarCrossref
45.
Kantrowitz  JT, Epstein  ML, Beggel  O,  et al.  Neurophysiological mechanisms of cortical plasticity impairments in schizophrenia and modulation by the NMDA receptor agonist D-serine.  Brain. 2016;139(pt 12):3281-3295.PubMedGoogle ScholarCrossref
×