Effects of Moderate-Dose Treatment With Varenicline on Neurobiological and Cognitive Biomarkers in Smokers and Nonsmokers With Schizophrenia or Schizoaffective Disorder

Context The administration of nicotine transiently improves many neurobiological and cognitive functions in schizophrenia and schizoaffective disorder. It is not yet clear which nicotinic acetylcholine receptor (nAChR) subtype or subtypes are responsible for these seemingly pervasive nicotinic effects in schizophrenia and schizoaffective disorder.

Objective Becauseα4β2 is a key nAChR subtype for nicotinic actions, we investigated the effect of varenicline tartrate, a relatively specificα4β2 partial agonist and antagonist, on key biomarkers that are associated with schizophrenia and are previously shown to be responsive to nicotinic challenge in humans.

Design A double-blind, parallel, randomized, placebo-controlled trial of patients with schizophrenia or schizoaffective disorder to examine the effects of varenicline on biomarkers at 2 weeks (short-term treatment) and 8 weeks (long-term treatment), using a slow titration and moderate dosing strategy for retainingα4β2-specific effects while minimizing adverse effects.

Setting Outpatient clinics.

Participants A total of 69 smoking and nonsmoking patients; 64 patients completed week 2, and 59 patients completed week 8.

Intervention Varenicline.

Main Outcome Measures Prepulse inhibition, sensory gating, antisaccade, spatial working memory, eye tracking, processing speed, and sustained attention.

Results A moderate dose of varenicline (1) significantly reduced the P50 sensory gating deficit in nonsmokers after long-term treatment (P = .006), (2) reduced startle reactivity (P = .02) regardless of baseline smoking status, and (3) improved executive function by reducing the antisaccadic error rate (P = .03) regardless of smoking status. A moderate dose of varenicline had no significant effect on spatial working memory, predictive and maintenance pursuit measures, processing speed, or sustained attention by Conners' Continuous Performance Test. Clinically, there was no evidence of exacerbation of psychiatric symptoms, psychosis, depression, or suicidality using a gradual titration (1-mg daily dose).

Conclusions Moderate-dose treatment with varenicline has a unique treatment profile on core schizophrenia-related biomarkers. Further development is warranted for specific nAChR compounds and dosing and duration strategies to target subgroups of schizophrenic patients with specific biological deficits.

Trial Registration clinicaltrials.gov Identifier: NCT00492349

The smoking or nicotinic challenge in humans transiently influences many biomarkers associated with schizophrenia, including prepulse inhibition (PPI),^1-4 sensory gating,^5,6 antisaccade,^7,8 eye tracking,^9-12 sustained attention,^13-18 information processing speed,^18-22 and spatial information processing,^15,23-25 leading to the current effort to develop novel drugs that target nicotinic acetylcholine receptors (nAChRs) in the central nervous system. There are 17 known nicotinic receptor subunits.²⁶ It is unclear which nAChR subtype or subtypes are responsible for these seemingly pervasive nicotinic effects, and identifying these subtypes would be instrumental in guiding the development of biologically based drugs.

Of the nAChR subtypes,α4β2,α3β4, andα7 are the primary ones in the brain.²⁶ Until recently, clinical efforts in nAChR therapeutics for schizophrenia have been focused more onα7, including neurocognitive and P50 gating improvements obtained in an initial trial of the partialα7 agonist dimethoxybenzylidene anabaseine.²⁷ In the subsequent study,²⁸ P50 was not reported; an improvement of negative symptoms was found. Treatment with tropisetron, a serotonin receptor antagonist with partialα7 agonist effect, did not improve negative symptoms but improved visual sustained attention.²⁹ Treatment with galantamine hydrobromide, a cholinergic compound withα7 andα4β2 allosteric modulation properties, improved processing speed,³⁰ although, in another trial,³¹ treatment with galantamine did not improve cognition. Anotherα7 nAChR, the partial agonist R3487, failed to show cognitive improvement.³² Overall, the findings on whether treatment withα7 compounds improves clinical symptoms or has an effect on biomarkers are not consistent. The inconsistent use of end-point measures also poses a challenge to interpret reproducibility, although the positive effects appeared less reproducible than acute nicotine effects. Alternatively, nicotine's effects on these biomarkers might not primarily originate fromα7 but instead fromα4β2. Data systematically comparing clinicalα4β2 nAChR action across schizophrenia-related biomarkers are not available.

At therapeutic levels, varenicline tartrate is highly selective forα4β2²⁶ and displays robust agonistic and antagonistic properties of nicotine.³³ Varenicline is a partial agonist forα4β2,α3β2, andα6 and a full agonist forα7. However, the equilibrium-binding affinity is hundreds of times more forα4β2 than forα7 or other subtypes,²⁶ and the functional affinity is also 8- to 24-fold higher forα4β2 than forα7 orα3β4.³⁴ We chose a reduced dosing strategy to further separate the effect onα4β2 vs the effect onα7 andα3β4, thus likely yielding a more specificα4β2 effect. We also selected biomarkers previously associated with positive response in humans during nicotinic or smoking challenges as our primary end points: PPI, sensory gating, antisaccade, visual spatial working memory, eye tracking, processing speed, and sustained attention. Additional rationale of biomarker selection is described in the“Methods” section. This design of including“nicotine-responsive biomarkers” in the same trial should facilitate cross-marker comparisons ofα4β2 effects.

Our study tests the hypothesis that the nicotinic effect on biomarker deficits in schizophrenia is due to anα4β2 mechanism and should help us determine whether the development of drugs that target nAChRs in the central nervous system in schizophrenic patients should focus on this subtype. We also planned to examine whether short-term biomarker improvement by varenicline treatment, if present, may predict longer-term improvement in clinical outcomes. Herein, the term biomarker refers to electrophysiological, neurophysiological, and cognitive measures. Varenicline provides the first relatively specificα4β2 compound for human studies; however, it is not simply an agonist or antagonist, so one does not necessarily expect an identical biomarker profile compared with the agonistic effect of nicotine. Theα4 receptor regulates sustained dopamine release in the striatum.³⁵ This dopaminergic modulation of the mesolimbic pathway is considered the key mechanism of varenicline.²⁶ Varenicline as anα4β2 partial agonist and antagonist for smoking cessation is thought to (1) provide sustained dopaminergic tone to limit craving by its agonistic quality and (2) attenuate dopaminergic reward response to nicotine by its antagonistic property,²⁶ thereby breaking the reward-craving cycle leading to addiction.^36-38 Schizophrenia treatment might benefit from sustained dopaminergic tone enhancement (the first mechanism) and/or through modest antagonism of hyperdopaminergic activity (the second mechanism). Dysregulation ofα4β2 is documented in schizophrenia^39-43 not secondary to smoking,⁴¹ andα4β2 is involved in cognitive functions.^44,45 Varenicline offers a potential alternative treatment for the putative nicotinic/dopaminergic dysfunction in schizophrenia.

We recruited smoking and nonsmoking schizophrenic patients to evaluate the effects of varenicline with and without potential smoking-related confounders. We chose a moderate dose (1 mg/d), which is half of the recommended dose for smoking cessation. Compared with a 2-mg/d dose, a 1-mg/d dose resulted in a more than 50% reduction in the primary adverse effect of nausea yet reduced the quit rate by only a fraction.⁴⁶ Therefore, a moderate-dose strategy should (1) reduce potential adverse effects, especially in nonsmoking patients; (2) still allow for testing to determine whether sustainedα4β2 modulation would influence biomarkers; and (3) further capitalize on the differential affinity of varenicline toα4β2 vs other subunits and ensure that significant effects, if found, are likely due toα4β2 rather than toα7 orα3β4 nAChR subunits.

Participants gave informed consent that was approved by the University of Maryland institutional review board. They were 18 to 60 years of age with schizophrenia or schizoaffective disorder and received antipsychotic medication and were clinically stable for 4 weeks or longer. Two patients received first-generation antipsychotics; the remainder received second-generation antipsychotics. Patients undergoing smoking cessation therapy were excluded, as were patients with major medical conditions, atrioventricular block identified on an electrocardiogram, and/or renal insufficiency. We randomly assigned 69 patients (Figure 1). Age, sex, and baseline smoking status were matched between the treatment group and the placebo group (Table 1).

A double-blind, parallel-groups design was used. Patients were randomly assigned to receive varenicline or placebo at a ratio of 1:1, stratified by smoking status and sex. Smoking status was either current smokers (daily smokers of any amount for more than 1 year) or nonsmokers (never smoked or past smokers who had not smoked for more than 1 year). Varenicline and placebo were packaged in identical capsules placed in blister packs and were dispensed in person with assessments weekly for the first 2 weeks and then biweekly. Patients followed a slow titration of 0.5 mg daily for 1 week and then 0.5 mg twice daily for 7 weeks. The uniqueα4β2 profile of varenicline could yield slow but continuous modulation, which was seen in the long-term administration of nicotine in animals.⁴⁷ Therefore, key biomarkers were measured at baseline, week 2, and week 8. After the last dose, patients were monitored for 2 weeks, and our study was terminated at week 10. After completing our study, smokers who wished to continue receiving varenicline for smoking cessation with his or her own physician could request a disclosure of whether they were treated with varenicline or placebo. This unblinding carries a risk of biasing raters and patients, although this possibility was minimized by restricting knowledge of the treatment to 1 coordinator. To recruit a representative sample and avoid potential bias by patients seeking smoking cessation, desire to quit smoking was not a requirement for participation. Smoking cessation counseling was also not implemented, other than encouraging smoking cessation as routine clinical practice, to minimize different levels of clinical attention between smokers and nonsmokers.

The primary measure of psychiatric symptoms was the Brief Psychiatric Rating Scale (BPRS), done at each visit. At baseline and week 8, negative symptoms were assessed with the Schedule for Assessment of Negative Symptoms, depression was assessed with the Hamilton Scale for Depression, and function was assessed with the Level of Functioning and Global Assessment of Functioning scales. Suicidality was assessed at every visit. The number of cigarettes smoked per day (CPD) was the primary measure of change in smoking. The end-expired carbon monoxide level (not timed to the last cigarette) was collected as an approximate validation of the CPD report. To test under a relatively steady varenicline level, participants took the study medication at least 2 hours before each biomarker testing. Smokers were required to refrain from smoking for 1 hour before testing. The Minnesota Nicotine Withdrawal Scale was used immediately after each laboratory test to evaluate potential confounding effects of withdrawal. We used a checklist for the adverse effects of varenicline treated to rate adverse effects from 0 to 3 (none, mild, moderate, or severe).

The PPI measures the suppression of the acoustic startle eyeblink response by a prepulse, whereas startle reactivity measures the amplitude of the startle eyeblink itself. Prepulse inhibition abnormality in schizophrenia can be reduced by smoking and nicotine.^2-4,48 The PPI was recorded using methods previously described.^4,49 A session started with a 3-minute acclimation to 70-dB white noise. Startling pulse-alone trials contained 116-dB white noise lasting 40 milliseconds. The prepulse-pulse trials contained a 20-millisecond, 80-dB white noise prepulse. The test included 18 pulse-alone trials (measuring startle reactivity) and 12 prepulse-pulse trials with a 120-millisecond interstimulus interval for PPI. The percentage of PPI was calculated by use of electromyographic amplitudes as follows: (startle alone − prepulse-pulse condition)/startle alone × 100. Twenty-five percent of patients were classified as nonresponders⁴⁹ and were excluded from analysis (with no difference between treatment groups).

The use of nicotine reduces the double-click evoked P50 sensory gating deficit in schizophrenia.^5,6 As previously described,⁴⁹ data were collected in a sound chamber using a Neuroscan SynAmp 64-channel system (Neuroscan, Charlotte, North Carolina) with a 1-kHz sampling rate of 0.1 to 200.0 Hz.² Subjects listened to 150 pairs of clicks (1-millisecond, 75-dB, and 500-millisecond interclick interval and 10-second intertrial interval). Linked mastoid electrodes served as reference. Electrode impedance was kept below 5 kΩ. The vertex electrode CZ was used for scoring.^50,51 Records were filtered at 3 to 100 Hz (24 octave), threshold-filtered at±75μV, and averaged to obtain the first (S1) and second (S2) stimulus P50 waves. P50 gating was the S2/S1 P50 ratio.

The use of nicotine improves performance in several eye-tracking measures.^9-12 We used a recently developed foveal stabilization–based paradigm to examine the predictive mechanism of eye tracking.⁵² Data were collected using an EyeLink II eye tracker sampling at 500 Hz, using target speeds of 18.7°/s at 24° of visual angle. We stabilized the target onto the fovea and covertly measured the predictive mechanism during eye tracking without the subject's awareness.⁵² We calculated the predictive pursuit gain averaged over the 1-second stabilization period. Pursuit gain is the averaged artifact-free eye velocity divided by target velocity.⁵² Maintenance pursuit gain was calculated as the eye velocity during the regular eye-tracking period (without foveal stabilization) divided by target velocity.

Antisaccade is an eye movement measure of disinhibition, which is frequently abnormal in schizophrenia.⁵³ The administration of nicotine reduces the antisaccadic error rate.^7,8 In our study, subjects fixated on a center crosshair target for 1.5 to 2.5 seconds. A peripheral cue was presented 5° or 10° to the right or left of center. The center crosshair was turned off 200 milliseconds after the appearance of the peripheral cue.⁵⁴ Subjects were instructed to make a saccadic movement equidistant to the position of the cue but in the opposite direction. The antisaccadic error rate measures the inability to inhibit the reflexive response to the target, and this rate is calculated as the number of trials in which the subject looked toward the cue, instead of the opposite direction, divided by the total number of valid trials.

Nicotine use increases spatial information processing^15,55 and affects spatial working memory,²⁵ although some findings appear to be contradictory.^12,15,56 We assessed spatial working memory by memory saccade, which is often impaired in schizophrenia.⁵³ Subjects were required to fixate on a central target while a peripheral cue was briefly (250 milliseconds) flashed. After another 10 seconds, subjects were signaled by the removal of the central target to make a saccadic movement to the horizontal cue location. There were 8 target locations 2.5° apart, ranging from 2.5° to 10° left or right of center. Spatial working memory was measured by the positional error, calculated as the distance between the saccadic position and the target position.

Deficits in sustained attention and in processing speed are known problems in schizophrenia.^57,58 In the majority of subjects included in previous studies,^7,12,13 nicotine use did not affect sustained attention as measured by the Continuous Performance Test (CPT)–identical pair (CPT-IP) d′ calculation. However, nicotine use did improve sustained attention as measured by Connors' CPT.^14-17 The processing speed measured by the third edition of the Wechsler Adult Intelligence Scale's Digit Symbol Substitution Task accounts for a disproportionate amount of the cognitive impairment seen in schizophrenia.^59-62 Nicotine use speeds up the evaluation of stimuli and the processing of information.^18-20,22 Based on these data, attention (using Conners' CPT d′ as the unit of measure) and processing speed (using the Digit Symbol Substitution Task as the unit of measure) were the primary neuropsychological end points. The above measures form the“nicotine-responsive biomarker battery” used in this study. The MATRICS (Measurement and Treatment Research to Improve Cognition in Schizophrenia) Consensus Cognitive Battery (MCCB)^63,64 was considered secondary because several tasks (eg, the CPT-IP) embedded in this battery may not be sensitive to nicotine.^{7,12,16,17,23}

All laboratory measures were processed and scored under blinded conditions.

All models were fitted using the PROC MIXED procedure (SAS Institute Inc, Chicago, Illinois). Treatment effects were analyzed using a mixed model for incomplete repeated measures analysis of covariance: end point = baseline measurement + treatment + time + baseline smoking status + all interaction terms. The terms for smoking status were controlled for potential confounding or moderating effects of baseline nicotine use; significant treatment × smoking interactions were followed by post hoc analyses of treatment effects in smokers vs nonsmokers. The main effect in this model was to estimate the average (across weeks) difference between treatments, whereas treatment × time interactions led to post hoc analysis of how treatment effects changed between visits. Appropriate transformation was applied to skewed measures. If a treatment effect was found in smokers only, with post hoc analyses, we would combine this effect with CPD (as a covariable) to examine the potential effect secondary to behavioral change in smoking. We employed a restricted maximum likelihood method using an unstructured covariance matrix for the correlation among observations. For measures in which the unstructured covariance model did not converge, the generalized estimating equations method was used with a compound symmetry covariance matrix. The Spearman rank correlation was used in biomarker-clinical measure analyses and was limited to biomarkers that showed significant treatment effects.

There was no treatment effect (F_1,41.0 = 0.65, P = .42) or treatment × baseline smoking status interaction for PPI (Figure 2). However, there was a treatment effect for startle reactivity in which treatment with varenicline reduced startle reactivity in schizophrenic patients (F_1,42.9 = 6.44, P = .02) (Figure 2). The effect was significant at week 8 (P = .008) but not at week 2 (P = .11). The treatment × smoking status interaction was not significant. From baseline to week 8, change in PPI and change in startle reactivity were correlated in the placebo group (ρ = 0.55, P = .011) and the varenicline group (ρ = 0.65, P = .001). Changes in startle reactivity and BPRS total were positively correlated in the placebo group (ρ = 0.48, P = .032) but not the varenicline group (ρ = −0.23, P = .26). The difference between the coefficients was significant on the basis of the Fisher z transformation (P = .048), suggesting that dampening of the startle reactivity by varenicline altered the relationship. Reduction in startle reactivity was also correlated with increased MCCB composite scores (r = −0.45, P = .005); the coefficients in the varenicline group and those in the placebo group were not significantly different (P = .10).

There was a treatment × week interaction (F_1,56.3 = 8.20, P = .006) such that long-term treatment with varenicline corrected for some of the P50 gating deficit in schizophrenic patients at week 8 (t = 3.07, P = .003) but not at week 2 (P = .67). The treatment × smoking interaction (P = .009) indicated that the treatment effect was significant for nonsmokers (P = .001) but not for smokers (P = .61), although the direction was consistent in both groups (Figure 2). Although an increase in the S2/S1 ratio from baseline to week 8 was observable in the placebo group (Figure 2G), the change was significant in the varenicline group only (P = .02) but not in the placebo group only (P = .54). An exploration of P50 amplitudes showed that treatment with varenicline reduced the S2 amplitude but not the S1 amplitude, suggesting a gating effect not secondary to the conditioning response (Figure 2). A change in P50 gating was not correlated with a change in the MCCB (P = .96) or the BPRS (P = .10).

Compared with treatment with placebo, treatment with varenicline reduced the antisaccadic error rate (F_1,55.3 = 4.73, P = .034) (Figure 3). There was no smoking × treatment interaction. The change score in antisaccadic error rate was not correlated with change scores for the MCCB (P = .34) or the BPRS (P = .51). Antisaccade was correlated in a replicable way with the MCCB (r = −0.50, P < .001 at baseline vs r = −0.49, P < .001 at week 8), yet the changes of the 2 were not correlated (r = −0.14, P = .34). No treatment or treatment-related interaction was found for memory saccade (Figure 3).

There was no treatment effect on maintenance pursuit gain (F_1,55.6 = 0.04, P = .85) (Table 2) or on predictive pursuit gain (F_1,55.9 = 3.82, P = .06). The trend showed reduced performance with varenicline compared with placebo, although it was not significant (Table 2). There was no treatment × smoking interaction for either measure.

Processing speed as measured by the Digit Symbol Substitution Task showed no significant effect of treatment (F_1,51.5 = 1.69, P = .20) (Table 2) or treatment × smoking status interaction (F_1,51.6 = 3.18, P = .08). Treatment effects on sustained attention by Connors' CPT d′ (Table 2) or hit rate and their treatment × smoking status interactions were all not significant. The treatment difference for the MCCB composite score was not significant (Table 2). Tests for variation in treatment differences among 7 MCCB domains (treatment × domain interaction; F_6,56.1 = 0.22, P = .97), or between smokers and nonsmokers, either across domains (treatment × smoking interaction; F_1,42.4 = 0.37, P = .55) or by domain (treatment × smoking × domain interaction; F_12,81.4 = 0.83, P = .62), were not significant.

Although enrollment was not restricted to those desiring to quit smoking, a reduction in CPD was noted in patients who received varenicline compared with patients who received placebo (F_2,64 = 3.33, P = .04) (Figure 4). Carbon monoxide levels were also reduced in the varenicline group compared with the placebo group, although this was not significant (P = .21) (Table 2). Changes in the carbon monoxide level and CPD were correlated (r = 0.53, P = .002). Two patients who received varenicline and 1 patient who received placebo quit smoking by week 8. A change in CPD was not correlated with changes in those biomarker or clinical end points that showed treatment effects in the varenicline or placebo group (all P ≥ .26). Dividing the varenicline group into patients with a reduction in CPD (n = 11) and patients without a reduction in CPD (n = 8), we also did not find a difference in end-point measure changes (all P ≥ .14), ruling out large confounding effects on biomarkers due to change in smoking quantity. Smokers' carbon monoxide levels were not significantly correlated with any dependent measures at baseline (all r ≤ 0.29, all P ≥ .08).

For the BPRS total, there were no significant treatment or interaction effects, with a trend toward reduced psychiatric symptoms in the varenicline group compared with the placebo group (F_1,54.2 = 3.32, P = .07) (Figure 4). Cases of exacerbation of psychosis by treatment with varenicline have been reported⁶⁵; however, the BPRS psychosis subscale showed a trend toward reduced psychosis in the varenicline group compared with the placebo group (F_1,58 = 3.89, P = .053). There were no differences in treatment effects in smokers vs nonsmokers (all P ≥ .30). We found no significant effect of treatment on negative symptoms assessed using the Schedule for Assessment of Negative Symptoms, on functions assessed using the Level of Functioning and Global Assessment of Functioning scales, or on depression assessed using the Hamilton Scale for Depression (Table 2). Assessments of depression, anxiety, and suicidality were further probed using other rating sources given the prominent safety concerns associated with varenicline in these areas. Item 3 of the Hamilton Scale for Depression, suicidality, showed no treatment effect (P = .73), and only 1 patient (in the placebo group) had a score of more than 0 at week 8. There was also no treatment effect on item 13 of the BPRS (depression) (P = .19; with a numerically higher depression rating in the placebo group than in the varenicline group from baseline to week 8). There was no treatment effect on the BPRS anxiety rating (P = .37, both groups showed reduced levels of anxiety). Therefore, there was no evidence that treatment with slowly titrated varenicline at 1 mg/d increased these psychiatric symptoms.

Other adverse effects at weeks 2 and 8 were compared with those at baseline to determine ratings for symptoms that were newly present or more severe than at baseline (Table 3). The number of abnormal dreams (P = .03) were reduced in the varenicline group compared with the placebo group. Comparing the varenicline group with the placebo group, we found that increased vomiting (15.6% vs 3.1% of patients; P = .20), dry mouth (34.4% vs 18.8% of patients; P = .26), and appetite (31.3% vs 18.8% of patients; P = .39) were not significantly associated with treatment with varenicline.

We found that 8 weeks of moderate-dose treatment with varenicline (1) reduced the sensory gating deficit in schizophrenic patients, (2) reduced startle reactivity but did not change PPI, and (3) improved executive function as measured by the antisaccadic error rate. There were no significant effects on spatial working memory, predictive and maintenance pursuit, processing speed, sustained attention, or the MCCB. There was no evidence of exacerbation of psychiatric symptoms in schizophrenic patients in this gradual titration, moderate-dose strategy; instead, a trend toward decreasing symptoms of psychosis was observed. The use of moderate-dose varenicline was designed to retain varenicline's pharmacologicalα4β2 actions while simultaneously minimizing the effects on other nAChR subtypes (based on preclinical data) and potential adverse effects (based on phase 2 clinical data) on schizophrenic patients.

Human data for nicotinic effects on PPI and sensory gating have been largely based on brief challenge studies. In our trial, the effects of treatment with varenicline on sensory gating and startle reactivity were significant only at week 8. Nicotine is a full agonist, whereas varenicline is a 30% to 60% partial agonist (of the nicotinic effect on dopamine turnover) and also a partial antagonist ofα4β2.²⁶ This profile could modulateα4β2 and the downstream pathways through a more gradual time course different from a full agonist. One study⁶⁶ reported no effect of single-dose varenicline on P50 gating in 6 patients; another 2-week study⁶⁷ also reported no effect on P50 gating in smokers. Short-term treatment designs may be appropriate for a full agonistic mechanism but could miss important effects exerted by the unique partial agonistic-antagonistic modulation, as shown herein.

Varenicline did not mimic the acute nicotinic effects on PPI.⁴ Rollema et al⁶⁸ reported a weak enhancement of PPI and startle reactivity in rodents receiving 1 dose but not in other doses of varenicline, and additional tests failed to show the effect. In humans, nicotine use enhanced PPI,^69-72 but opposite effects have also been observed.⁷³ For startle reactivity, nicotine use either does not change it⁷³ or increases it.^74,75 Startle reactivity is enhanced by the activation of dopamine receptors,^76,77 whereas the use of dopamine antagonists and antipsychotic medications dampen it.^68,78-81 Because long-term treatment with varenicline mimics the aspect of antipsychotics that reduces startle reactivity, it may indicate a gradual downregulation of dopaminergic function by theα4β2 antagonistic aspect of varenicline. The lack of significant findings on spatial working memory may also support an antagonistic mechanism in varenicline because a previous study¹⁵ had shown that antagonism to high-affinity nAChRs blocks the improvement of spatial working memory by nicotine. However, this may not explain the lack of effect on PPI because antipsychotics reverse the PPI deficits induced by dopamine agonists.¹ A 2-week treatment with varenicline increases striatal D2/3 receptor availability by 11% to 15%.⁸² Our findings encourage additional long-term exposure studies to determine the time course of varenicline and its effect on dopaminergic modulation.

Varenicline also did not mimic the nicotinic effects of improving maintenance pursuit and sustained attention⁷ but showed a similar effect of reducing antisaccadic error.^7,8,83 These findings imply that error reduction may be more specifically influenced byα4β2, whereas improvement in other measures may be associated with other nAChRs. Antisaccade assesses the executive ability to inhibit distraction and attend to the instructed target. Antisaccadic error in schizophrenia is thought to reflect an impaired frontostriatal pathway⁸⁴ and its striatal dysfunction ofγ-aminobutyric acid (GABA).⁸⁵ Alkondon et al⁸⁶ speculated that GABAergic postsynaptic currents can be activated by theα4β2 nAChR present in presynaptic terminals of interneurons, a possible route by whichα4β2 nAChR treatment could affect GABAergic inhibitory and, thus, antisaccade function.

Some aspects of the pathology of schizophrenia appear to be related to nicotinic receptor abnormalities irrespective of smoking.⁴¹ Therefore, we did not expect that varenicline would affect smokers but not nonsmokers (or vice versa), as was found in startle reactivity and antisaccade. The significant reduction in the P50 gating deficit that was found in nonsmokers but not smokers suggests that this reduction was not due to smoking per se. Changes in P50 gating and in CPD (r = −0.20. P = .41) or carbon monoxide level (r = −0.13, P = .63) in smokers who received varenicline were not correlated. The better P50 gating in the smokers who received placebo (Figure 2E) was a chance bias from randomization that could reduce the power in the smoker group.

Treatment with varenicline improved sustained attention and working memory after less than 3 days of mandatory abstinence in nonpsychiatric subjects,⁸⁷ possibly by reversing dysfunctions associated with abstinence-induced withdrawal. Under the current nonabstinence condition, 1 mg of varenicline did not improve sustained attention, spatial working memory, or predictive pursuit (a task related to oculomotor working memory⁵²), although a higher dose could be tried. Nicotine use may improve working memory in the spatial domain,²⁵ although, in several studies,^{7,12,16,17,23} it did not improve working memory in patients with schizophrenia and may even worsen it.⁸⁸ The nicotinic effect on maintenance pursuit^7,12 was also not replicated by varenicline. Thus, unlike several positive findings reported by an open-labeled study,⁸⁹ our study suggests that theα4β2 partial agonistic-antagonistic effect on cognition is modest. In fact, varenicline at the current dosage does not replicate that many acute full agonistic effects of nicotine. Instead, it reduces selected biomarker deficits, particularly P50 gating and antisaccadic deficits. Varenicline's long-term, but not short-term, effect on specific biomarkers is a novel finding and differs from acute nicotine. It would be intriguing to see whether biomarkers that are responsive to nicotine but not varenicline would be responsive to compounds targeting non-α4β2 nAChR subtypes. This also illustrates the advantage of comparing key biomarkers in the same trial to identify plausible specific receptor–clinical biomarker relationships.

Safety concerns, especially regarding the exacerbation of symptoms in mentally ill populations, has been raised in case reports and in black box warnings from the US Food and Drug Administration. Data from randomized controlled trials of nonpsychiatric populations showed no major psychiatric symptom exacerbations⁹⁰ or even improvement in mood.^87,91 We observed no excessive somatic or psychiatric symptoms using the current dosing, which is consistent with the prediction based on phase 2 data in which a 1-mg dose reduces the number of adverse effects but maintains a reasonable efficacy.⁴⁶

Our study examined 21 biomarker, clinical, and smoking-related measures instead of a single primary end point, which raises the possibility of false positives. Our goals were to compare biomarkers that previously responded to nicotine and test them simultaneously for a chance of head-to-head comparisons regarding theα4β2 effect. None of the findings would hold up after correcting for the multiple comparisons, although the previous nicotinic effect on individual biomarkers was typically tested 1 biomarker at a time. Nevertheless, replication studies are needed. The small number of nonsmokers who received placebo at week 8 could have also reduced the power and led to false negatives. All patients were receiving antipsychotic drugs. Because the biomarker end-point values were compared with their baseline values, our findings are less likely due to antipsychotic treatment, although we cannot rule out potential varenicline × antipsychotics interactions.

We opted for a biomarker-based trial, assuming that the biomarkers should be associated with more specific biological pathways and more informative for translational follow-up studies. Our study reveals that moderate-dose treatment with varenicline has a long-term effect on specific biomarkers. A longer-term treatment and/or a dose-ranging design could reveal further improvements, because core neurophysiological impairments in schizophrenia are chronic and entrenched. Because biomarker improvements were seen at week 8, we could not examine whether improvement at week 2 would predict clinical improvement at week 8. However, in a still longer trial, one could examine whether improvement seen at week 8 would predict clinical outcome later. The findings suggest that previously described nicotinic effects on P50 gating and antisaccade are likely, in part, related to specificα4β2 nAChR modulation. Agents with specificα4β2 actions could potentially be used to target these specific biomarker deficits. Most individual patients do not have all of the cognitive or biomarker deficits that are statistically associated with schizophrenia. A given biomarker deficit is typically present in 30% to 50% of patients, depending on cutoff criteria, and many biomarkers are not correlated.⁴⁹ We might expect a novel agent targeting a specific receptor subtype to correct for specific biomarker(s) rather than to correct for all of the heterogeneous symptoms and neurobiological deficits covered under the diagnosis of schizophrenia.

In summary, we observed no evidence that a moderate-dose, 8-week treatment regimen of varenicline is unsafe for stable, medicated schizophrenic patients. There is evidence of long-term neurobiological improvement in sensory gating and antisaccadic functions and of a nonsignificant reduction in psychotic symptoms, suggesting a unique efficacy profile of the presumed partial agonistic-antagonisticα4β2 nAChR modulation. These findings should encourage the further development ofα4β2 nAChR–modulating compounds that optimize dosing and treatment duration and that are safe and effective for treating specific neurobiological deficits, a critical unmet need in the treatment of schizophrenia.

Correspondence: L. Elliot Hong, MD, Maryland Psychiatric Research Center, PO Box 21247, Baltimore, MD 21228 (ehong@mprc.umaryland.edu).

Submitted for Publication: April 5, 2011; final revision received May 25, 2011; accepted May 26, 2011.

Published Online: August 1, 2011. doi:10.1001 /archgenpsychiatry.2011.83

Funding/Support: This work was supported by the Stanley Medical Research Institute (grant 06TAF-966), the National Institutes of Health (grants DA027680, MH085646, and MH077852), and the Neurophysiology Core of the University of Maryland General Clinical Research Center (grant M01-RR16500).