Utility of Combinations of Biomarkers, Cognitive Markers, and Risk Factors to Predict Conversion From Mild Cognitive Impairment to Alzheimer Disease in Patients in the Alzheimer's Disease Neuroimaging Initiative

Context Biomarkers have become increasingly important in understanding neurodegenerative processes associated with Alzheimer disease. Markers include regional brain volumes, cerebrospinal fluid measures of pathological Aβ1-42 and total tau, cognitive measures, and individual risk factors.

Objective To determine the discriminative utility of different classes of biomarkers and cognitive markers by examining their ability to predict a change in diagnostic status from mild cognitive impairment to Alzheimer disease.

Design Longitudinal study.

Participants We analyzed the Alzheimer's Disease Neuroimaging Initiative database to study patients with mild cognitive impairment who converted to Alzheimer disease (n = 116) and those who did not convert (n = 204) within a 2-year period. We determined the predictive utility of 25 variables from all classes of markers, biomarkers, and risk factors in a series of logistic regression models and effect size analyses.

Setting The Alzheimer's Disease Neuroimaging Initiative public database.

Outcome Measures Primary outcome measures were odds ratios, pseudo- R²s, and effect sizes.

Results In comprehensive stepwise logistic regression models that thus included variables from all classes of markers, the following baseline variables predicted conversion within a 2-year period: 2 measures of delayed verbal memory and middle temporal lobe cortical thickness. In an effect size analysis that examined rates of decline, change scores for biomarkers were modest for 2 years, but a change in an everyday functional activities measure (Functional Assessment Questionnaire) was considerably larger. Decline in scores on the Functional Assessment Questionnaire and Trail Making Test, part B, accounted for approximately 50% of the predictive variance in conversion from mild cognitive impairment to Alzheimer disease.

Conclusions Cognitive markers at baseline were more robust predictors of conversion than most biomarkers. Longitudinal analyses suggested that conversion appeared to be driven less by changes in the neurobiologic trajectory of the disease than by a sharp decline in functional ability and, to a lesser extent, by declines in executive function.

Biomarkers have become increasingly important in understanding the neurodegenerative processes associated with Alzheimer disease (AD), their staging, and response to treatment. Such markers are directly or indirectly relevant to the histopathology of AD and disease stage. They include cerebrospinal fluid (CSF) measures of pathological Aβ1-42 and tau and regional brain volumes. Cognition, a behavioral marker that may be considered a surrogate for neural systems function, may also be related to regional pathologic characteristics and disease stage. Other individual risk factors, including susceptibility genes and some demographic variables (eg, age or educational level), may also influence disease vulnerability and progression. Especially pertinent for our study is the value of biomarkers and behavioral markers in predicting conversion from mild cognitive impairment (MCI) to AD. Such predictions have obvious utility for clinical decision making, understanding the nature of neurobiologic changes at a critical phase in the illness, and assessment of response to treatment.

Several comprehensive reviews of the area have emphasized the predictive power of biomarkers in identifying an individual's likelihood of converting to AD, as well as differences in biomarker values among cognitively healthy control subjects and MCI and AD individuals.^1-5 Sensitivities and specificities have ranged as high as 0.95 and 0.83, respectively, although more recent large-scale studies that have involved multiple sites have found lower values, perhaps for technical reasons or case ascertainment differences.^6,7 In an Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort, total tau and Aβ1-42 were significant discriminators in logistic regression between controls and those with mild AD.⁸

Hippocampal atrophy is an important brain volume biomarker and is also a predictor of conversion from MCI to AD.^9-11 In a large ADNI cohort, left hippocampal volume at baseline was the best single predictor among brain volume measures of conversion from MCI to AD in a 1-year period.¹² Other regions of cortex (eg, posterior cingulate or temporal lobe) may also be predictive in this cohort.¹³

Studies have examined the role of cognitive markers in predicting conversion of MCI to AD; tests of episodic memory (including memory after a delay) were consistently robust predictors.^14-19 Predictive accuracy for one such set of cognitive variables (composed of episodic memory and processing speed measures) has been found to be as high as 0.86 (sensitivity, 0.76; specificity, 0.90).²⁰ Of genetic risk factors for late-onset AD, apolipoprotein E (APOE) is the strongest (with an odds ratio of approximately 3.8) and best replicated.^22-24 Nevertheless, theε4 variant is not thought to be a strong predictor of conversion from MCI to AD.²⁵ Of nongenetic risk factors, educational level may influence age of onset or progression of AD.²⁶

Relatively few studies have combined these different classes of biomarkers, cognitive markers, and individual risk factors. One such study²⁷ found that cognitive variables including verbal list learning, story recall, and composite cognition resulted in a predictive accuracy of 0.79. Magnetic resonance imaging (MRI) measures of hippocampal volume and ventricular volume, APOE genotype, and demographic variables did not appreciably improve on this. Other studies have found that cognitive markers remained significant predictors even when MRI structural measures were added to predictive accuracy in regression models.^10,28

Thus, despite formidable evidence for the predictive validity of individual biomarkers and behavioral markers, they have rarely been examined in combined models. To our knowledge, our study is the first that has examined CSF biomarkers, brain volumes, and cognitive markers in combination to predict MCI to AD conversion. We chose to examine this issue using ADNI, a public data set very well suited for this task because of its large samples, breadth of cognitive markers and biomarkers, and prospective nature. Specifically, we included biomarkers directly related to the disease; behavioral markers (including cognitive measures), also directly subject to the impact of disease; individual genetic risk factors (ie, sex and APOE); and demographic risk factors (ie, age and educational level). We hypothesized that episodic memory variables would be strong predictors of conversion. The previously mentioned analyses used baseline variables. We also sought to determine whether magnitude of decline (from baseline to 1 or 2 years) was disproportionate in either cognitive markers, biomarkers, or everyday function. We believed that this analysis would have bearing on how to interpret the MCI-AD transition in this sample. Overall, we believe that our approach is relatively unbiased, comprehensive, and of possible clinical and economic utility.

Participants in this longitudinal study are part of a multisite observational project, ADNI (http://adni.loni.ucla.edu), created to seek an adequate explanation of the progression of MCI and early AD by using serial MRI, positron emission tomography, and CSF-derived biomarkers, as well as clinical and neurocognitive measures. A more detailed description of ADNI is in eMethods. Our sample comprised ADNI patients included in the public database as of August 3, 2009. Patients in the ADNI protocol who completed visits at 12 and 24 months and who had at least 1 follow-up examination were included in the present study.

Criteria for MCI were the same as defined by Petersen²⁹: subjective or informed-by-partner memory complaints confirmed by impaired memory function (scoring below the education-adjusted cutoff on the Logical Memory II subscale of the Wechsler Memory Scale), a Folstein Mini-Mental State Examination score greater than 23, Clinical Dementia Rating equal to 0.5, absence of significant levels of impairment in other cognitive domains, and essentially preserved activities of daily living. Patients with MCI were subdivided into 2 groups: (1) MCI patients who had not converted to AD in the 2 years of follow-up (n = 204) (MCI diagnosed at baseline and at least 2 follow-up visits being MCI), and (2) MCI patients who had converted to AD during the 2 years of follow-up (n = 116). The cognitively healthy control group had no memory complaints aside from those common in other individuals of that age range with no abnormalities, a Mini-Mental State Examination score greater than 23, Clinical Dementia Rating equal to 0, and absence of significant levels of impairment in cognitive functions or activities of daily living. Only controls who were cognitively healthy at baseline and who remained healthy in at least 1 follow-up were included in the study (n = 197). Controls were not included in any of the conversion analyses. Patients with AD were used in the present study for descriptive purposes only. They had Mini-Mental State Examination scores from 20 to 26 and a Clinical Dementia Rating score of 0.5 or 1, and met criteria for probable AD as established by the National Institute of Neurological and Communicative Disorders and Stroke– Alzheimer's Disease and Related Disorders Association. In addition, they had memory complaint by subject or study partner that is verified by a study partner, and abnormal memory function documented by scoring below the education-adjusted cutoff on the Logical Memory II subscale (delayed recall) from the Wechsler Memory Scale–Revised.

Global indices of cognitive performance and global functional status were obtained using both the Mini-Mental State Examination and the Clinical Dementia Rating. The Functional Assessment Questionnaire (FAQ) measured functional activities of older adults using the patient's study partner as an informant. Certified professionals performed clinical and neuropsychologic assessments.

The neuropsychologic battery included measures of learning and memory, attention, visuospatial abilities, psychomotor speed, and executive function. Verbal episodic memory was assessed by the Alzheimer Disease Assessment Scale–Cognitive using a composite score for the memory tests (word recall test, delayed word recall, and word recognition). Other measures of verbal memory included were the Rey Auditory Verbal Learning Test (trial 5 and delayed recall, recognition at delay) and immediate and delayed memory from the Logical Memory Test of the Wechsler Memory Scale. Alzheimer Disease Assessment Scale–Cognitive“nonmemory” domains (language comprehension, constructional, ideational praxis, orientation, and attention) were included in a composite score. Attention and working memory were assessed by the digit span subtest (forward plus reverse) from the Wechsler Memory Scale–Revised. Semantic processing and speed were assessed using a category fluency test. Semantic knowledge and visuospatial processing were assessed by the Clock Drawing Test. Trail Making Test, parts A and B (hereafter Trails A and B), and the Wechsler Adult Intelligence Scale–Revised Digit Symbol Test were used to measure executive functions and speed of processing.

Samples of CSF were collected at baseline and at the 12-month visit through lumbar puncture. Samples were transferred into polypropylene transfer tubes followed by freezing on dry ice within 1 hour after collection; they were shipped overnight to the ADNI Biomarker Core laboratory at the University of Pennsylvania Medical Center on dry ice. The complete details can be found at http://www.adni-info.org. A standardized protocol was implemented to quantify biomarker concentrations (total tau and Aβ1-42) in each of the CSF ADNI baseline aliquots using a multiplex xMAP Luminex platform (Luminex Corp, Austin, Texas) with Innogenetics (INNO-BIA AlzBio3, Ghent, Belgium; reagents for research use only) immunoassay kit–based reagents. Because of nonnormal distributions, data were log transformed. The ratio of the log-transformed Aβ1-42 /total tau was also examined.

A blood sample for APOE genotyping was extracted at the screening. Genotyping was performed at the ADNI biomarker core laboratory.

All sites met all the requirements for the Alzheimer Disease Cooperative Study start-up and have completed the MRI certification for 1.5 T MRI. All the scans were reviewed for quality control by personnel in the ADNI MRI quality center at Mayo Clinic. For the present study, volumetric and cortical thickness data were downloaded from the ADNI database using the measures extracted by the Multimodal Imaging Laboratory of the University of California, San Diego (“Dale” methods), including entorhinal and lateral temporal cortical thicknesses and hippocampal, ventricular, and whole-brain volumes. Briefly, images were automatically corrected for spatial distortion due to gradient nonlinearity and B1 field inhomogeneity, registered, and averaged to improve signal-to-noise ratio. Volumetric segmentation methods based on FreeSurfer software, optimized for use on large, multisite data sets, were used (for details, see Walhovd et al³⁰).

Comparisons between the groups on demographic and biomarker-related data were analyzed by means ofχ² tests or analyses of variance followed by t tests for pairwise comparisons. The initial goal was to identify the baseline biomarkers and cognitive markers that predicted conversion anytime in the 2-year interval of the study using binary logistic regression models (“anytime” analyses). Our general analytic procedure was conducted in SAS (using PROC Logistic)³¹ and was sequenced in the following way: First, independent regression analyses were separately conducted using clusters of independent variables: (1) demographic and constitutional-related risk factors (APOEε 4 carrier or noncarrier), age, educational level, and sex; (2) cognitive markers (listed previously), (3) CSF biomarkers (listed previously), and (4) brain volumetric biomarkers (listed previously). Then, (as step 5) only the significant predictors (ie, the“winners”) obtained in the clustered regression models were entered in the regression model to compare the magnitude of predictive power between clustered and composite approaches. Because this was our critical analysis, we subjected the model to a k-fold cross-validation procedure adapting SAS macros developed by Gonen³¹ to determine the robustness of our findings. We predicted that cognitive markers, including those related to episodic memory, would be significant predictors of conversion.

Relative predictive power of the logistic regression models was obtained through coefficient of determination of Nagelkerke expressed as a pseudo- R². Predictive accuracy of the resulting models was calculated using receiver operating characteristic curve analysis. Entry was set at P < .05. In all regressions, lower cognitive performance, smaller brain volumes/thicknesses, and more abnormal CSF values were associated with greater risk of conversion (ie, were in a plausible neurobiologic relationship to conversion). Sample sizes for the regression models varied because of missing data and the number of variables initially included.

Significant predictors of time to conversion to AD from baseline status were obtained using time-dependent Cox regression models. The regression procedure followed the same set of steps as the logistic procedure. Patients with MCI who did not convert were censored. For MCI patients for whom a conversion to AD was reported, date of conversion was calculated as the midpoint between their last visit without AD and their first visit with AD. Four points of conversion were possible: 6, 12, 18, or 24 months. Results are reported in eResults and the eTable.

Finally, we sought to determine whether magnitude of decline (baseline to 1 or 2 years) was proportionate among CSF and brain-volume biomarkers, cognitive markers, and everyday function. We reasoned that a disproportionate decline in a biomarker might indicate a shift in the underlying neurobiologic characteristics of the disease that was associated with diagnosis of AD. If, on the other hand, a disproportionate decline was found in a measure of functional activities, this would suggest that diagnosis had to do with a“caseness call” independent of changes in neurobiologic trajectories. To implement this, we first computed the effect size of change scores between baseline and 12 or 24 months in CSF, cognitive, and brain morphometric measures, and functional activities. We then examined the predictive power of declines in these variables in identifying individuals who converted from MCI to AD in a logistic regression.

Table 1 represents the demographic and clinical features at baseline. No differences between groups were found in age, sex distribution, and years of education. The APOE genotype frequency differed among the groups. All staging variables (Clinical Dementia Rating and Mini-Mental State Examination) differed among the groups. Functional status (FAQ) differed among the groups (including MCI converters and nonconverters).

All groups significantly differed from each other on each cognitive measure with a single exception (Table 2). Compared with cognitively healthy controls, MCI patients demonstrated z score impairments in the −1.5 to −2.0 range in memory measures; for nonmemory measures, z score impairments were generally in the−0.6 to−1.0 range. These results suggested that a significant minority of MCI patients had impairments in multiple cognitive domains.

All groups significantly differed from each other on each of the CSF biomarker measures (Table 3). Aβ1-42 was decreased in the MCI and AD groups; tau was increased.

All groups significantly differed from each other on each of the regional morphometric biomarker measures (Table 4). Whole-brain volume and ventricular volume did not differ between the cognitively healthy controls and nonconverter MCI groups.

In the demographic and individual risk cluster, APOE was a significant predictor. Of the cognitive markers, Auditory Verbal Learning Test list recall at 30 minutes, logical memory delayed recall, Alzheimer Disease Assessment Scale memory, Clock Drawing Test, and Trails A were significant predictors of conversion during the 2-year duration of the study (Table 5). Of the CSF biomarkers, only Aβ1-42/total tau was a significant predictor. Among the brain volumetric biomarkers, left hippocampus and left middle temporal lobe cortical thickness were significant predictors.

When only the significant predictors of the clustered models were entered to predict conversion during the 2-year period, 3 variables entered as listed in Table 5: Logical Memory delayed, left middle temporal lobe cortical thickness, and Auditory Verbal Learning Test delayed recall. Pseudo- R² was 0.34, with Logical Memory accounting for 0.18; middle temporal, 0.10; and Auditory Verbal Learning Test, 0.06. The receiver operating characteristic curve for this set of variables is shown in Figure 1. The area under the curve was 0.80, and the percentage of cases classified correctly was 71.9 at c = 0.50. Sensitivity was 0.56 and specificity was 0.82 at c = 0.50. The positive predictive value for the set of markers was 0.65; the negative predictive value was 0.75.

Because this was our key analysis, we conducted a k-fold cross validation to ascertain the robustness of the findings. In this analysis, the sample is broken into randomly chosen subgroups of more or less equal size. With k-fold (here 5), the analysis was repeated k times. The predictor variables used were those found to be significant in the previously mentioned winners model using the total sample of MCI patients. For each analysis in k-fold, one of the subsets is left out as the test set and the others are used as the training set. Parameter estimates are derived from the training set and applied to the test set. Predictive accuracies are then computed and averaged across the 5 separate test sets. Because we set k = 5, randomly generated subsets had samples ranging from 58 to 73. We used 5 because having 60 to 70 patients per test set is considered necessary for power but allowed for a reasonable number of validations.

The 3 independent variables from the previously mentioned analysis served as predictors (Auditory Verbal Learning Test list memory after a delay, story memory after a delay, and left middle temporal thickness), and conversion was the binary dependent measure. The area under the curve for the k-fold analysis was 0.78. Thus, cross-validation shrinkage was quite small and the model appeared to be robust.

We first examined difference scores for the risk factor FAQ, cognitive markers, and biomarkers between baseline and 12 months in MCI converters. When difference scores were converted to effect sizes to compare and contrast them in a common psychometric space, the FAQ difference score expressed in effect size units (Cohen d) was larger than any other variable. The effect size for select cognitive variables, all brain volumetric measures, and all CSF measures are illustrated in Figure 2.

Critically for our purposes, we also sought to determine whether the magnitude of the difference scores between baseline and 12 months for FAQ, various cognitive measures (verbal list learning delayed, Trails B, fluency), and CSF and brain volume biomarkers could predict conversion from MCI to AD in a single logistic regression model (24-month data for biomarkers were not available). Only 2 variables entered the model: FAQ difference score (P < .001) and Trails B difference score (P = .01). Remarkably, they accounted for nearly 50% of the predictive variance (pseudo- R² = 0.47). The resulting receiver operating characteristic curve yielded an area under the curve of 0.85, a specificity of 0.93, and a sensitivity of 0.52 at c = 0.5.

In a systematic series of stepwise logistic regression analyses, we demonstrated that baseline brain morphometric and cognitive variables were the strongest predictors of conversion from MCI to AD. In the predictive model, verbal episodic memory measures and left middle temporal lobe cortical thickness entered significantly. Interestingly, the memory measures that were predictive generally involved a delay, suggesting the importance of consolidation factors. The APOE status and CSF levels of tau or Aβ1-42 did not add predictive value to this composite model.

One might argue that the use of cognitive markers in this context is a tautology because they are used in the diagnosis of AD itself. However, this may not be correct because the diagnosis of MCI already requires episodic memory impairment at a level consistent with AD itself. Irrespective of this line of argument, use of cognitive markers has certain advantages: clear and significant effect on odds ratios, objectivity in scoring, comparative economy in terms of expense and time, and reliability. From a pragmatic standpoint, using all available data to make an accurate prediction would appear optimal in clinical practice or to enrich samples in a clinical trial.

There are several possible reasons that CSF biomarkers were not more powerful predictors when used in combination with other biomarkers or cognitive markers. We do not think that colinearity was an issue, because when we tested for this in our models, all condition values (a measure of colinearity) were well below 30 (data not shown). On the other hand, these CSF measures may be more or less effective as a function of disease stage. Nevertheless, we note that it is not the case that these biomarkers were ineffective in ADNI samples (see our results when examining only brain volumes or CSF biomarkers in the clustered regressions, as well as prior work in smaller ADNI samples that examined CSF biomarkers, global measure of AD-related atrophy, and/or positron emission tomography with radiolabeled [18F]-2-fluoro-deoxy-D-glucose^32,33), only that other types of biomarkers or cognitive markers were more robust predictors in our study. In addition, we note that different biomarkers and behavioral markers may have differentially predictive values at different times in disease progression. That is, biomarkers are dynamic and they will vary in their informativeness depending on when they are measuring, an important and heuristic point recently made by Jack and colleagues.³⁴ They may be most informative in very early prodromal stages, a perspective that has been incorporated into proposed diagnostic criteria for preclinical AD.

Several recent studies are consistent with our results. In an ADNI sample, Fjell et al³⁵ found that baseline morphometric volume measures were better predictors of Clinical Dementia Rating sum of boxes decline than CSF markers. The findings by Ewers et al,³⁶ in a smaller ADNI cohort than our own and using different methods, were nevertheless broadly consistent with our own in that memory measures and Trails B were significant predictors of MCI conversion to AD. In a large meta-analysis that did not include ADNI data, baseline cognitive markers were better predictors of conversion than brain volumetric or CSF biomarkers.⁵

In addition to our study of baseline measures, we also examined decline in cognitive markers, biomarkers, and a measure of function. We found that the decline in functional competence was disproportionate to the decline in biomarkers and cognitive markers, based on 1- and 2-year change scores in both effect size analyses and logistic regression examining predictors of conversion. This suggested to us that diagnosis was not the result of a shift in the underlying neurobiologic characteristics of the disorder but rather was due to some combination of psychometrics of the FAQ, informant factors, and the tendency of a site neurologist to make a caseness call. We have argued elsewhere²¹ that it would be unlikely that if biomarkers, histopathology, and cognition were on a continuum from MCI to AD, functional impairment would be categorical (preserved or impaired). Reinforcing this view are the high rates of conversion from MCI to AD in this sample (approximately 40% within 2 years) and the fact that MCI patients had cognitive impairments in other domains. In the ADNI MCI group, Alzheimer Disease Assessment Scale–Cognitive nonmemory scores (a composite of language and praxic and gnosic abilities) were approximately−2.0. Trails B, a measure of executive function, had a z score of−1.5. Other scores (eg, fluency, Clock Drawing Test) were approximately−1.0. Given that z -score impairments of approximately−1.5 to−2.0 were present in multiple domains at the group level, at the level of individual cases approximately 35% to 45% of MCI patients demonstrate impairments of−2.0 or worse, ie, would meet cognitive criteria for AD based on a second domain beyond memory.

In effect, it appeared to us that many MCI patients may already have met cognitive criteria for AD, and the diagnostic trigger had little to do with a shift in the disease trajectory. In addition to this possibility, we suggest that a second smaller factor in conversion may reflect compromises in executive and cognitive control functions that reduce compensatory potential (because Trails B decline was a second, weaker predictor of conversion).

In summary, we believe that this study was comprehensive and systematic, in that we examined combinations of cognitive markers, brain volumetric biomarkers, and CSF biomarkers in predictive regression models. The sample that we used (from ADNI) was large, and the data were collected uniformly, rigorously, and prospectively. We demonstrated that cognitive markers were consistently significant and generally stronger predictors than biomarkers, and moreover, that conversion itself did not so much reflect a shift in the neurobiologic characteristics of the disease but rather a large functional decline.

Correspondence: Terry E. Goldberg, PhD, Litwin Zucker Alzheimer's Center/Zucker Hillside Hospital/A, 75-59 263rd St, Glen Oaks, NY 11004 (tgoldber@nshs.edu).

Submitted for Publication: February 8, 2011; final revision received April 29, 2011; accepted May 1, 2011.

Funding/Support: This study was supported by grants from the Litwin-Zucker Alzheimer's Center (Drs Gomar, Conejero-Goldberg, Davies, and Goldberg and Ms Bobes-Bascaran), and by grant FI00322 from Instituto de Salud Carlos III (Dr Gomar).

Additional Information: Data used in the preparation of this article were obtained from the ADNI database (http://adni.loni.ucla.edu/.. As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in the analysis or writing of this article. A complete listing of ADNI investigators is available at http://adni.loni.ucla.edu/. Data collection and sharing for this project was funded by ADNI (grant U01 AG024904 from the National Institutes of Health). The ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and generous contributions from the following: Abbott, AstraZeneca AB, Bayer Schering Pharma AG, Bristol-Myers Squibb, Eisai Global Clinical Development, Elan Corporation, Genentech, GE Healthcare, GlaxoSmithKline, Innogenetics, Johnson& Johnson, Eli Lilly and Co, Medpace, Inc, Merck and Co, Inc, Novartis AG, Pfizer Inc, F. Hoffman-La Roche, Schering-Plough, Synarc, Inc, as well as nonprofit partners the Alzheimer's Association and Alzheimer's Drug Discovery Foundation, with participation from the US Food and Drug Administration. Private sector contributions to ADNI are facilitated by the Foundation for the National Institutes of Health (http://www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Disease Cooperative Study at the University of California, San Diego. The ADNI data are disseminated by the Laboratory of Neuro Imaging at the University of California, Los Angeles. This research was also supported by grants P30 AG010129 and K01 AG030514 from the National Institutes of Health, and the Dana Foundation.

Additional Contributions: We thank Ted Huey, MD, for commenting on the manuscript.