Variable importance plot of protein biomarkers measured by the random forest built from the training set. ACE indicates angiotension converting enzyme; creatine kinase MB, creatine kinase muscle-brain; IL, interleukin; MCP1, monocyte chemoattractant protein 1; MIF, macrophage migration inhibitory factor; MIP1α, macrophage inflammatory protein 1-α; PARC, pulmonary and activation-regulated chemokine; TIMP1, tissue inhibitor of metalloproteinase 1; and TNF-α, tumor necrosis factor α.
Receiver operating characteristic curve for clinical variables alone and in conjunction with biomarker data.
Venn diagram demonstrating consistency across methods for identifying altered protein in test set expression in Alzheimer disease. Growth-regulated oncogene α (GRO-α) was only identified by the Wilcoxon test but not the other 2 methods; vitamin D binding protein (VDBP) was only identified by the logistic regression but not the other 2 methods. SAM indicates significant analysis of microarray.
Customize your JAMA Network experience by selecting one or more topics from the list below.
O’Bryant SE, Xiao G, Barber R, Reisch J, Doody R, Fairchild T, Adams P, Waring S, Diaz-Arrastia R, . A Serum Protein–Based Algorithm for the Detection of Alzheimer Disease. Arch Neurol. 2010;67(9):1077–1081. doi:10.1001/archneurol.2010.215
Copyright 2010 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2010
To develop an algorithm that separates patients with Alzheimer disease (AD) from controls.
Longitudinal case-control study.
The Texas Alzheimer's Research Consortium project.
We analyzed serum protein–based multiplex biomarker data from 197 patients diagnosed with AD and 203 controls.
Main Outcome Measure
The total sample was randomized equally into training and test sets and random forest methods were applied to the training set to create a biomarker risk score.
The biomarker risk score had a sensitivity and specificity of 0.80 and 0.91, respectively, and an area under the curve of 0.91 in detecting AD. When age, sex, education, and APOE status were added to the algorithm, the sensitivity, specificity, and area under the curve were 0.94, 0.84, and 0.95, respectively.
These initial data suggest that serum protein-based biomarkers can be combined with clinical information to accurately classify AD. A disproportionate number of inflammatory and vascular markers were weighted most heavily in the analyses. Additionally, these markers consistently distinguished cases from controls in significant analysis of microarray, logistic regression, and Wilcoxon analyses, suggesting the existence of an inflammatory-related endophenotype of AD that may provide targeted therapeutic opportunities for this subset of patients.
There is clearly a need for reliable and valid diagnostic and prognostic biomarkers of Alzheimer disease (AD), and in recent years, there has been an explosive increase of effort aimed at identifying such markers. It has been previously argued that, because of significant advantages, the ideal biomarkers would be gleaned from peripheral blood.1 Peripheral blood can be collected at any clinic (or in-home visit) whereas most clinics are not capable of conducting lumbar punctures. Furthermore, advanced neuroimaging techniques are typically only available in large medical centers of heavily urbanized areas. A blood-based algorithm greatly increases access to advanced detection, and while nearly all patients are willing to undergo venipuncture, fewer elderly patients agree to lumbar puncture and many are unable to undergo neuroimaging for a range of reasons (eg, pacemakers).
Even though there is a large literature demonstrating altered levels of a range of biomarkers (cerebrospinal fluid, serum, and plasma) in patients with AD (as well as patients with mild cognitive impairment) relative to controls, attempts to identify a single biomarker specific to AD have failed. In the highly publicized Ray et al2 publication, a large set of plasma-based proteins was analyzed in an effort to identify a biomarker profile indicative of AD. The overall classification accuracy for their algorithm was 90%; additionally, their algorithm accurately identified 81% of patients with mild cognitive impairment who would progress to AD within a 2- to 6-year follow-up period. To date, however, these findings have not been cross-validated nor has an independent blood-based (particularly serum-based) algorithm been published.
In addition to offering more accessible, rapid, and cost- and time-effective methods for assessment, biomarkers (or panels of biomarkers) also hold great potential for the identification of endophenotypes within AD populations that are associated with particular disease mechanisms. Once identified, targeted therapeutics specifically tailored to endophenotype status could be tested. Drawing on an example from cardiovascular disease, by identifying a subset of patients where atherosclerosis is pathogenically related to hypercholesterolemia, plasma cholesterol level is a useful biomarker in the management of coronary artery disease. Plasma cholesterol measurements are useful as indicators of efficacy of treatment with HMG-CoA reductase inhibitors. Translating this conceptual framework to AD would be a major advancement in this field.3 The identification of a proinflammatory endophenotype of AD would have implications for targeted therapeutics for a subgroup of patients such that those with an overexpression of the proinflammatory biomarker profile may benefit from treatment with anti-inflammatory compounds while those patients with an underexpression of this profile may get worse while receiving such treatment.
In the current study we sought to (1) determine if a serum-based biomarker algorithm would significantly predict AD status, (2) evaluate if inclusion of demographic variables directly into the algorithm would improve the overall classification accuracy, and (3) determine if there was a predominance of inflammatory-related markers that were overexpressed or underexpressed in AD, which would be an initial step toward the concept of an inflammatory-related AD endophenotype.
Participants included 400 individuals (197 subjects with AD, 203 controls) enrolled in the Texas Alzheimer's Research Consortium. The methods of the Texas Alzheimer's Research Consortium project have been described in detail elsewhere4; each participant underwent a standardized annual examination at the respective site, which included a medical evaluation, neuropsychological testing, and interview. Each participant also provided blood for storage in the Texas Alzheimer's Research Consortium biobank. Diagnosis of AD status was based on National Institute of Neurological and Communicative Disorders–Stroke and the Alzheimer's Disease and Related Disorders Association criteria5 and controls performed within normal limits on psychometric assessment. Institutional review board approval was obtained at each site and written informed consent was obtained for all participants.
Nonfasting blood samples were collected in serum-separating tubes during clinical evaluations, allowed to clot at room temperature for 30 minutes, centrifuged, aliquoted, and stored at −80°C in plastic vials. Batched specimens from either baseline or year 1 follow-up examinations were sent frozen to Rules-Based Medicine (RBM) (Austin, Texas, www.rulesbasedmedicine.com) where they were thawed for assay without additional freeze-thaw cycles using the RBM multiplexed immunoassay human multianalyte profile (HumanMAP). Multiple proteins were quantified though multiplex fluorescent immunoassay using colored microspheres with protein-specific antibodies. Information regarding the least-detectable dose, interrun coefficient of variation, dynamic range, overall spiked standard recovery, and cross-reactivity with other HumanMAP analytes can be readily obtained from RBM. As with all such technologies, rapid evolution is expected; therefore, the complete list of analytes used from the HumanMAP at the time of the current analyses is provided in Appendix 1 (http://www.txalzresearch.org/).
Analyses were performed using R statistical software (version 2.10).6 Fisher exact and Mann-Whitney U tests were used to compare cases vs controls for categorical variables (APOE ε4 allele frequency, sex, race, or ethnicity) and continuous variables (age and education). The biomarker data were log transformed and then standardized for each analyte. The random forest prediction model was performed using R package randomForest (version 4.5),7 with all software default settings. We used the method by Bair et al8 to decorrelate the RBM biomarker data and clinical variables. The receiver operating characteristic curves were analyzed and area under the curve (AUC) was calculated using R package DiagnosisMed (version 0.2.2.2). The significant analysis of microarray was performed using R package samr (version 1.27).9 The false discovery rate (FDR) was calculated to address the multiple comparison issues. The FDR from significant analysis of microarray analysis (R package SAMR version 1.28) was determined by permutation and those from the Wilcoxon test and logistic regression model were determined by fitting the P values to beta-uniform models.10 The beta-uniform models were fitted using R package ClassComparison (version 2.5.0) (http://bioinformatics.mdanderson.org/Software/OOMPA/).
Demographic characteristics of the study population are shown in Table 1. Patients with AD were significantly older (P < .001), less educated (P < .001), and more likely to carry at least 1 copy of the APOE ε4 allele (P < .001) than control participants.
Once randomized into a training set or a testing set via a random-number generator, a random forest prediction model was built with the training set using all of the markers in the RBM HumanMAP. Using the training set as a guide, the random forest algorithm assigned a risk score to each subject in the test set that was reflective of the probability of being diagnosed with AD. Using the HumanMAP markers, when the cutoff for the risk score was set to optimize performance at 0.47 (ie, patient's risk score >0.47 = AD, ≤0.47 = control), the AUC for the biomarker algorithm was 0.91 (95% confidence interval [CI], 0.88-0.95) and the sensitivity and specificity were equal to 0.80 (95% CI, 0.71-0.87) and 0.91 (95% CI, 0.81-0.94), respectively. When the nonoptimized cutoff of 0.5 was used, the results did not change significantly (AUC = 0.91, sensitivity = 0.73, specificity = 0.91). To test the robustness against allocation to training and test sets, randomization was also done by Texas Alzheimer's Research Consortium site, which yielded an AUC of 0.88 demonstrating the robustness of the algorithm against choice of methods. Figure 1 presents a variable importance plot of protein markers measured by the random forest built from the training set.
Next, the biomarker data were decorrelated8 from the clinical variables of age, sex, education, and APOE status and an additional random forest prediction model was generated. Results from the multivariate logistic regression model (Table 2) demonstrate that the biomarker risk score was a significant, independent predictor of case status. As can be seen in Table 3, clinical data alone accurately classified a large portion of the sample, which was comparable with, though somewhat less accurate than, the performance of the biomarker profile alone. However, a combined algorithm using biomarker and clinical data was superior to either alone (Table 3 and Figure 2). Using the nonoptimized cutoff for the biomarker risk score did not change the findings for the algorithm using both clinical and biomarker data (AUC = 0.95, sensitivity = 0.90, specificity = 0.87).
Significant analysis of microarray analysis with an FDR of less than 0.001 identified a total of 23 proteins that were either differentially overexpressed (n = 14) or underexpressed (n = 9) in patients with AD relative to controls (Table 4). There were 22 proteins identified by the Wilcoxon test with an FDR less than 0.0025 and 22 proteins by logistic regression with an FDR less than 0.01. Figure 3 demonstrates the consistency between methods used. Supporting our notion of a possible inflammatory-related endophenotype present in patients with AD, 10 (macrophage inflammatory protein 1, eotaxin 1, tumor necrosis factor α, fibrinogen, interleukin 5 [IL-5], IL-7, IL-10, C-reactive protein, monocyte chemoattractant protein 1, and von Willebrand factor) of the total 30 markers identified in Figure 1 were inflammatory in nature.
In a recently highly publicized study, Ray et al2 identified a subset of 18 plasma-based proteins that yielded excellent classification accuracy in cases vs controls and our serum protein-based algorithm yielded comparable accuracy. The markers from our study (Figure 1) have only minimal overlap with those presented by Ray at al (angiopoietin 2 and tumor necrosis factor α). It is likely that this differential signature profile resulted from different sample media as well as different assay platforms. Our study has multiple distinct advantages over that of Ray et al. First, our serum protein assays were conducted by RBM, who have developed high-throughput methods for reliable assay of high volumes of samples and analytes. Rules-Based Medicine is the leading biomarker company in the United States, working with multiple pharmaceutical companies and the Alzheimer's Disease Neuroimaging Initiative as well as several of the leading AD biomarker research laboratories both within and outside the United States. Second, our sample size of controls and AD cases is more than twice as large as the sample used by Ray and colleagues. Third, our study included demographic information in the predictive algorithm (age, sex, education, APOE status) and we demonstrated that the combination of biomarker and clinical information yields superior results to either alone. Finally, our study is unique in that we are the first group, to our knowledge, to present serum-based findings.
In support of our theory of the existence of an inflammatory endophenotype, many of the proteins with the highest importance from the random forest analyses were inflammatory in nature (Figure 1). Additionally, when significant analysis of microarray analyses were conducted, a large portion of the proteins identified either as overexpressed or underexpressed were inflammatory in nature. Taken together, these data suggest the existence of an inflammatory endophenotype within AD cases, which could offer targeted therapeutic options for this subgroup of patients.
It is possible that the algorithm identified in the current study is not AD specific. The current findings are preliminary in nature and follow-up is necessary to test the ability of the algorithm to detect AD when mixed in with non-AD dementia samples. It is also possible that the inflammatory signature observed is not specific to AD but rather is related to other comorbid factors (eg, cardiovascular disease). In fact, it is likely that a proinflammatory endophenotype exists within patients diagnosed with other dementia syndromes. Such a finding would further support the utility of a proinflammatory endophenotype, as it is likely to represent a common pathway for a wide array of diseases.
The identification of blood-based biomarker profiles with good diagnostic accuracy would have a profound impact worldwide and requires further validation. Additionally, the identification of pathway-specific endophenotypes among patients with AD would likewise have implications for targeted therapeutics as well as understanding differential progression among diagnosed cases. With the rapidly evolving technology and analytic techniques available, AD researchers now have the tools to simultaneously analyze exponentially more information from a host of modalities, which is likely going to be necessary to understand this very complex disease.
Correspondence: Sid E. O’Bryant, PhD, Texas Tech University Health Science Center, Department of Neurology, 3601 4th St, STOP 6232, Lubbock, TX 79430 (email@example.com).
Accepted for Publication: September 1, 2009.
Author Contributions:Study concept and design: O’Bryant, Doody, Waring, and Diaz-Arrastia. Acquisition of data: O’Bryant, Barber, Reisch, Doody, Fairchild, Waring, and Diaz-Arrastia. Analysis and interpretation of data: O’Bryant, Xiao, Barber, Doody, Adams, Waring, and Diaz-Arrastia. Drafting of the manuscript: O’Bryant, Xiao, Barber, Adams, and Waring. Critical revision of the manuscript for important intellectual content: O’Bryant, Barber, Reisch, Doody, Fairchild, Adams, Waring, and Diaz-Arrastia. Statistical analysis: O’Bryant, Xiao, Reisch, Adams, Waring, and Diaz-Arrastia. Obtained funding: O’Bryant, Doody, and Waring. Administrative, technical, and material support: O’Bryant, Barber, Reisch, Fairchild, and Adams. Study supervision: O’Bryant and Diaz-Arrastia.
Financial Disclosure: A patent is being filed covering the biomarker algorithm created from this work. The following authors are listed on the patent: Drs O’Bryant, Xiao, Barber, Diaz-Arrastia, Reisch, Doody, Adams, and Fairchild.
Texas Alzheimer's Research Consortium Investigators: Baylor College of Medicine: Eveleen Darby, Kinga Szigeti, Aline Hittle; Texas Tech University Health Science Center: Paula Grammas, Benjamin Williams, Andrew Dentino, Gregory Schrimsher, Parastoo Momeni, Larry Hill; University of North Texas Health Science Center: Janice Knebl, James Hall, Lisa Alvarez, Douglas Mains; University of Texas Southwestern Medical Center: Roger Rosenberg, Ryan Huebinger, Janet Smith, Mechelle Murray, Tomequa Sears.
Funding/Support: This study was made possible by the Texas Alzheimer's Research Consortium funded by the state of Texas through the Texas Council on Alzheimer's Disease and Related Disorders. Investigators at the University of Texas Southwestern Medical Center at Dallas also acknowledge support from the University of Texas Southwestern Alzheimer's Disease Center National Institutes of Health, National Institute on Aging grant P30AG12300.
Role of the Sponsors: The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.