Receiver operating characteristic (ROC) curves of the linear discriminant function (LDF), average thickness, and nasal quadrant thickness between healthy controls and patients with glaucoma in the validating set. These 3 parameters had the greatest areas under the ROC curve (AUCs): AUC for our LDF, 0.922 (95% confidence interval [CI], 0.901-0.943); AUC for average thickness, 0.914 (95% CI, 0.892-0.937); and AUC for the nasal quadrant thickness, 0.877 (95% CI, 0.849-0.906).
Ferreras A, Pablo LE, Pajarín AB, Larrosa JM, Polo V, Honrubia FM. Logistic Regression Analysis for Early Glaucoma Diagnosis Using Optical Coherence Tomography. Arch Ophthalmol. 2008;126(4):465-470. doi:10.1001/archopht.126.4.465
Copyright 2008 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2008
To determine and validate the diagnostic ability of a linear discriminant function (LDF) based on the retinal nerve fiber layer thickness at each of the 12 clock-hour positions obtained using optical coherence tomography for discriminating between healthy eyes and eyes with early glaucomatous visual field loss.
We prospectively selected 62 consecutive healthy individuals and 73 patients with open-angle glaucoma to calculate the LDF. Another independent prospective sample of 280 healthy eyes and 302 glaucomatous eyes was used to evaluate the diagnostic accuracy of the LDF.
The proposed function was LDF = 15.584 – (12-o’clock segment thickness × 0.032) – (7-o’clock segment thickness × 0.041) – (3-o’clock segment thickness [nasal side] × 0.121). The greatest area under the receiver operating characteristic curve was observed for our LDF in both populations: 0.962 and 0.922. Our LDF and the average thickness yielded sensitivities of 74.5% and 67.8%, respectively, at a fixed specificity of 95%.
The LDF increased the diagnostic ability of the isolated retinal nerve fiber layer thickness at the 12 clock-hour positions. Compared with optical coherence tomography–provided parameters, our LDF had the highest sensitivities at 85% and 95% fixed specificities to discriminate between healthy and early glaucomatous eyes.
The detection of defects in the retinal nerve fiber layer (RNFL) is key for early glaucoma diagnosis.1,2 Red-free photographs have been used for decades to qualitatively assess RNFL status. The highly subjective nature of this method and the requirement for experienced evaluators, however, limit its general applicability.3 In recent years, different instruments have been introduced to quantitatively measure peripapillary RNFL thickness. One of these techniques is optical coherence tomography (OCT), which provides objective, quantitative, and reproducible data.
The Stratus OCT 3000 (Carl Zeiss Meditec, Dublin, California) is a computer-assisted precision optical instrument that delineates cross-sectional anatomy of the retina with a 10 μm or smaller axial resolution. The OCT assesses RNFL thickness as the distance between the vitreoretinal interface and the RNFL posterior boundary.4,5
The aim of this study was to optimize the sensitivity-specificity balance of RNFL thickness parameters of the OCT using a binary logistic regression analysis. This method can be used to find a linear combination of the variables whose value is as similar as possible within groups and as different as possible between groups. The linear combination is called a linear discriminant function (LDF). In our study, we used this procedure to determine which RNFL parameters of the OCT were more useful for differentiating between healthy eyes and eyes with early glaucomatous visual field defects.
Several factors threaten the internal and external validity of a study of diagnostic accuracy, which inspired the launch of the Standards for Reporting of Diagnostic Accuracy initiative.6 Its objective is to improve the quality of the reporting of studies of diagnostic accuracy. The design of our study followed all 25 items of the Standards for Reporting of Diagnostic Accuracy guidelines. To our knowledge, this is the first study to assess the diagnostic ability of an LDF designed for the Stratus OCT 3000 based on RNFL thickness. The strength of this study lies in the validation of our LDF using an independent sample.7
The prospective study protocol was approved by the ethics committee of Miguel Servet University Hospital, and informed written consent was obtained from all participants. The design of the study followed the tenets of the Declaration of Helsinki for biomedical research.
From January to December 2006, 2 samples of consecutive healthy controls and patients with glaucoma were prospectively pre-enrolled from 2 outpatient clinics under the area of influence of our hospital. One outpatient clinic was randomly selected to provide the population for calculating the discriminant analysis (teaching set) and the other was selected to supply the population for validating the performance of the LDF in an independent group (validating set).
Five of the participants did not provide informed consent, 14 did not complete all of the required tests, and 31 could not perform at least 1 of the tests included in the study protocol (19 of them did not perform a reliable standard automated perimetry [SAP] and the other 12 had poor-quality OCT scans, after 3 attempts in both tests); they were excluded from further analysis. Finally, 717 eyes from white individuals were included in the statistical analysis. One eye from each participant was randomly chosen for the study, unless only 1 eye met the inclusion criteria.
Healthy eyes were consecutively recruited from hospital staff, relatives of patients in our hospital, and from patients who were referred for refraction who underwent routine examination without abnormal ocular findings. Patients with glaucoma were recruited consecutively from an ongoing longitudinal follow-up study at the Miguel Servet University Hospital.
Participants had to meet the following inclusion criteria: best-corrected visual acuity of 20/40 or better, refractive error within ± 5.00 diopters equivalent sphere and ± 2.00 diopters astigmatism, transparent ocular media (nuclear color/opalescence and cortical or posterior subcapsular lens opacity < 1) according to the Lens Opacities Classification System III system,8 and open anterior chamber angle. Participants with previous intraocular surgery, diabetes, or other systemic diseases, history of ocular or neurologic disease, current use of medication that could affect visual field sensitivity, or moderate or severe visual field defect in SAP based on the Hodapp-Parrish-Anderson score9 were excluded. All participants underwent a full ophthalmologic examination, including clinical history, visual acuity measurement, biomicroscopy of the anterior segment using a slitlamp, gonioscopy, Goldmann applanation tonometry, central corneal ultrasonic pachymetry (model DGH 500; DGH Technology, Exton, Pennsylvania), and ophthalmoscopy of the posterior segment.
At least 2 reliable SAP tests per eye were performed using a Humphrey field analyzer (model 750; Zeiss Humphrey Systems, Dublin, California) with the Swedish interactive threshold algorithm standard 24-2 test. If fixation losses and false-positive or false-negative rates were greater than 20%, the test was repeated. The second reliable perimetry test obtained was used in this study to minimize the learning effect.10,11 Abnormal SAP results were considered reproducible glaucomatous visual field loss in the absence of any other abnormalities to explain the defect. A visual field loss was defined as the presence of a cluster of 3 points lower than P < .05 or a cluster of 2 points lower than P < .01 on a pattern deviation plot12 and/or a pattern SD significantly elevated beyond the 5% level and/or a Glaucoma Hemifield Test result outside normal limits. Each perimetry was performed on different days to avoid a fatigue effect.
The Zeiss Stratus OCT 3000 was used to measure peripapillary RNFL thickness. The RNFL thickness 3.46-mm scan protocol was used to acquire the OCT images. The RNFL thickness average (OU) analysis protocol was used to obtain the variables included in our study. Good-quality scans had to have focused images from the ocular fundus and a centered circular ring around the optic disc. Examinations with a signal-to-noise ratio less than or equal to 33 dB or less than 95% accepted A-scans were performed again. All the ophthalmic examinations were performed within 1 month of the participant's date of enrollment into the study.
Eyes with an intraocular pressure less than 21 mm Hg, no history of increased intraocular pressure, and a normal SAP were considered healthy. Eyes with an intraocular pressure greater than 21 mm Hg (on ≥ 3 readings on different days) and typical SAP defects, regardless of the appearance of the optic disc, were considered glaucomatous. Two glaucoma specialists masked to patient identity and clinical history classified the eyes. Any disagreement was resolved by consensus.
All statistical analyses were calculated using SPSS, version 15.0 (SPSS Inc, Chicago, Illinois), and MedCalc, version 188.8.131.52 (MedCalc Software, Mariakerke, Belgium). The teaching set was used to perform a binary logistic regression, which is a form of regression used when the dependent variable is dichotomous (healthy or diseased) and the independent variables are of any type. The dependent variable was the presence of disease and the relative importance of each independent variable was assessed by stepwise binary logistic regression analysis using the forward Wald method. The stepwise probability test determined the criteria through which variables were entered into and removed from the model. The independent variables were the RNFL thickness at each of the 12 clock-hour positions (with the 3-o’clock position as nasal, 6-o’clock position as inferior, 9-o’clock position as temporal, and 12-o’clock position as superior, regardless of which side the eye was on). The stepwise procedure identified the segment that accounted for the greatest amount of error, then included the next best variable and so on. The LDF score was obtained by taking a weighted sum of the predictor variables.
The significant RNFL thickness parameters of the OCT were combined to generate a new variable (the LDF) in such a way that the measurable differences between the groups were maximized. Our LDF was defined as follows: 15.584 – (12-o’clock segment thickness × 0.032) – (7-o’clock segment thickness × 0.041) – (3-o’clock segment thickness × 0.121).
The validating set was used to test and compare the diagnostic ability of our LDF and other RNFL parameters of the OCT. The receiver operating characteristic curves were plotted for all of them and compared with the proposed LDF. The areas under the receiver operating characteristic curve (AUCs) were compared using the Hanley-McNeil method.13 The cut-off points were calculated by the MedCalc software as the points with the best sensitivity-specificity balance. Sensitivities at 85% and 95% (5% false-positive rate) fixed specificities, and positive and negative likelihood ratios (LRs) were also calculated.
The teaching set consisted of 135 eyes divided into 62 healthy eyes and 73 glaucomatous eyes (61 with primary open-angle glaucoma, 10 with pseudoexfoliative glaucoma, and 2 with pigmentary glaucoma). The mean (SD) age was 59.3 (9.7) years for the healthy group and 61.6 (7.2) years for the glaucoma group (Table 1). The validating set included 280 healthy controls and 302 patients with glaucoma (245 with primary open-angle glaucoma, 43 with pseudoexfoliative glaucoma, and 14 with pigmentary glaucoma). The mean (SD) age of the healthy group was 60.1 (10.2) years and the mean (SD) age of the glaucomatous group was 61.4 (7.4) years. Age and central corneal thickness did not differ significantly (P > .05) between the groups in either sample.
Table 2 presents the mean (SD) values of all parameters evaluated in the teaching and validating sets. At the 12 clock-hour positions and in the 4 quadrants, mean RNFL thickness values were distributed according to the inferior-superior-nasal-temporal (ISNT) rule14- 17 in the healthy groups for both populations. Nevertheless, in the glaucoma groups, the ISNT rule was not maintained because the differences between quadrant thicknesses were reduced: superior and inferior quadrant thicknesses were similar, and nasal and temporal quadrant thicknesses were similar.
In the teaching set, the highest sensitivity-specificity balance was observed for our LDF (89.0% and 91.9%, respectively) and the average thickness (86.3% and 95.2%, respectively). The RNFL thickness at the 12 clock-hour positions had worse diagnostic ability than our LDF. Our LDF had the greatest AUC (0.962; SE, 0.016). The largest AUCs for the provided OCT parameters were 0.958 (SE, 0.017) for the average thickness and 0.922 (SE, 0.025) for the nasal quadrant thickness. There was no significant difference between the AUCs of these parameters. Our LDF and the average thickness yielded similar diagnostic ability: 93.1% and 91.7% sensitivities at a fixed specificity of 85%, respectively.
In the validating set, the average thickness and our LDF had the best pairs of sensitivity-specificity (Table 3): 77.8%/93.6% and 81.8%/88.6%, respectively. The average thickness (LR = 12.10) and our LDF (LR = 7.16) had the highest positive LRs, while the nasal quadrant thickness (LR = 0.19), our LDF (LR = 0.21), and the RNFL thickness at the 3-o’clock position (LR = 0.21) had the lowest negative LRs.
The greatest AUCs were 0.922 (SE, 0.012) for our LDF, followed by the average thickness (0.914; SE, 0.012) and the nasal quadrant thickness (0.877; SE, 0.014). Significant differences were found between the AUC of nasal quadrant thickness and our LDF (P < .001) and the average thickness (P = .005). There were no significant differences between the AUCs of our LDF and the average thickness (P = .33) (Table 3 and the Figure). At a fixed specificity of 85%, our LDF and the average thickness yielded sensitivities of 82.7% and 79.1%, whereas at a fixed specificity of 95%, the sensitivities were 74.5% and 67.8%, respectively.
Previous studies16- 22 have reported the sensitivity and specificity of OCT for discriminating between healthy and glaucomatous eyes, for which OCT-provided parameters have the best ability to detect RNFL glaucomatous defects. The purpose of our study was to search for an optimal combination of RNFL thickness parameters (30° sectors) to improve the ability of OCT to diagnose glaucoma. Very few studies23- 25 have tried to increase the diagnostic ability of OCT using an LDF. All of the previous studies combined RNFL and optic nerve head variables, but to our knowledge, our study is the only one aimed at calculating an LDF based solely on RNFL parameters.
Huang and Chen23 and Chen et al24 compared automated classifications for glaucoma and developed a logistic regression analysis, including both RNFL thickness and optic nerve head parameters obtained with OCT. They included 4 sets of 20 patients with glaucoma and 20 healthy Taiwanese Chinese individuals and conducted a 4-fold cross-validation study. They reported an AUC of 0.911 with 83.7% sensitivity at 80% specificity. Medeiros et al25 also calculated an LDF and validated it in an independent population. They obtained an AUC of 0.97 in both populations, but the size of the validation sample was relatively small and contained a higher proportion of moderate and advanced cases.
These studies23- 25 included optic nerve head parameters in their analyses and all of them used normal optic disc morphology to classify the healthy group. These inclusion criteria might overestimate the diagnostic accuracy of OCT owing to the optic nerve head parameters. In our study, only the RNFL thicknesses at each of the 12 clock-hour positions were included in the logistic regression, and groups were divided regardless of optic disc appearance; thus, if we had included participants with preperimetric glaucoma in the healthy group, we might have underestimated the diagnostic accuracy. These previous studies required 2 scan protocols and 2 analysis protocols, which potentially introduce an additional source of variability, lengthen the time required to perform the test, extend the time needed to interpret the results (2 analyses and more variables in the equation), and increase the cost per examination. Our LDF formula was based on only 3 of the 12 clock-hour positions and the validation sample showed 83% and 75% sensitivity at 85% and 95% fixed specificities, respectively, for early glaucoma diagnosis. Differing designs, inclusion and exclusion criteria, and the level of damage in the visual field defects make it difficult to compare the results across several studies. Obviously, the severity of visual field loss has an important influence on imaging instrument sensitivity.26 More severe disease is associated with increased sensitivity; therefore, in populations with patients with moderate and severe visual field loss, a higher sensitivity-specificity balance for the discriminant functions might be expected.
Our results are consistent with those of Chen et al24 and Medeiros et al25 in that the 7-o’clock and 12-o’clock RNFL thickness positions were included as variables in the LDF. The RNFL bundles are thicker in the superior and inferior regions and thinner in the temporal and nasal areas. Thus, OCT can measure changes in the vertical axis more easily because changes in horizontal meridians are smaller. Furthermore, the superior and inferior poles of the optic nerve head are the sectors more commonly affected at early stages of glaucoma disease.14- 17,27 In our study, the nasal sectors (3-o’clock position) also had good diagnostic ability, but temporal sectors (papillomacular bundle) were less sensitive for detecting glaucomatous changes. This is in agreement with a previous study27 reporting that the RNFL thickness is usually preserved in the region of the papillomacular bundle until late in the course of the disease.
Depending on the pretest probability, the positive or negative LR tells us how much the odds of disease will increase or decrease, respectively. An LR28 close to 1 indicates insignificant effects, whereas LRs higher than 10 or lower than 0.1 often indicate large changes in posttest odds of the disease. In both populations, the average thickness and our LDF gave the highest positive LRs, indicating that abnormal results would be associated with important posttest effects. On the other hand, the nasal quadrant thickness, our LDF, and the RNFL thickness at the 3-o’clock position showed the lowest negative LRs in both samples; thus, normal results are associated with a big change in the posttest probability of disease for these variables and a better ability to exclude the presence of glaucoma.
In general, sensitivities of the best RNFL OCT parameters ranged from 70% to 80% at a fixed specificity of 85% in the teaching set, while the sensitivities were slightly higher (80%-90%) in the validating set. The teaching set had a higher pattern SD of SAP, which might be the cause of such a small difference. The average thickness was the best OCT-provided parameter showing almost the same diagnostic ability as our LDF. Many studies16,19,22,25 also report that this parameter yields a high sensitivity-specificity balance for perimetric glaucoma diagnosis.
The ethnic characteristics of the validation set were similar to those of the teaching set, and this might have biased results toward our LDF when compared with other OCT parameters in the second population. The quality of the data obtained by the imaging devices is influenced by the media opacity, retinal pigment epithelium status, instrument variability, and positioning and centering of the images. These limitations must be taken into account in clinical practice. Other statistical analyses23,29,30 could provide alternative formulas to increase the diagnostic performance of OCT parameters. All these studies demonstrated that automated classifiers based on OCT had a good diagnostic ability for distinguishing patients with glaucoma from healthy controls.
Retinal nerve fiber layer thickness can vary widely among healthy individuals, limiting the usefulness of isolated thickness values to differentiate them from patients with glaucoma. Our LDF combined the most useful RNFL thicknesses of the 12 clock-hour positions and increased the ability of the OCT to diagnose glaucoma. The results in the second sample confirmed those obtained in the teaching set.
Correspondence: Antonio Ferreras, MD, PhD, Department of Ophthalmology, Miguel Servet University Hospital, Isabel la Católica 1-3, 50009 Zaragoza, Spain (email@example.com).
Submitted for Publication: April 11, 2007; final revision received October 15, 2007; accepted October 19, 2007.
Financial Disclosure: None reported.