Discriminating Between Normal and Glaucomatous Eyes Using the Heidelberg Retina Tomograph, GDx Nerve Fiber Analyzer, and Optical Coherence Tomograph

Objective  To compare the ability of 3 instruments, the Heidelberg Retina Tomograph(HRT), the GDx Nerve Fiber Analyzer (GDx), and the Optical Coherence Tomograph(OCT), to discriminate between healthy eyes and eyes with early to moderate glaucomatous visual field loss.

Subjects and Methods  Forty-one patients with early to moderate glaucomatous visual field loss and 50 healthy subjects were included in the study. The HRT, GDx, and OCT imaging and visual field testing were completed on 1 eye from each subject within a 6-month interval. Statistical differences in sensitivity at fixed specificities of 85%, 90%, and 95% were evaluated. In addition, areas under the receiver operating characteristic (ROC) curve were compared.

Results  No significant differences were found between the area under the ROC curve and the best parameter from each instrument: OCT thickness at the 5-o'clock inferior temporal position (mean ± SE, 0.87 ± 0.04), HRT mean height contour in the nasal inferior region (mean ± SE, 0.86 ± 0.04), and GDx linear discriminant function (mean ± SE, 0.84 ± 0.04). Twelve HRT, 2 GDx, and 9 OCT parameters had an area under the ROC curve of at least 0.81. At a fixed specificity of 90%, significant differences were found between the sensitivity of OCT thickness at the 5-o'clock inferior temporal position (71%) and parameters with sensitivities less than 52%. Qualitative assessment of stereophotographs resulted in a sensitivity of 80%.

Conclusion  Although the area under the ROC curves was similar among the best parameters from each instrument, qualitative assessment of stereophotographs and measurements from the OCT and HRT generally had higher sensitivities than measurements from the GDx.

NEW METHODS including confocal scanning laser ophthalmoscopy, scanning laser polarimetry, and optical coherence tomography have been developed to provide real-time, quantitative information describing the optic disc and retinal nerve fiber layer (RNFL). Each of these methods uses different features of the retina or RNFL, and different properties of light to obtain clinical measurements of optic disc topography or RNFL thickness. Because RNFL and optic disc damage have been shown to precede visual field loss,^1-8 objective methods of measuring these conditions may help physicians diagnose and monitor glaucoma.

Despite considerable overlap in optic disc and RNFL measurements, statistically significant differences between glaucomatous and healthy eyes using either confocal scanning laser ophthalmoscopy, scanning laser polarimetry, or optical coherence tomography parameters have been reported.^9-14 Several different strategies are available for summarizing the information on optic disc topography and RNFL thickness obtained with these instruments. First, each method provides several summary measurements of single parameters(eg, RNFL thickness, rim area, or cup shape). The use of combinations of parameters in discriminant analysis has also been evaluated.^10,15 Finally, normative databases have been used to identify parameters outside a normal range of measurements.

There are, however, conflicting reports regarding the ability of these methods to discriminate between normal and glaucomatous eyes. Because of the large variability in reported sensitivity and specificity measures, studies are needed to confirm estimates of these measures by different investigators in varied study populations. Comparison across studies can be difficult because of differences in study design and characteristics of the study population. Because the ability to detect glaucoma improves with increasing severity of the disease, it is important to compare methods using similar inclusion and exclusion criteria as well as similar population characteristics, including age and severity of glaucoma. To best reduce the influence of population characteristics across studies, it is valuable to compare the ability of each method to detect glaucoma in the same study population.

This study was designed to compare, in one study population, the ability of the 3 methods to discriminate between healthy eyes and eyes with early to moderate glaucomatous visual field loss using commercially available instruments: a confocal scanning laser ophthalmoscope (Heidelberg Retina Tomograph [HRT]; Heidelberg Engineering, Heidelberg, Germany), a scanning laser polarimeter(GDx Nerve Fiber Analyzer [GDx]; Laser Diagnostic Technologies, San Diego, Calif), and the Optical Coherence Tomograph (OCT; Humphrey-Zeiss, Dublin, Calif).

The study included 1 randomly selected eye from each of 41 glaucoma patients with early to moderate glaucomatous visual field loss and each of 50 healthy subjects. Healthy subjects were included if their age equaled or exceeded that of the youngest patient with glaucoma (40 years of age). Ninety percent of normal subjects and patients with glaucoma were younger than 78 years. Mean age of the glaucomatous patients and normal subjects was 66.2 years (95% confidence interval [CI], 62.2-70.4 years) and 56.1 years (95% CI, 52.6-59.5 years), respectively.

Prior to ocular imaging, all subjects underwent a complete ophthalmologic examination that included slitlamp biomicroscopy, intraocular pressure (IOP) measurement, dilated stereoscopic fundus examination, stereoscopic photography of the optic disc, and standard (achromatic) full-threshold visual field testing with program 24-2 (Humphrey Field Analyzer; Humphrey-Zeiss). Informed consent was obtained from all participants, and the University of California–San Diego Human Subjects Committee approved all methods used.

Glaucomatous visual field loss was defined as a corrected-pattern standard deviation outside of the 95% normal limits or a glaucoma hemifield test outside of the 99% normal limits. Two abnormal visual fields were required. Mean deviation of the visual field test in these glaucomatous (open angle) eyes was −5.14 dB (95% CI, −3.47 to −6.81 dB), indicating early to moderate visual field damage. In 3 cases, when repeated achromatic visual field testing was unavailable, repeated abnormal fields were defined as 1 abnormal achromatic field and 1 abnormal short-wavelength automated perimetry field. The same criteria for defects were used for short-wavelength automated perimetry fields and achromatic fields. Patients with glaucoma had no history of diabetes and were not using medication known to affect visual sensitivity at the time of visual field testing. Best-corrected visual acuity was 20/40 or better. To avoid overstating the area under the receiver operating characteristic (ROC) curves and sensitivities of the tests,¹⁶ appearance of the optic disc and/or RNFL were not inclusion or exclusion criteria for the glaucoma group.

Healthy eyes had a measured IOP of 22 mm Hg or less with no history of elevated IOP. These eyes had intact rims; no evidence of hemorrhage, notching, excavation, or RNFL defect; and symmetrical optic discs (asymmetry of vertical cup-disc ratio <0.2) based on clinical examination. Visual field results showed a corrected-pattern standard deviation, mean deviation, and glaucoma hemifield test results within normal limits. Healthy subjects had no history of diabetes or other systemic disease, had no ophthalmological or neurological surgery or disease, and were not using medication known to affect visual sensitivity at the time of visual field testing. Best-corrected visual acuity at the time of testing was 20/40 or better.

All subjects underwent ocular imaging with the HRT, GDx, and OCT. For each subject, all ocular imaging and visual field examinations were completed within 6 months.

The HRT employs confocal scanning diode technology to provide topographical measures of the optic disc and peripapillary retina. The topographical image is derived from 32 optical sections at consecutive focal depth planes. Each image consists of 256×256 pixels, with each pixel corresponding to retinal height at its location.

Topographical parameters included with HRT software and investigated in this study were mean cup depth, maximum cup depth, height variation in contour, mean height contour, cup shape, disc area, cup area, cup-disc area ratio, cup volume below surface, rim area, rim volume above reference plane, rim disc ratio, RNFL thickness, RNFL cross-section, and reference height. We also examined values from the discriminant analysis formula of Mikelberg et al¹⁰ (the HRT classification in current HRT software version 2.01; Heidelberg Engineering, Heidelberg, Germany) and one developed by Bathija et al.⁹ This instrument and these parameters have been discussed in more detail elsewhere.^10,17,18

Several of these parameters were further examined by region. Temporal superior (45°-90° unit circle), nasal superior (91°-135°), nasal inferior (226°-270°), and temporal inferior (271°-315°) regions for disc area, cup area, mean height contour, cup volume, rim volume, maximum cup depth, cup shape, rim area, rim disc ratio, and RNFL thickness were all evaluated. Particular attention was given to the superior and inferior regions because these areas have proved informative in glaucoma diagnosis.^19,20

Three 15° field-of-view scans judged to be of acceptable quality were obtained for each test eye. A mean topographic image of these 3 scans was created using HRT software version 2.01. A trained technician outlined the optic disc margin on the mean topographic image while viewing stereoscopic photographs of the optic disc.

The GDx Nerve Fiber Analyzer uses scanning laser technology coupled with an integrated polarization modulator to provide a retardation map of the peripapillary retina based on the birefringent properties of the RNFL. This instrument measures retardation of light that has double passed the birefringent fibers of the RNFL. Each resulting image consists of 256×256 pixels, with each pixel corresponding to the retardation value at its location.

GDx software–provided parameters investigated in this study were the GDx number (a neural network assessment of glaucoma likelihood); average thickness, volume, and symmetry (superior quadrant thickness/inferior quadrant thickness); superior ratio (superior quadrant thickness/temporal quadrant thickness); inferior ratio (inferior quadrant thickness/temporal quadrant thickness); superior/nasal ratio; maximum modulation (thickest quadrant/thinnest quadrant)/(thinnest quadrant); superior maximum (average of thickest 1500 pixels in superior quadrant); inferior maximum; ellipse modulation; ellipse average; superior average; inferior average; and superior integral. The latter 5 parameters are measured relative to an ellipse surrounding the optic disc. GDx quadrants are defined as temporal (334°-24° unit circle), superior(25°-144°), nasal (145°-214°), and inferior (215°-334°). The value of a discriminant analysis model proposed by Weinreb et al¹⁵ was also investigated. Details of this instrument and these parameters have been discussed elsewhere.^{11,12,15,21,22}

Three scans judged to be of acceptable quality were obtained for each test eye. A mean retardation map composed of these 3 scans was created using GDx software version 2.0.09. The optic disc margin was outlined on the mean retardation image by a trained technician.

The OCT employs low-coherence interferometry to assess peripapillary RNFL thickness. Details of this instrument have been described elsewhere.^14,23,24 In brief, the OCT measures RNFL thickness by the difference in temporal delay of back-scattered light from the RNFL and a reference beam. The RNFL is differentiated from other retinal layers using an edge detection algorithm. Nerve fiber layer thickness is defined as the number of pixels between the anterior and posterior edges of the RNFL (software version A4X1; Humphrey-Zeiss). Each resulting image consists of RNFL thickness measurements (in micrometers) at 100 points along a 360° circular ring around the optic disc.

Optical Coherence Tomography parameters automatically calculated with existing software included mean RNFL thickness (360° measure), temporal quadrant thickness (316°-45° unit circle), superior quadrant thickness(46°-135°), nasal quadrant thickness (136°-225°), inferior quadrant thickness (226°-315°), and thickness for each of 12 clock-hour positions with the 3-o'clock position as temporal; 6-o'clock position, inferior; 9-o'clock position, nasal; and 12-o'clock position, superior. The following parameters were derived from the existing measurements in an attempt to measure modulation or differences between the superior and inferior regions and between the nasal and temporal RNFL thickness measurements: inferior 3 minus nasal(mean RNFL thickness of the inferior 3 clock-hour positions [5-, 6-, and 7-o'clock positions] minus RNFL thickness of the nasal clock-hour position [9]), inferior 3 minus temporal, superior 3 minus nasal, superior 3 minus temporal, and maximum minus minimum (mean RNFL thickness in thickest quadrant minus mean RNFL thickness in the thinnest quadrant).

Three circular 3.4-mm-diameter scans, centered on the optic disc and judged to be of acceptable quality, were obtained for each test eye. This approximate scan diameter was found to be optimal for RNFL analysis in a prototype instrument.²⁵ The landmark option was used to facilitate placement of the scan circle at the same location on repeated scans. Mean RNFL thickness for quadrant, clock-hour measurements, and derived parameters were determined from the 3 images obtained.

Simultaneous stereophotographs were obtained using a Topcon camera (TRC-SS; Topcon Instrument Corp of America, Paramus, NJ) after maximal pupil dilation. All photograph evaluations were performed using a stereoscopic viewer (Asahi Pentax Stereo Viewer II; Asahi Optical Co, Tokyo, Japan) with a standard fluorescent lightbulb. At least 2 experienced graders (C.V., E.Z.B., and C.A.S.), masked to patient identification and diagnosis, reviewed each photograph independently and recorded the photograph as either glaucomatous, normal, indeterminate, or poor quality. For this study, indeterminate was classified as normal, and poor-quality photographs were excluded from the analysis. Disagreements in grading were resolved by consensus.

t Tests were used to evaluate optic disc and RNFL measurement differences between glaucomatous and healthy eyes. Bonferroni adjustments were made based on the number of comparisons within each analysis(eg, HRT, GDx, and OCT analyses).

Receiver operating characteristic curves were used to describe the ability of each parameter to differentiate glaucomatous from healthy eyes. An area under the ROC curve of 1.0 represents perfect discrimination, whereas an area of 0.5 represents chance discrimination. The method of DeLong et al²⁶ was used to compare areas under the ROC curve. Minimum specificity cutoffs of 85%, 90%, and 95% were used for comparing the sensitivity of the best single parameter with the sensitivity of the remaining parameters by the McNemar test for paired proportions.

Measures for the 20 global and regional HRT parameters with the largest areas under the ROC curve for both glaucomatous and healthy eyes are presented in Table 1. After the Bonferroni adjustment (α = .0008; 59 comparisons), significant differences between study groups were found for all HRT parameters except mean cup depth, maximum cup depth, disc area, cup volume, and reference height, as well as the 4 disc area regions (superior temporal, superior nasal, inferior temporal, and inferior nasal) and the 4 maximum cup depth regions.

The HRT parameters with the greatest area under the ROC curve (ROC area) were mean height contour nasal inferior (0.86), linear discriminant function(0.85),⁹ and rim disc ratio temporal inferior(0.84) (Table 1, Figure 1). Nine of the top 10 regional parameters reflected measurements in the inferior region. There were no significant differences between ROC curve areas for these 20 parameters (for all comparisons, P>.05).

Measures for GDx parameters for both glaucomatous and normal eyes and areas under the ROC curve are presented in Table 2. After the Bonferroni adjustment (α = .003; 15 comparisons), significant differences between study groups were found for all GDx parameters except symmetry (P = .024), superior ratio (P = .007), and inferior ratio (P= .149).

The GDx parameters with the greatest area under the ROC curve (ROC area) were linear discriminant function¹⁵ (0.85), GDx number (0.81), superior-nasal ratio (0.77), superior maximum (0.77), and superior average (0.77). There were no significant differences between ROC curve areas for these parameters (for all comparisons, P>.05). Areas under the ROC curve for the 3 parameters with the largest areas are shown in Figure 2.

Measures for all OCT parameters for both glaucomatous and normal eyes and areas under the ROC curve are presented in Table 3. After the Bonferroni adjustment (α = .002; 22 comparisons), significant differences between study groups were found for all OCT parameters except nasal quadrant thickness, thickness at the 9-o'clock position (nasal clock-hour), thickness at the 3-o'clock position (temporal clock-hour), and thickness at the 8-o'clock position.

The OCT parameters with the greatest area under the ROC curve were all related to the inferior region: thickness at the 5-o'clock position (0.87), inferior quadrant thickness (0.84), and inferior 3 minus nasal (0.83) (Figure 3). There were no significant differences between ROC curve areas for these parameters (for all comparisons, P>.05).

We also determined whether a linear discriminant function combining individual OCT parameters would be a better predictor of glaucoma than single parameters by using Fisher linear discriminant functions to develop classification rules. These rules differentiate glaucomatous from healthy eyes by using all subsets of measures from the OCT. We created linear discriminant functions by examining the ratios for subsets consisting of 1 variable, 2 variables, 3 variables, and so on. No linear discriminant function resulted in a greater area under the ROC curve than the single best individual parameter (thickness at the 5-o'clock position).

Differences in RNFL thickness may have been affected by differences in disc size between diagnostic groups; in subjects with larger discs, OCT RNFL measurements are taken nearer to the disc because of the set radius of the circular scan. We examined the difference in disc size between the diagnostic groups (t test) and found no significant results(P>0.1 for all).

The 3 parameters with the greatest area under the ROC curve for each instrument were OCT thickness at the 5-o'clock position (0.87), HRT mean height contour nasal inferior (0.86), and GDx linear discriminant function (0.84)(Figure 4). There were no significant differences between ROC curve areas for these parameters (for all comparisons, P>.05).

Among parameters with an area under the ROC curve of at least 0.81, 9 were OCT parameters, 12 were HRT parameters, and 2 were GDx parameters (Table 4). Areas under the ROC curve of the parameters in Table 4 were not significantly different from each other (for all comparisons, P>.05). However, significant (P<.05) differences between the sensitivity of the single best individual parameter, OCT thickness at the 5-o'clock position, and parameters with sensitivities of approximately 47% or less were found at specificities of at least 90% (6 parameters) and 95% (18 parameters) (Table 4). At a specificity of 85%, significant differences were found between the variable with the highest sensitivity, HRT mean height contour nasal inferior (81%), and 8 parameters with sensitivities less than 59%.

Interestingly, at specificities of 90% or greater and 95% or greater, 6 (14.6%) of 41 and 10 (24%) of the 51 glaucomatous eyes in this study, respectively, were classified as normal by each of the "best" (largest ROC area) parameters from each instrument (Figure 5). At a specificity of 85%, all 41 patients were classified as having glaucomatous eyes by at least 1 instrument. There also was limited agreement among these parameters concerning which eyes were glaucomatous. Depending on the specificity level chosen, between 5 and 17 eyes were correctly identified as glaucomatous by the best parameters of all 3 instruments (Figure 5). There was better agreement in the correct classification of normal eyes. Thirty-two (64%), 40 (80%), and 46 (92%) of the 50 normal eyes were correctly classified by the best parameters from each instrument for specificities of 85%, 90%, and 95%, respectively. All normal eyes were correctly classified by at least 1 instrument.

We also compared the sensitivity for detecting glaucoma by standardized qualitative grading of stereophotographs. Photographs were graded for 40 of the 41 glaucomatous eyes; 1 photograph was of poor quality and could not be graded. Glaucoma was detected in 32 of the 40 eyes, for a sensitivity of 80%. At a specificity of 90% or greater, statistically significant differences were found between the sensitivity of photography and parameters with a sensitivity of 55% or lower (P<.001). Specificity was not reported for stereophotography because normal appearance of the optic disc, based on clinical examination, was part of the inclusion criteria for normal subjects.

This study is unique because it compares the discriminating ability of 3 instruments (HRT, GDx, and OCT) and stereophotography in 1 study population. Our results did not find significant differences between the ability to detect early to moderate glaucomatous visual field loss among the best parameters of the 3 instruments as measured by the area under the ROC curve. However, fewer GDx parameters had areas under the ROC curve of 0.81 or greater than did parameters from the OCT or HRT. At a fixed specificity of at least 90%, GDx number (sensitivity = 41%) was found to be significantly lower than the best HRT and OCT individual parameters, OCT thickness at the 5-o'clock position(sensitivity = 71%), and HRT mean height contour in the nasal inferior region(sensitivity = 63%). The 3 instruments did not correctly identify the same glaucomatous eyes, suggesting that each instrument may be measuring different characteristics. Qualitative evaluation of stereophotographs had a high sensitivity(80%) for detecting glaucoma.

These results confirm other investigators' results^{9,10,12,13,15,22,27-30} that report considerable overlap between HRT, GDx, and OCT measurements in normal and glaucomatous eyes. In part, this is due to the large range in the number of nerve fibers and axons across the normal population (660 000-1.5 million^31-33)and the variability in the appearance of the healthy eye. This limits the ability of any instrument to differentiate perfectly between normal and glaucomatous eyes.

Despite limitations, these instruments provide information that can help differentiate between healthy and glaucomatous eyes. Several investigators have provided estimates of the sensitivity and specificity for detecting glaucoma with imaging instruments.^{9,10,12,13,15,19,20,22,27,30,34-37}

Using the HRT classification developed by Mikelberg et al, sensitivity and specificity estimates range from 74% and 88%³⁸ to 87% and 84%.¹⁰ In the present study, we reported lower sensitivity and specificity values of 42% and 90% for the same parameter. When specificity is relaxed to 85%, sensitivity increases to 56%, but it is still lower than that reported by Mikelberg et al¹⁰ and Iester et al.³⁸

Using parameters available from the Nerve Fiber Analyzer 1 scanning laser polarimeter, Tjon-Fo-Sang et al²² reported sensitivity and specificity values of 96% and 93%. These values may be high because of the inclusion of patients with advanced glaucoma likely increasing the difference in RNFL measurements between patients and normal subjects. More recently, sensitivity and specificity estimates using the GDx have ranged from 64% sensitivity and 77% specificity using the GDx number, to 74% sensitivity and 92% specificity using a linear discriminant function.¹⁵ Using the same discriminant function in a similar population, our estimates of sensitivity and specificity were 49% and 92%. The GDx and HRT linear discriminant functions were developed on subjects from our center with age, ethnic, and glaucoma severity characteristics similar to the current study. Therefore, these linear discriminant functions may perform better in our population than in others.

Using OCT parameters, estimates of sensitivity and specificity in the current study were better than those reported for detecting focal defects: 65% and 85%, respectively.¹ For comparison at a similar level of specificity, our best estimates using the inferior temporal thickness at the 5-o'clock position measurement were higher, with a sensitivity of 76% and a specificity of 86%. Differences between these values may be attributed to the possibility that detecting focal defects is more difficult than detecting glaucoma because of the limited number of data points available with the OCT.

Although information is limited, our stereophotography results have confirmed those of previous reports that qualitative assessment of photographs can detect glaucoma at least as well as quantitative techniques.^39,40 O'Connor et al³⁹ compared qualitative observer assessment of stereoscopic color disc and monochromatic nerve fiber layer photographs with quantitative planimetric measurements of the disc rim area and nerve fiber layer height. They found that the total proportion of correct diagnoses for the presence or absence of visual field loss was greatest for the qualitative evaluations of the optic disc (82%) and nerve fiber layer (74%). Nakla et al⁴⁰ found that qualitative assessment of stereophotographs could discriminate between normal and glaucomatous eyes as well as HRT and OCT measurements.

There are several possible explanations for the discrepancies in the literature in estimates of sensitivity and specificity and ROC areas. Differences in reported estimates of the discriminating ability between instruments may be due to differences in the characteristics and severity of glaucoma in the study population. Not all of the variation in sensitivities and specificities can, however, be attributed to differences in the severity of glaucoma in the patients studied. Estimates of the area under the ROC curve for the HRT parameter cup shape were higher in a study conducted by Uchida et al,³⁰ which included patients with early to moderate glaucoma, than in one by Iester et al³⁶ that included some patients with advanced glaucoma (0.93 and 0.81, respectively). In the present study of patients with early to moderate visual field damage, ROC area for this parameter was 0.78. Evidently, differences among studies cannot be attributed to severity of glaucoma alone. Other characteristics such as the pattern of glaucoma damage, age, and race, as well as differences in image acquisition and processing, are likely to influence comparison across studies. Broadway et al³⁷ reported that detection of glaucoma varied by pattern of damage; the HRT classification was better at detecting focal ischemic discs (sensitivity = 93%) than senile sclerotic discs(sensitivity = 67%). By comparing instruments in the same study population, observed differences are not confounded by differences between study populations.

Limitations of this study include its relatively small sample size and differences in mean age between the normal and glaucoma groups. Age differences were accounted for by including age as a variable in the models. We also found similar results when separately evaluating the areas under the ROC curve in participants younger and older than 60 years. Because there is no evidence that one instrument has superior discriminating ability in one age group over another, the difference in age is likely to affect the discriminating ability of each instrument in a similar manner. Despite the relatively small sample size, significant differences in mean parameter measurements between study groups were found. However, our inability to find significant differences in the area under the ROC curve for the best parameters of each method may be partly due to the sample size of this study.

Longitudinal studies are under way to determine whether these instruments can improve our ability to detect glaucoma and monitor its progression. Detection of subtle changes in optic disc topography or the RNFL may also be important for assessing structural end points in clinical trials of neuroprotective therapies for treating glaucoma.^41,42

In conclusion, qualitative assessment of stereophotographs and measurements from the OCT and HRT generally had greater sensitivities than measurements from the GDx. However, the area under the ROC curve was similar among the best parameters from each instrument. This illustrates the importance of considering area under the ROC curve in relationship to various levels of sensitivity and specificity. At particular levels of specificity, certain instruments or parameters may perform better than at other levels.

Accepted for publication December 1, 2000.

This study was supported in part by grant EY11008 from the National Institutes of Health (Dr Zangwill), Bethesda, Md, and the Foundation for Eye Research (Drs Blumenthal and Vasile), Rancho Santa Fe, Calif.

Presented in part at the Annual Association for Research in Vision and Ophthalmology Meeting, Ft Lauderdale, Fla, May 14, 1999.

Corresponding author and reprints: Linda M. Zangwill, PhD, Glaucoma Center, Department of Ophthalmology, University of California–San Diego, La Jolla, CA 92093-0946.