[Skip to Navigation]
Sign In
Figure 1.  Bland-Altman Plots of the Differences vs the Means of the Automated Central Subfield Thickness (CST) Test-Retest Values
Bland-Altman Plots of the Differences vs the Means of the Automated Central Subfield Thickness (CST) Test-Retest Values

A, Stratus results within the Cirrus/Stratus group (n = 531). B, Cirrus results within the Cirrus/Stratus group (n = 531). C, Stratus results within the Spectralis/Stratus primary cohort (n = 429). D, Spectralis results within the Spectralis/Stratus primary cohort (n = 429). E, Stratus results within the Spectralis/Stratus normative cohort (n = 288). F, Spectralis results within the Spectralis/Stratus normative cohort (n = 288). Solid reference lines indicate mean difference; dashed lines indicate the limits of agreement.

Figure 2.  Bland-Altman Plots of the Machine Differences (Spectral Domain Minus Stratus) vs the Automated Stratus Test-Retest Means
Bland-Altman Plots of the Machine Differences (Spectral Domain Minus Stratus) vs the Automated Stratus Test-Retest Means

A, Central subfield thickness (CST) results within the Cirrus/Stratus group (n = 479). B, CST results within the Spectralis/Stratus group (n = 689). C, Retinal volume results within the Cirrus/Stratus group (n = 420). D, Retinal volume results within the Spectralis/Stratus group (n = 632). Solid reference lines indicate mean difference; dashed lines indicate the limits of agreement.

Figure 3.  Bland-Altman Plots of the Relative (%) Differences Between the Predicted Stratus Value (Using the Conversion Equation) Minus the Observed Stratus Value
Bland-Altman Plots of the Relative (%) Differences Between the Predicted Stratus Value (Using the Conversion Equation) Minus the Observed Stratus Value

A, Central subfield thickness (CST) results within the Cirrus/Stratus group (n = 272). B, CST results within the Spectralis/Stratus group (n = 375). C, Retinal volume results within the Cirrus/Stratus group (n = 253). D, Retinal volume results within the Spectralis/Stratus group (n = 381). Solid reference lines indicate mean difference; dashed lines indicate the limits of agreement.

Table 1.  Conversion Equation Data for Stratus Compared With Cirrus and Spectralis
Conversion Equation Data for Stratus Compared With Cirrus and Spectralis
Table 2.  CST Reproducibility Data
CST Reproducibility Data
1.
Diabetic Retinopathy Clinical Research Network.  Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema.  Ophthalmology. 2010;117(6):1064-1077.e35. doi:10.1016/j.ophtha.2010.02.031. PubMedGoogle ScholarCrossref
2.
Diabetic Retinopathy Clinical Research Network; Googe  J, Brucker  AJ, Bressler  NM,  et al.  Randomized trial evaluating short-term effects of intravitreal ranibizumab or triamcinolone acetonide on macular edema after focal/grid laser for diabetic macular edema in eyes also receiving panretinal photocoagulation.  Retina. 2011;31(6):1009-1027.PubMedGoogle ScholarCrossref
3.
Chalam  KV, Bressler  SB, Edwards  AR,  et al; Diabetic Retinopathy Clinical Research Network.  Retinal thickness in people with diabetes and minimal or no diabetic retinopathy: Heidelberg Spectralis optical coherence tomography.  Invest Ophthalmol Vis Sci. 2012;53(13):8154-8161.PubMedGoogle ScholarCrossref
4.
Bland  JM, Altman  DG.  Measuring agreement in method comparison studies.  Stat Methods Med Res. 1999;8(2):135-160.PubMedGoogle ScholarCrossref
5.
Diabetic Retinopathy Clinical Research Network; Krzystolik  MG, Strauber  SF, Aiello  LP,  et al.  Reproducibility of macular thickness and volume using Zeiss optical coherence tomography in patients with diabetic macular edema.  Ophthalmology. 2007;114(8):1520-1525.PubMedGoogle ScholarCrossref
6.
Forooghian  F, Cukras  C, Meyerle  CB, Chew  EY, Wong  WT.  Evaluation of time domain and spectral domain optical coherence tomography in the measurement of diabetic macular edema.  Invest Ophthalmol Vis Sci. 2008;49(10):4290-4296.PubMedGoogle ScholarCrossref
7.
Comyn  O, Heng  LZ, Ikeji  F,  et al.  Repeatability of Spectralis OCT measurements of macular thickness and volume in diabetic macular edema.  Invest Ophthalmol Vis Sci. 2012;53(12):7754-7759.PubMedGoogle ScholarCrossref
8.
Domalpally  A, Gangaputra  S, Peng  Q, Danis  RP.  Repeatability of retinal thickness measurements between spectral-domain and time-domain optical coherence tomography images in macular disease.  Ophthalmic Surg Lasers Imaging. 2010;41(suppl):S34-S41.PubMedGoogle ScholarCrossref
9.
Leung  CK, Cheung  CY, Weinreb  RN,  et al.  Comparison of macular thickness measurements between time domain and spectral domain optical coherence tomography.  Invest Ophthalmol Vis Sci. 2008;49(11):4893-4897.PubMedGoogle ScholarCrossref
10.
Sohn  EH, Chen  JJ, Lee  K, Niemeijer  M, Sonka  M, Abràmoff  MD.  Reproducibility of diabetic macular edema estimates from SD-OCT is affected by the choice of image analysis algorithm.  Invest Ophthalmol Vis Sci. 2013;54(6):4184-4188.PubMedGoogle ScholarCrossref
11.
Abedi  G, Patal  P, Doros  G, Subramanian  ML.  Transitioning from Stratus OCT to Cirrus OCT: a comparison and a proposed equation to convert central subfield macular thickness measurements in healthy subjects.  Graefes Arch Clin Exp Ophthalmol. 2011;249(9):1353-1357.PubMedGoogle ScholarCrossref
12.
Krebs  I, Hagen  S, Smretschnig  E, Womastek  I, Brannath  W, Binder  S.  Conversion of Stratus optical coherence tomography (OCT) retinal thickness to Cirrus OCT values in age-related macular degeneration.  Br J Ophthalmol. 2011;95(11):1552-1554.PubMedGoogle ScholarCrossref
13.
Ibrahim  MA, Sepah  YJ, Symons  RC,  et al.  Spectral- and time-domain optical coherence tomography measurements of macular thickness in normal eyes and in eyes with diabetic macular edema.  Eye (Lond). 2012;26(3):454-462.PubMedGoogle ScholarCrossref
Original Investigation
September 2014

Reproducibility of Spectral-Domain Optical Coherence Tomography Retinal Thickness Measurements and Conversion to Equivalent Time-Domain Metrics in Diabetic Macular Edema

Diabetic Retinopathy Clinical Research Network Writing Committee
JAMA Ophthalmol. 2014;132(9):1113-1122. doi:10.1001/jamaophthalmol.2014.1698
Abstract

Importance  Understanding measurement variability and relationships between measurements obtained on different optical coherence tomography (OCT) machines is critical for clinical trials and clinical settings.

Objective  To evaluate the reproducibility of retinal thickness measurements from OCT images obtained by time-domain (TD) (Stratus; Carl Zeiss Meditec) and spectral-domain (SD) (Cirrus; Carl Zeiss Meditec, and Spectralis; Heidelberg Engineering) instruments and formulate equations to convert retinal thickness measurements from SD-OCT to equivalent values on TD-OCT.

Design, Setting, and Participants  A cross-sectional observational study was conducted in private and institutional practices. Persons with diabetes mellitus who had at least 1 eye with central-involved diabetic macular edema, defined as Stratus central subfield thickness (CST) of 250 μm or greater, participated. An additional normative cohort (individuals with diabetes but without diabetic macular edema) was enrolled. Each study eye underwent 2 replicate Stratus scans followed by 2 replicate Cirrus or Spectralis scans (real-time image registration used) centered on the fovea.

Main Outcomes and Measures  Optical coherence tomography CST and macular volume.

Results  The Bland-Altman coefficient of repeatability for relative change in CST (the degree of change that could be expected from measurement variability) was lower with Spectralis (7%) compared with Cirrus (14%) and Stratus (12% and 15% within Cirrus/Stratus and Spectralis/Stratus groups, respectively). For each cohort, the initial Stratus CST was within 10% of the replicate Stratus measurement nearly all of the time; the conversion equations predicted a Stratus CST within 10% of the observed thickness 86% and 89% of the time for Cirrus/Stratus and Spectralis/Stratus groups, respectively, which is similar to the agreement on Stratus test-retest. The Bland-Altman limits of agreement for relative change in CST between machines (the degree of change that could be expected from measurement variability [combining within and between instrument variability]) were 21% for Cirrus and 19% for Spectralis when comparing predicted vs actual Stratus measurement.

Conclusions and Relevance  Reproducibility appears to be better with Spectralis than with Cirrus and Stratus. Conversion equations to transform Cirrus or Spectralis measurements to Stratus-equivalent values, within 10% of the observed Stratus thickness values, appear feasible. Central subfield thickness changes beyond 10% when using the same machine or 20% when switching machines, after conversion to Stratus equivalents, are likely due to a change in retinal thickness rather than measurement error.

Advances in retinal imaging have led to the development of multiple optical coherence tomography (OCT) instruments that incorporate spectral-domain (SD) technology, which addresses limitations that were imposed by time-domain (TD) technology. High-speed A scan acquisition available with SD-OCT yields higher B scan resolution images, reduces motion artifact, and provides time-efficient scan sampling density. The SD-OCT instruments also provide registration of images obtained in the same eye from different encounters. This point-to-point direct comparison between scans, performed in real time or by postimage acquisition warping, provides a more efficient means to reproducibly evaluate retinal change over time.

The management of diabetic macular edema (DME) frequently involves measurements from OCT devices. Stratus (Carl Zeiss Meditec) OCT was used widely in clinical practice and was the principal instrument used in numerous studies conducted by the Diabetic Retinopathy Clinical Research Network (DRCR.net) until 2011.1,2 Advantages of using SD-OCT instruments have led to their rapid adoption, and the Cirrus (Carl Zeiss Meditec) and Spectralis (Heidelberg Engineering) devices have often replaced Stratus instruments in DRCR.net clinical centers.

The DRCR.net uses macular retinal thickness measurements to guide eligibility for participation, to apply retreatment criteria, and to serve as an outcome measure of DME in clinical trials. The DRCR.net clinics presently have a variety of OCT instruments, each determining retinal thickness based on different locations for the outer retinal boundary. Measured retinal thickness varies among machines and is typically greater when images are obtained with SD-OCT compared with TD-OCT.

An objective of the present study was to compare macular thickness measurements between Stratus and Cirrus and between Stratus and Spectralis with the intent to develop and assess conversion equations that translate thickness measurements from these SD-OCT machines into a standardized and comparable TD-OCT Stratus value. Valid conversion equations would enable trials to pool data from these 3 instruments and compare results within and between protocols across groups that are using these instruments. An additional objective of this study was to compare the reproducibility of thickness measurements from Stratus, Cirrus, and Spectralis using their respective software segmentation analysis algorithms.

Methods

This DRCR.net observational study was conducted at 31 clinical sites. The protocol and Health Insurance Portability and Accountability Act–compliant informed consent forms (or the ability to obtain verbal consent) were approved by institutional review boards. Study participants gave written informed consent for study participation and received one $50 gas card for travel expenses. The protocol is available at http://www.drcr.net.

Study Population

Eligible individuals were at least aged 18 years and had type 1 or type 2 diabetes mellitus with media clarity adequate to obtain OCT images. Cohort 1 (primary cohort) enrolled both eyes of participants with DME in at least 1 eye (Stratus central subfield thickness [CST] ≥250 µm). Cohort 2 (normative cohort) enrolled participants with at least 1 eye meeting the normative criteria detailed in a separate publication3 in which the objective was to collate a normal macular thickness database specific to the Spectralis instrument. Both eyes were enrolled if each eye met the criteria; otherwise, a single eye was enrolled. The primary cohort enrollment goal was 75 eyes in each of 4 Stratus CST strata (<250 µm, 250-300 µm, 301-450 µm, and >450 µm) within each SD/TD OCT group (the Cirrus/Stratus group and the Spectralis/Stratus group).

Study Procedures

For each study eye (following pupil dilation), a single certified operator was to obtain 2 replicate TD-OCT Stratus scans followed by 2 replicate Cirrus or Spectralis SD-OCT scans. Stratus scans consisted of the fast macular thickness scan (six 6.0-mm radial 128 A scans/B scan) analyzed with Stratus software, version 4.0 or higher. Cirrus Macular Cube 512 × 128 protocol (6 × 6 mm) scans were analyzed with Cirrus software, version 3 or higher. Spectralis volume scans were acquired with 49 high-speed B scans (512 A scans/B scan) covering 20° × 20° (approximately 6 × 6 mm) with an automatic real-time mean of 16 and analyzed with Spectralis software, version 5.1 or higher. For images obtained in version 5.1, which limits analysis to the region within which the data points were acquired, the observed retinal volume value was converted to volume as determined by version 5.3a, which uses the entire 6-mm Early Treatment Diabetic Retinopathy Study grid, using the formula footnoted in Table 1. The first Spectralis scan was set as a reference such that the second scan underwent real-time image registration.

Statistical Analysis

Central subfield thickness and macular volume were the primary variables used in both the conversion and reproducibility analyses. The scans evaluated by the reading center and the values used for each analysis are summarized in eTable 1 in the Supplement.

Reproducibility for each variable was evaluated separately for each OCT machine, within each of the SD/TD groups, and separately for the normative cohort. The relationships of differences between the test-retest scans (scan 1−scan 2) of each machine were explored using Bland-Altman methods. Computations of the Bland-Altman coefficient of repeatability (CR) and the 95% CI used the standard method 1.96 × √2 × √MSE, where MSE is the mean squared error from repeated-measures regression models, with the dependent variable being the given measurement and the independent variables being the participant and the eye nested within the participant. The computations were performed on both the original outcome measurement scale as well as on the relative difference scale.4 The reproducibility of each type of thickness measurement was compared between machines using linear mixed models that evaluated the relative absolute differences (RADs) as the dependent variable and the machine as the independent variable and accounted for the correlation within participants (between study eyes and machines).

Conversion equation analyses for CST and macular volume were evaluated within each of the SD/TD groups (including the normative cohort). The conversion equations were derived using data from a random sample of half of the participants in the available SD/TD groups for each of the 4 equations; the second half of each sample was used to validate the formulas. Each of the 4 conversion equations was built using repeated-measures models with generalized estimating equations to account for the correlation in participants with 2 study eyes, with the Stratus measurement as the dependent variable and SD measurement as the independent variable. Transformations, including log and log-log, were explored for each model; ultimately, the Cirrus to Stratus volume equation was the only analysis in which a transformation was used to improve the model. The Cirrus machine calculates macular volume based on the entire 6 × 6-mm2 area; for this analysis, Cirrus volume was recalculated (transformed), limiting it to the 6-mm-diameter circle using a weighted mean of the 9 subfields, as footnoted in Table 1. The validity of each equation was evaluated by comparing predicted vs observed metrics computed from the second half of the data that were not used to build the model.

The applicability of conversion equations for making clinically relevant decisions on the individual patient level was evaluated via the Bland-Altman limits of agreement and the 95% CI on the relative differences between the observed and predicted automated Stratus values using the validation half sample of scan 1 values, computed using linear mixed models.

All reported P values were 2-sided and unadjusted for multiple testing. In view of the number of analyses, P < .01 values were the only ones considered to be unlikely due to chance. Analyses were conducted with SAS, version 9.3 (SAS Institute Inc).

Results

The Cirrus/Stratus group included 540 eyes from 292 participants, and the Spectralis/Stratus group enrolled 758 eyes from 400 participants (468 eyes [242 participants] from the primary cohort and 290 eyes [158 participants] from the normative cohort). Ninety-four eyes were excluded because the investigator identified an ocular abnormality that could affect the OCT image. No participant was in both SD/TD groups.

The participant and ocular characteristics of individuals in each SD/TD group are described in eTable 2 in the Supplement. The Stratus median CST was 290 µm in the Cirrus/Stratus group and 239 µm in the Spectralis/Stratus group (including the normative cohort).

Replicate scans on both the SD and TD machines were completed in 531 (98.3%) and 717 (94.6%) eyes in the Cirrus/Stratus and Spectralis/Stratus groups, respectively. In each SD/TD group, the same OCT operator obtained nearly all (96%-98%) scan pairs for a given individual.

Reproducibility

Reproducibility of the 2 metrics was evaluated within each OCT instrument and within each group and cohort separately (CST in Figure 1 and Table 2 and volume in the eFigure and eTable 3 in the Supplement). The CR and RAD were smaller for Spectralis (CR, 7%; RAD, 1%) than for either Cirrus (CR, 14%; RAD, 1%) or Stratus (CR, 12%/15%; RAD, 2%/2% in the Cirrus/Stratus and Spectralis/Stratus groups, respectively) (P < .001). Although the CR was greater with Cirrus compared with Stratus, the RAD in Cirrus was less than that of Stratus (P = .008). Figure 1 reflects this graphically, with a tighter cluster of CST differences around zero on Cirrus compared with Stratus yet more observations falling far from the mean. Within all machines, the RAD showed less variation than the differences on the absolute scale. For macular volume, the reproducibility for the Spectralis was better than that for the Cirrus or Stratus.

Reproducibility of the measurements is in part operator dependent, as evident in a secondary analysis restricted to the technicians who performed test-retest scans on at least 20 eyes within each SD/TD group. For the Cirrus/Stratus group, the individual technicians’ (n = 7) CST CRs ranged from 8 to 95 µm on Cirrus and from 25 to 54 µm on Stratus. For the primary cohort in the Spectralis/Stratus group, the individual technicians’ (n = 4) CST CRs ranged from 8 to 22 µm on Spectralis and from 28 to 48 µm on Stratus.

Conversion Equation

eTable 1 in the Supplement summarizes the values used for this analysis. In the Cirrus/Stratus group, there were 479 and 420 eyes for the CST and volume analyses, respectively, and there were 689 and 632 eyes in the Spectralis/Stratus group for the CST and volume analyses, respectively.

The median (5th and 95th percentile) difference in CST measurement between Cirrus and Stratus was +43 µm (5 µm and 70 µm) and +67 µm (29 µm and 86 µm) between Spectralis and Stratus (Table 1). Likewise, the difference in the macular volume measurement between Cirrus and Stratus was +1.1 mm3 (0.5 mm3 and 1.5 mm3) and 1.6 mm3 (1.1 mm3 and 1.9 mm3) between Spectralis and Stratus (Table 1). Conversion equations to translate Cirrus and Spectralis CST or macular volume into Stratus equivalents (Table 1) were derived from half samples of the full data sets depicted in the Bland-Altman plots in Figure 2.

The models used to develop the conversion equations were validated by calculating the predicted Stratus thickness and volume for each machine from the observed SD CST and macular volume measurements obtained in the remaining half of the data set and comparing this predicted value with the actual observed Stratus value (eTable 4 in the Supplement). The predicted CST values fell within 10% of the observed Stratus measurement in 86% of the Cirrus/Stratus pairs and 89% of the Spectralis/Stratus pairs. When the calculated group mean CST was compared with the observed group mean, the difference was 1 µm or less. When the database was used to categorize eyes by normal thickness (Stratus OCT CST of <250 µm), a discordance rate of less than 10% was identified when assigning eyes by the predicted compared with the observed values, without any favored directionality to the discordance. By comparison, the same statistics computed using Stratus test vs retest values demonstrated similar or only slightly better agreement (eTable 4 in the Supplement). Validation for the volume equations yielded even greater agreement between the predicted and observed values for each of these analyses.

We applied the conversion equations to automated SD values from the validation half-sample data set to calculate converted Stratus values; the relative differences between the observed and predicted values are depicted in Bland-Altman plots in Figure 3. Limits of agreement on the relative difference scale for Cirrus/Stratus and Spectralis/Stratus CST were 21% (95% CI, 19%-22%) and 19% (95% CI, 17%-20%), respectively; for volume, the limits of agreement were 9% (95% CI, 8%-10%) and 9% (95% CI, 9%-10%), respectively.

Discussion

Clinical practice and clinical research rely heavily on OCT images to detect the presence and severity of DME. Technologic advancements have led to rapid adoption of SD-OCT instruments, which provide high image resolution, dense image sampling, and the ability to register images between visits. Clinical research studies engaged in analyzing OCT data have had to rapidly evaluate these instruments to establish normative databases, assess reproducibility in the measurements, and develop means of integrating data from a variety of instruments within a trial.

To determine whether changes in software-generated retinal thickness measurements are more likely to indicate real changes in thickness status rather than change resulting from measurement variability (measurement error), it is essential to evaluate retinal thickness measurement reproducibility. The DRCR.net accepts up to a 10% change in CST between visits, as measured by Stratus at each time point, as within-measurement error based on a previous DRCR.net study.5 In that study, following reading center review of all images (and adjustment of values when indicated), the CR for absolute and relative change in CST was 38 µm and 11%, respectively. The corresponding data for macular volume were 0.27 mm3 and 3%. The present study confirms the previous observations of DRCR.net while using a much larger number of eyes with broad representation of macular thickness values, a different group of clinical centers and OCT operators, and analysis limited to the raw software-generated metrics. The last design feature was meant to provide the CR that would be most applicable to clinical practice where measurements are not reviewed or corrected by a reading center.

Based on the reproducibility data generated with the Cirrus and Spectralis instruments in the present study, DRCR.net requires at least a 10% change in CST between visits as the amount of change necessary to indicate an actual change in macular thickness when the measurements are made consistently with these machines. Other investigators6-10 have reported smaller values for CR; however, these studies involved a more limited number of participants, some of whom had noncentral DME or normal macular architecture, and very few OCT technicians. In addition, these studies frequently excluded scans with imperfect quality (particularly of eyes with segmentation errors), thereby limiting the variability introduced by the population, the spectrum of operators, and the machine. The present study also confirms an earlier observation5 made when evaluating measurement reproducibility with the Stratus instrument and extends this observation to Cirrus and Spectralis. The earlier observation indicated that the absolute value for measurement reproducibility varies according to the absolute thickness value being evaluated, whereas assessments made as relative change are more constant over a range of thickness values.

In the present study, the test-retest differences in CST measurements were generally smaller for both SD instruments than for Stratus, and Spectralis was superior to Cirrus. Although Cirrus had smaller median absolute and relative differences between scans 1 and 2 in comparison with Stratus, there were also more eyes with large discrepancies, leading to a CR that was larger than with Stratus. Although no significant difference in reproducibility was identified between Stratus and Cirrus for the volume measurement, Spectralis showed significantly less test-retest variation in this measurement compared with either Stratus or Cirrus. Operator technique could be a source of variability in replicate measurements; however, this was minimized by having the same operator perform nearly all of the SD/TD scan pairs in the present study. These reproducibility findings, which favor the Spectralis instrument, are more likely the result of the real-time image registration function that was available and used only with the Spectralis instrument. We did not test the post hoc image registration feature available in the Cirrus instrument software; had we used this feature, the measurement variability between the 2 replicate Cirrus scans may have been reduced. However, this step would have required an investigator to review every Cirrus scan pair for post hoc registration quality and perform manual registration when the software failed to accomplish the task. These methods were not used because they have not been adopted within DRCR.net studies, and our goal was to report measurement variability replicating the methods used by the network in our trials.

As anticipated, Spectralis and Cirrus generated higher retinal thickness values relative to a TD-OCT machine because the SD segmentation algorithms measure retinal thickness starting from more posterior structures, such as the retinal pigment epithelium apical surface or the Bruch’s membrane, compared with TD algorithms that measure retinal thickness from the ellipsoid zone to the internal limiting membrane. The median difference between Stratus and Cirrus CST was 43 µm and the median difference between Stratus and Spectralis CST was 67 µm. Given the inherent differences in how the instruments measure retinal thickness, it would be incorrect to simply pool the software-generated raw data to describe populations that are measured with different instruments. One potential solution to this challenge would be customized software that automatically identifies a common outer retinal boundary in images from all machines from which thickness values are calculated. Some reading centers have customized software for this purpose, but trials dependent on this solution must accept the time and costs associated with a reading center providing thickness measurements. It is also unlikely that this solution would be available in clinical practice to assist in clinical judgments when patients’ eyes are scanned with different machines at different time points.

An alternative solution to this problem of variability is development of conversion equations between TD and SD factors. Other studies11-13 have created equations to convert Stratus CST values to either Cirrus or Spectralis CST values based on cohorts of either healthy individuals or eyes with age-related macular degeneration. In the present study, conversion equations were derived to allow conversion of SD metrics to a common TD or “Stratus language” for studies that permit a variety of OCT machines. This process avoids analysis and reporting of trial data within different OCT machine subgroups. These equations were derived independently for Cirrus and Spectralis machines, and they transform SD CST and macular volume values into Stratus “equivalent” measurements. Strengths of this analysis include the comprehensive approach taken to obtain the SD and TD measurements, the validation exercise performed on the predicted TD values, and the emphasis on a patient population with diabetes selected for the purposes of generalizing the results to future cohorts with diabetic retinopathy and DME. The findings of the validation exercise demonstrate relatively little difference between the Stratus values predicted based on SD observations and the actual observed Stratus values compared with the differences observed between test and retest values on the Stratus machine. The conversion equations for both CST and volume are satisfactory to transform values from Cirrus and Spectralis to equivalent Stratus values to report cross-sectional population thickness and volume measurements on the same scale across all patients. The DRCR.net network is also using these equations in circumstances in which participants’ eyes are imaged with Stratus at baseline and switched to SD-OCT imaging during follow-up. The SD-OCT metrics are converted to Stratus-equivalent values to allow changes between visits to be evaluated on the same scale. Although a variety of OCT instruments are permitted for data collection, investigators are encouraged to consistently use the same type of OCT machine within individuals over the course of a study to minimize the need for conversion and the increase in measurement variability introduced by this need.

For the purpose of making clinical decisions at an individual patient level when measurements from one machine earlier in a patient’s course are compared with subsequent measurements obtained on a different machine, additional variability must be considered by the change in instrumentation (ie, beyond the approximate 10% threshold for within-machine measurement error alone). A difference between 2 measurements (after conversion of the SD value to the Stratus scale) increases the threshold to approximately 20% because the error is associated with both the within-machine measurement error and the variability incurred by changing instrumentation. Therefore, at the individual level, changes of less than 10% within one machine or less than 20% between multiple machines on the Stratus scale might be real or they might be within the measurement error, which creates a potential challenge for clinical decision making. In these situations, monitoring changes in values over more than 2 time points may provide greater certainty as to whether changes of smaller magnitude from one visit to another are real or if they represent measurement variability. Qualitative clinical observations of changes in macular anatomy also may help in the decision as to whether the observed changes in measured thickness are clinically important.

A limitation of this study was the absence of Cirrus and Spectralis OCT scans on study eyes from the same encounter. As such, it was not possible to develop conversion equations to translate metrics directly between the Cirrus and Spectralis instruments. In the future, when Stratus becomes increasingly less common, an SD equivalent may become the standard reference value. Alternatively, the manufacturers of OCT machines might be persuaded to offer a common outer retinal boundary line for software computations. The conversion equations developed provide a solution to combine data derived from a variety of commonly available OCT machines to report cross-sectional observations and to evaluate longitudinal changes when individual patients’ eyes are imaged initially with TD-OCT and then switched to specific SD-OCT instruments within clinical trials.

Conclusions

The management of DME frequently involves measurements from OCT devices. Advancements in SD-OCT technology require an understanding of the variability of the measurements and the relationship between retinal thickness measurements obtained on different machines. Reproducibility of central macular thickness and macular volume appears better on Spectralis than Cirrus and Stratus. Conversion equations to transform Cirrus or Spectralis measurements to Stratus-equivalent values, within 10% of the observed Stratus thickness values, appear feasible. Central subfield thickness changes beyond 10% when using the same machine or 20% when switching machines, after conversion to Stratus equivalents, are likely the result of a real change in retinal thickness rather than measurement error.

Back to top
Article Information

Group Information: A complete list of the members of the Diabetic Retinopathy Clinical Research Network (DRCR.net) who participated in this protocol are listed in the eAppendix in the Supplement.

Submitted for Publication: July 2, 2013; final revision received February 26, 2014; accepted March 3, 2014.

Corresponding Author: Allison R. Edwards, MS, Jaeb Center for Health Research, 15310 Amberly Dr, Ste 350, Tampa, FL 33647 (drcrstat1@jaeb.org).

Published Online: July 24, 2014. doi:10.1001/jamaophthalmol.2014.1698.

Authors/Writing Committee: The following members of the Diabetic Retinopathy Clinical Research Network take authorship responsibility for the study results: Susan B. Bressler, MD; Allison R. Edwards, MS; Kakarla V. Chalam, MD; Neil M. Bressler, MD; Adam R. Glassman, MS; Glenn J. Jaffe, MD; Michele Melia, ScM; David D. Saggau, MD; Oren Z. Plous, MD.

Affiliations of Authors/Writing Committee: Wilmer Eye Institute, the Johns Hopkins University School of Medicine, Baltimore, Maryland (S. B. Bressler, N. M. Bressler); Jaeb Center for Health Research, Tampa, Florida (Edwards, Glassman, Melia); Department of Ophthalmology, Jacksonville Health Science Center, University of Florida College of Medicine, Jacksonville (Chalam); Department of Ophthalmology, Duke University, Durham, North Carolina (Jaffe); Wolfe Eye Clinic, Des Moines, Iowa (Saggau); Gulf Coast Retina Center, Port Richey, Florida (Plous).

Author Contributions: Ms Edwards had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: S. B. Bressler, Chalam, N. M. Bressler, Saggau.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: S. B. Bressler, Edwards, Glassman, Saggau, Plous.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Edwards, Glassman, Melia, Saggau, Plous.

Obtained funding: N. M. Bressler, Glassman.

Administrative, technical, or material support: Chalam, N. M. Bressler, Jaffe.

Study supervision: S. B. Bressler, N. M. Bressler, Melia, Saggau.

Conflict of Interest Disclosures: A complete list of all DRCR.net financial disclosures is at http://www.drcr.net. Dr S. B. Bressler reports serving as a consultant to GlaxoSmithKline and receiving clinical or laboratory research grants from Bausch & Lomb, Bayer, Emmes, Boehringer-Ingelheim, Notal Vision, Novartis, Regeneron, Thrombogenics, and sanofi-aventis. Dr N. M. Bressler reports receiving clinical or laboratory research grants from Bayer, Genentech, and Regeneron; providing expert testimony for Novartis; and receiving funding from Emmes. Dr N. M. Bressler also reports that grants to investigators at The Johns Hopkins University are negotiated and administered by the institution (eg, the School of Medicine) that receives the grants, typically through the Office of Research Administration. Individual investigators who participate in the sponsored projects are not directly compensated by the sponsor but may receive salary or other support from the institution to support their effort on the projects. Dr N. M. Bressler is principal investigator of grants at The Johns Hopkins University sponsored by the following entities (not including the National Institutes of Health): Abbott Medical Optics, Allergan, Bausch & Lomb, Bristol-Meyers Squibb, Carl Zeiss Meditec, EMMES Corporation, ForSight Labs LLC, Genentech, Genzyme Corporation, Lumenis, Notal Vision, Novartis, and Regeneron. Dr Jaffe reports serving as a consultant to Alcon, Heidelberg Engineering, Neurotech, and Novartis. No other disclosures were reported.

Funding/Support: The present study was supported through a cooperative agreement from the National Eye Institute and the National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, US Department of Health and Human Services grants EY14231, EY023207, and EY018817.

Role of the Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr N. M. Bressler is the editor of JAMA Ophthalmology. He was not involved in the editorial evaluation or decision to accept this article for publication.

References
1.
Diabetic Retinopathy Clinical Research Network.  Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema.  Ophthalmology. 2010;117(6):1064-1077.e35. doi:10.1016/j.ophtha.2010.02.031. PubMedGoogle ScholarCrossref
2.
Diabetic Retinopathy Clinical Research Network; Googe  J, Brucker  AJ, Bressler  NM,  et al.  Randomized trial evaluating short-term effects of intravitreal ranibizumab or triamcinolone acetonide on macular edema after focal/grid laser for diabetic macular edema in eyes also receiving panretinal photocoagulation.  Retina. 2011;31(6):1009-1027.PubMedGoogle ScholarCrossref
3.
Chalam  KV, Bressler  SB, Edwards  AR,  et al; Diabetic Retinopathy Clinical Research Network.  Retinal thickness in people with diabetes and minimal or no diabetic retinopathy: Heidelberg Spectralis optical coherence tomography.  Invest Ophthalmol Vis Sci. 2012;53(13):8154-8161.PubMedGoogle ScholarCrossref
4.
Bland  JM, Altman  DG.  Measuring agreement in method comparison studies.  Stat Methods Med Res. 1999;8(2):135-160.PubMedGoogle ScholarCrossref
5.
Diabetic Retinopathy Clinical Research Network; Krzystolik  MG, Strauber  SF, Aiello  LP,  et al.  Reproducibility of macular thickness and volume using Zeiss optical coherence tomography in patients with diabetic macular edema.  Ophthalmology. 2007;114(8):1520-1525.PubMedGoogle ScholarCrossref
6.
Forooghian  F, Cukras  C, Meyerle  CB, Chew  EY, Wong  WT.  Evaluation of time domain and spectral domain optical coherence tomography in the measurement of diabetic macular edema.  Invest Ophthalmol Vis Sci. 2008;49(10):4290-4296.PubMedGoogle ScholarCrossref
7.
Comyn  O, Heng  LZ, Ikeji  F,  et al.  Repeatability of Spectralis OCT measurements of macular thickness and volume in diabetic macular edema.  Invest Ophthalmol Vis Sci. 2012;53(12):7754-7759.PubMedGoogle ScholarCrossref
8.
Domalpally  A, Gangaputra  S, Peng  Q, Danis  RP.  Repeatability of retinal thickness measurements between spectral-domain and time-domain optical coherence tomography images in macular disease.  Ophthalmic Surg Lasers Imaging. 2010;41(suppl):S34-S41.PubMedGoogle ScholarCrossref
9.
Leung  CK, Cheung  CY, Weinreb  RN,  et al.  Comparison of macular thickness measurements between time domain and spectral domain optical coherence tomography.  Invest Ophthalmol Vis Sci. 2008;49(11):4893-4897.PubMedGoogle ScholarCrossref
10.
Sohn  EH, Chen  JJ, Lee  K, Niemeijer  M, Sonka  M, Abràmoff  MD.  Reproducibility of diabetic macular edema estimates from SD-OCT is affected by the choice of image analysis algorithm.  Invest Ophthalmol Vis Sci. 2013;54(6):4184-4188.PubMedGoogle ScholarCrossref
11.
Abedi  G, Patal  P, Doros  G, Subramanian  ML.  Transitioning from Stratus OCT to Cirrus OCT: a comparison and a proposed equation to convert central subfield macular thickness measurements in healthy subjects.  Graefes Arch Clin Exp Ophthalmol. 2011;249(9):1353-1357.PubMedGoogle ScholarCrossref
12.
Krebs  I, Hagen  S, Smretschnig  E, Womastek  I, Brannath  W, Binder  S.  Conversion of Stratus optical coherence tomography (OCT) retinal thickness to Cirrus OCT values in age-related macular degeneration.  Br J Ophthalmol. 2011;95(11):1552-1554.PubMedGoogle ScholarCrossref
13.
Ibrahim  MA, Sepah  YJ, Symons  RC,  et al.  Spectral- and time-domain optical coherence tomography measurements of macular thickness in normal eyes and in eyes with diabetic macular edema.  Eye (Lond). 2012;26(3):454-462.PubMedGoogle ScholarCrossref
×