Kaplan-Meier curves for time to progression and visual acuity (VA) decline in involved eyes by trial and stage of retinitis. Progress was defined as lesion border progression of 750 µm or more or occurrence of a new lesion as assessed by the Fundus Photograph Reading Center. Visual acuity loss was defined as a decline of VA of 15 or more standard letters on an Early Treatment of Diabetic Retinopathy Study chart. A, Foscarnet-Ganciclovir CMV [cytomegalovirus] Retinitis Trial (FGCRT): progress, 251 events in 318 involved eyes of 224 patients; VA loss, 142 events in 302 involved eyes of 215 patients. B, CMV Retinitis Retreatment Trial (CRRT): progress, 266 events in 403 involved eyes of 256 patients; VA loss, 176 events in 392 involved eyes of 259 patients. C, Monoclonal Antibody CMV Retinitis Trial (MACRT), newly diagnosed: progress, 61 events in 100 involved eyes of 73 patients; VA loss, 28 events in 100 involved eyes of 74 patients. D, MACRT, newly diagnosed: progress, 114 events in 188 involved eyes of 120 patients; VA loss, 58 events in 180 involved eyes of 119 patients.
Change in visual field by change in retinal area with retinitis for involved eyes of patients enrolled by trial and stage of retinitis. A, Foscarnet-Ganciclovir CMV [cytomegalovirus] Retinitis Trial (FGCRT): dots represent 573 observations from 207 involved eyes of 152 patients with newly diagnosed retinitis at baseline. Line is estimated as the change in field (°) = 13.8° + (6.0°/% × change in area[%]). B, CMV Retinitis Retreatment Trial (CRRT): dots represent 465 observations from 238 involved eyes of 162 patients with relapsed retinitis at baseline. Line is estimated as the change in field (°) = 47.8° + (5.8°/% × change in area [%]). C, Monoclonal Antibody CMV Retinitis Trial (MACRT), newly diagnosed: dots represent 124 observations from 60 involved eyes of 47 patients with newly diagnosed retinitis. Line is estimated as the change in field (°) = 19.4° + (4.3°/% × change in area [%]). D, MACRT, relapsed: dots represent 225 observations from 112 involved eyes of 77 patients with relapsed retinitis at baseline. Line is estimated as the change in field (°) = 21.4° + (6.8°/% × change in area [%]).
Holbrook JT, Meinert CL, Van Natta ML, Davis M, Hubbard L, Jabs DA, for the Studies of Ocular Complications of AIDS Research Group. Photographic Measures of Cytomegalovirus Retinitis as Surrogates for Visual Outcomes in Treated Patients. Arch Ophthalmol. 2001;119(4):554-563. doi:10.1001/archopht.119.4.554
To evaluate photographic measures of cytomegalovirus (CMV) retinitis as surrogate outcomes for changes in vision in patients with CMV retinitis related to the acquired immunodefiency syndrome.
Data from 3 clinical trials of CMV retinitis treatments were analyzed. Two photographic assessments of retinitis in eyes involved at baseline were evaluated: progression (lesion border movement ≥750 µm or occurrence of a new lesion) and change in area of retina involved with retinitis. Vision measures were decline in best-corrected visual acuity and change in visual field. Photographic measures were evaluated as surrogate outcomes based on 4 criteria: (1) association with vision measure; (2) ability to account for treatment-related differences in vision measure; (3) data completeness; and(4) sample size requirements.
Data from 1001 involved eyes (666 patients) were analyzed. Progression and change in area involved were predictive of declines in vision measures, accounted for 50% and 66% of the treatment effect on visual field, and were available from 93% and 64% of involved eyes, respectively. Sample size estimates for a clinical trial were smallest with progression as the design outcome.
Progression and change in area involved met the first and second criteria for surrogate outcomes for visual field loss; a complete evaluation for visual acuity decline was not possible because treatment-related differences were not observed. Progression met the logistical and sample size criteria better than change in area of retina involved with retinitis.
PIVOTAL EFFICACY trials supporting the approval of treatments for cytomegalovirus(CMV) retinitis were designed to compare time to retinitis progression as assessed from photographs between patients randomly assigned to immediate treatment or to deferral of treatment.1- 6 Retinitis progression was defined as 750 µm or more of movement along the border of an existing lesion or the appearance of a new lesion in either eye. Assessments of progression made from photographs by graders unaware of treatment assignment have proven to be a more reliable and sensitive method than assessments made by clinicians based on ophthalmoscopic examinations.2,7,8 Movement of 750 µm or more is a sensitive measure of retinitis advancement selected to protect patients assigned to deferral of therapy from clinically important vision loss.
The goal of retinitis treatment is to preserve vision. Despite the widespread use of photographic assessments of retinitis progression as a primary outcome measure in clinical trials, it has not been evaluated as a surrogate outcome for change in vision. The purpose of the current study was to evaluate progression as well as an alternative retinitis measure assessed from photographs, change in area of retina involved with retinitis, as surrogate outcomes for declines in visual acuity and visual field.
Prentice and others have proposed criteria by which to evaluate proposed surrogate outcomes.9- 12 To evaluate the retinitis measures as surrogates for changes in vision, we operationalized those criteria as follows: (1) association to vision loss;(2) the degree to which the measure accounted for treatment-related differences in vision measures; (3) completeness of data; and (4) sample size requirements. The first 2 criteria evaluate the association between a surrogate and a clinical outcome in general and with respect to treatment-related changes in these measures, respectively. The final criteria are related to the feasibility and practicality of the surrogate outcome.
Data from 3 clinical trials conducted by the Studies of Ocular Complications of AIDS Research Group were analyzed (Foscarnet-Ganciclovir CMV Retinitis Trial [FGCRT], the CMV Retinitis Retreatment Trial [CRRT], and the Monoclonal Antibody CMV Retinitis Trial [MACRT]).8,13- 15 Patients with newly diagnosed retinitis or relapsed retinitis enrolled in these trials over a 7-year period (1990-1996).
The protocols for all 3 trials were reviewed and approved by institutional review boards at the coordinating center and at the clinical centers; all patients signed institutional review board–approved consent statements. Details on the study designs, procedures, and results are described elsewhere.8,13- 17
For these analyses, progression was defined as the movement of a border 750 µm or greater along a front 750 µm or more in length, or the occurrence of a new lesion 750 µm or greater in diameter and separated from a previous lesion by 750 µm or more, in a previously uninvolved eye. Fundus Photograph Reading Center (FPRC) graders unaware of treatment assignment assessed progression by comparison of photographs taken at baseline to those taken at follow-up visits.
The extent of retinitis was evaluated from fundus photographs by measuring the area of retina with retinitis in zones 1 and 2; zone 3 was not evaluated because it could not be photographed reliably. The definition of retinal zones has been described.18 Area of retina involved was expressed as a percentage (ie, area of retina in zones 1 and 2 with retinitis divided by total area of retina in zones 1 and 2). In the FGCRT, retinal area involved was determined by FPRC graders from planimetric measurements on a digitized mosaic of the retina created from fundus photographs.19 In the CRRT and the MACRT, FPRC graders evaluated the area of retina involved using grids superimposed on the photographs.
The vision measures were visual acuity as measured on logarithmic charts developed for the Early Treatment Diabetic Retinopathy Study20 and visual field as measured via kinetic perimetry along 12 meridians on a Goldmann Visual Field Test with a IV4e test object; the degrees of field seen along each meridian were summed to calculate the visual field score.21
Data collected at visits that occurred during the first year of follow-up before or after the treatment protocols were suspended were included in these analyses.
For most analyses, data were limited to eyes with CMV retinitis at baseline as determined from FPRC evaluation of photographs. Baseline characteristics of involved eyes were summarized from standardized evaluations of photographs by FPRC graders as previously described.20 For the MACRT, initial treatment for CMV retinitis was defined as the primary treatment recorded at the baseline visit. Time-dependent variables for the retinitis measures (progression status and change in retinal area involved with retinitis) were defined as the value at the same visit at which vision measures were evaluated or at the most recent prior visit. For progression, once the event occurred the value of the time-dependent variable did not change.
For event-type measures (progression and visual acuity decline), median times to event were estimated according to the Kaplan-Meier method and Cox proportional hazards models were used to estimate relative risks (RRs).22,23 For baseline characteristics, P values associated with RRs of events were derived using procedures for estimating variance of correlated data (2 eyes from 1 patient).24 No corrections for correlated data were made for models including time-dependent covariates. Continuous outcomes were modeled with linear regression using general estimating equation procedures, which accounted for correlations among repeated observations from an eye.25 Analyses of associations of retinitis measures to vision measures were performed by trial and stage of retinitis (newly diagnosed or relapsed) because of the large variability in event rates across trials. If appropriate, data were combined for overall estimates of the associations.
The capacity of a retinitis measure to account for the treatment effect on visual field was evaluated by comparing the linear regression coefficient for CMV treatment estimated from a model without the time-dependent retinitis measure with those from a model that included the measure.9,12 The deviance scores (−2 log likelihood statistics) were used to compare the predictive value of the retinitis measures for visual acuity decline and visual field decline.23,26 These analyses were performed on the subgroup of involved eyes for which data on all measures of retinitis were available.
Sample size estimates were made for event-type outcomes with log rank tests and for continuous outcomes with t test procedures.27,28 Because of secular trends in event rates and mean changes, event rates, means, and SDs were estimated based on MACRT data.
Two hundred thirty-four patients with newly diagnosed retinitis were enrolled in the FGCRT; 279 patients with relapsed retinitis were enrolled in the CRRT; 209 patients (83 patients with newly diagnosed retinitis and 126 with relapsed retinitis) were enrolled in the MACRT. The analyses presented herein included data from 88% to 95% of patients enrolled in these trials(ie, those patients who had baseline and follow-up photographic data available for at least 1 eye with CMV retinitis at baseline) (Table 1). Eight patients were enrolled in more than 1 of the trials; separate data were collected for each trial in which they were enrolled.
Patients enrolled in the 3 trials had similar demographic profiles with the exception that relatively more whites were enrolled in the FGCRT (Table 1). Relatively more patients enrolled in the most recent trial, the MACRT, received combinations of antiretrovirals, and those were the only patients who received protease inhibitors at baseline. Patients enrolled in the first 2 trials were assigned to an anti-CMV treatment. Patients enrolled in the last trial selected their primary anti-CMV treatment and were most likely to have received intravenous (IV) ganciclovir as their primary treatment at baseline.
Characteristics of involved eyes that were different among the trials after adjustment for stage of retinitis were zone 1 involvement, activity, and microangiopathy (hemorrhage/microaneurysms and cotton-wool spots) (Table 2). Involved eyes of patients with newly diagnosed retinitis had smaller, more active lesions and greater percentages with cotton-wool spots, vitreous hemorrhage, and retinitis involvement of the optic disc than involved eyes of patients with relapsed retinitis (Table 2).
Data to evaluate progression in involved eyes were available for 88% to 96% of involved eyes of patients enrolled in the 3 trials. Median times to progression differed among the 3 trials (Table 3) (P = .001) (Figure 1), but were similar for eyes of newly diagnosed and relapsed retinitis after adjustment for trial and initial CMV treatment (Table 3) (P = .19).
Initial treatment for CMV retinitis was not associated with risk of progression in the FGCRT (IV ganciclovir vs IV foscarnet), but was associated in the CRRT (P<.001) and MACRT (P = .007). In the CRRT, the median times to first progression while receiving treatment in involved eyes were 2.5, 2.0, and 5.1 months for the IV ganciclovir, IV foscarnet, and IV combination treatment groups, respectively. In the MACRT, the median times to first progression were 2.7, 6.2, and 5.8 months for patients receiving IV or oral systemic monotherapy, IV combination therapy, or local therapy, respectively.
Data to evaluate the percentage of retinal area involved with retinitis at baseline and during follow-up were available for 55% to 68% of involved eyes. The mean rates of change in retinal area involved with retinitis, expressed as a percentage of zones 1 and 2, for the first 6 months of follow-up differed among the trials (Table 3) (P<.001): the rates were 3.5%, 2.6%, and 1.4% of retinal area per month in the FGCRT, CRRT, and MACRT, respectively. Change in area during follow-up was similar for eyes of patients with newly diagnosed and relapsed retinitis (Table 3) (P = .81).
Change in the percentage of retinal area involved with retinitis was related to treatment for CMV retinitis in the MACRT (P<.001), and there was a suggestion that it was in the CRRT (P= .08), but not in the FGCRT (Table 3). In the CRRT, IV ganciclovir therapy was associated with a mean increase in the percentage of retinal area involved with retinitis over the first 6 months of follow-up of 1.9% compared with IV combination therapy; IV foscarnet was associated with an increase of 1.3% compared with IV ganciclovir. For example, the estimated mean increases in retinal area involved after 6 months of follow-up were 15.2%, 13.3%, and 16.5% for IV ganciclovir, IV combination, and IV foscarnet, respectively. In the MACRT, IV or oral systemic monotherapy was associated with a mean increase of 5.2% as compared with IV combination therapy and an increase of 4.6% as compared with local therapy. After 6 months, the mean increases in retinal area with retinitis for involved eyes of newly diagnosed patients were 10.1%, 4.9%, and 5.4% for monotherapy, IV combination therapy, and local therapy, respectively.
Visual acuity results from follow-up were available on 88% to 94% of involved eyes. After adjustment for stage of retinitis and initial treatment for CMV, the median times to decline in visual acuity of 15 letters or more on an Early Treatment Diabetic Retinopathy Study chart (eg, going from 20/20 to 20/40 in Snellen equivalents) differed among the trials (Table 3) (P = .02) (Figure 1). Involved eyes of patients with newly diagnosed and relapsed retinitis had similar risks of events (Table 3) (P = .45). Initial treatment for CMV retinitis did not influence the risk of visual acuity decline in any of the trials (Table 3).
Visual field data were available on 64% to 82% of involved eyes. The rate of visual field loss differed among the 3 trials (Table 3) (P<.001). However, the rate of visual field loss was similar for involved eyes of patients with newly diagnosed and relapsed retinitis (Table 3) (P = .46).
In the FGCRT and CRRT, visual field loss during follow-up was not associated with initial treatment for CMV retinitis (Table 3). In the MACRT, there were treatment-related differences in visual field loss for all involved eyes (P = .01)(Table 3). Initial treatment with IV systemic combination therapy was associated with a smaller decline in visual field over follow-up (mean, 48.0° less) than the decline in the IV or oral systemic monotherapy group. Local therapy also tended to be associated with a smaller overall decline in visual field than in the systemic monotherapy group (mean, 27.4° less). The estimated mean decreases in visual field for involved eyes of patients with newly diagnosed retinitis over 12 months were 117.0°, 69.1°, and 89.0° for monotherapy, IV combination therapy, and local therapy, respectively.
Overall, progression was associated with about a doubling of the risk of a decline in visual acuity of 15 or more letters (Table 4) (RR = 2.1, P<.001), and the increase in risk did not differ across trials (Table 4) (P = .86) or by stage of retinitis(Table 4) (P = .27).
The associations of change in the percentage of retinal area (zones 1 and 2) involved with retinitis to decline in visual acuity of 15 or more letters were similar across groups defined by stages of retinitis (Table 4) (P =.62) but differed among the trials (Table 4) (P = .001). In the FGCRT and MACRT, a 10% increase in area involved with retinitis was associated with a 1.3 (P<.001) and 1.9 (combined MACRT data, P = .01) increase in the risk of the event, but there was no association of change in area with visual acuity decline in the CRRT.
The strength of the associations of time-dependent retinitis measures to visual acuity decline were compared based on the deviance score (−2 log likelihood statistic); larger deviance scores indicated that the retinitis measure explained more of the variability in the outcome. Because the patterns of missing data were different for each retinitis measure, the deviance scores were calculated for the subgroup of eyes for which data on all retinitis measures were available, effectively those with data on retinal area involved. The RRs for each time-dependent retinitis measure estimated from these subgroups were consistent with the RRs estimated from all available data (data not shown). Change in area of retina involved with retinitis was more predictive of visual acuity decline than progression in the FGCRT and MACRT (Table 4). In the CRRT, progression was more predictive than change in area.
Overall, the average decline in visual field associated with retinitis progression was 75.1° (Table 4)(P<.001). The associations were not different across trials (Table 4) (P = .53) or by stage of retinitis (Table 4) (P = .86).
Change in the percentage of retinal area involved also was associated with change in visual field (Table 4)(Figure 2). Overall, the average decline in visual field associated with an increase of retinal area involved by 10% was 52.1° (Table 4)(P<.001). The associations were similar across trials (Table 4) (P = .25), but tended to be larger for involved eyes of relapsed patients(Table 4) (P = .06).
The strength of the associations between retinitis measures and visual field loss were compared with deviance scores. In all groups, change in area of retina involved with retinitis was more predictive of decline in visual field than progression (Table 4).
In these analyses the only observed treatment effect on a vision measure was in the MACRT. Systemic combination therapy was associated with a smaller decline in visual field than systemic monotherapy. To evaluate how much of the treatment-related differences in visual field were accounted for by an intermediate retinitis measure (ie, progression or change in area involved), the treatment effect on visual field was estimated in models without and with a covariate for the time-dependent retinitis measure (Table 5). In model 1, which does not include a time-dependent retinitis measure, systemic combination therapy was associated with a smaller loss of visual field (mean, 48.0° less), than treatment with systemic monotherapy. Inclusion of progression as a time-dependent covariate in the model (model 2) resulted in a 50% reduction in the average treatment effect associated with combination systemic treatment (ie, from −48.0° to −23.9°), and the treatment effect was no longer significant. Inclusion of change in area of retina involved with retinitis in the model (model 3) resulted in a 66% reduction of the treatment effect, and the effect was no longer significant. Therefore, each measure accounted for some of the treatment-related differences in visual field, but change in area seemed to capture more of the effect.
Estimates of the sample size required to detect 50% differences in outcome(ie, an RR of 1.5 or a 50% change in continuous measure), for a clinical trial of 2 treatments were 182 patients for progression, 226 for change in area involved with retinitis, 312 for visual acuity loss (≥15 letters), and 352 for change in visual field.
The criteria used to evaluate measures of retinitis as surrogate outcomes for visual function measures were how well the measure predicted changes in visual function, how well treatment-related changes in vision outcomes were accounted for by changes in the measure, the completeness of data collection, and the sample size required for trials. Results for the 2 measures are summarized in Table 6.
Change in retinal area involved was more predictive of visual acuity decline for involved eyes than progression (Table 4). However, the association was not uniform; it was not associated with visual acuity decline in the involved eyes of patients with relapsed retinitis enrolled in the CRRT. On average, these eyes had the largest area involved with retinitis, and the highest rate of inner zone 1 involvement(Table 2). Therefore, relatively small increases in retinal area involved may have had detrimental effects on visual acuity thereby attenuating the association. Change in area involved with retinitis was a stronger predictor of visual field decline than progression and the associations were uniform across trials and by stage of retinitis(Table 4).
In the MACRT, change in area of retina involved with retinitis accounted for a larger portion of the treatment effect on visual field (66%) than progression(50%) (Table 5). Although the confidence intervals on these proportions may be large,11,12 the ranking is likely to be correct and is consistent with the observed associations of the retinitis measures to visual field loss (Table 4). Area of retina involved is a continuous measure able to incorporate both spatial and temporal aspects of retinitis spread, unlike the threshold measurement of progression.
Even with up to 1 year of follow-up, observed treatment effects on retinitis measures were not uniformly followed by treatment effects on visual function measures. Does the lack of consistent treatment effects on both types of measures indicate that retinitis measures are not good surrogate outcomes? We think not. Effects on visual acuity are likely to be influenced by location of retinitis. Eyes with retinitis in zone 1 are more likely to lose visual acuity regardless of the status of retinitis measures (data not shown). Furthermore, the Studies of Ocular Complications of AIDS protocols included safeguards against visual acuity loss that made it less likely that treatment-related differences would be observed. In the CRRT, a treatment effect on visual field was observed in the primary results of the trial.14 That analysis included all data collected before protocol suspension, whereas the present analysis included data collected during the first year of follow-up, regardless of whether the visits occurred before or up to 6 months after the protocol was suspended. The latter approach may have emphasized incongruities between treatment effects on retinitis and vision measures because of treatment modifications during follow-up. In the CRRT, patients assigned to the combination of IV ganciclovir and foscarnet were more likely to have treatment-related toxic effects and switch to monotherapy,14 and patients receiving monotherapy who progressed were likely to receive combination therapy, especially after the treatment protocol was suspended. These results highlight another potential advantage of a valid surrogate outcome—the ability to measure treatment response before irreparable harm to the patient occurs.
Data on progression were available for at least 1 follow-up visit for most involved eyes (93% overall), whereas available data on area of retina involved with retinitis were fewer (64% overall). Two factors that contributed to that high rate of missing data were less frequent assessments of retinal area involved and photograph quality. Area assessments were done less frequently because they required more time. Photograph quality is more critical for evaluating retinal area involved because the 10 photographs must overlap appropriately to document all of zones 1 and 2, and all of the photographs must be of good quality. Data on retinal area involved from 19% of photograph sets were not used because more than 10% of zones 1 and 2 could not be graded, whereas only 9% of photograph sets could not be graded for progression.
The sample size estimates were smallest for trials with progression as the design outcome, which required 13% fewer patients than if change in area involved was the design outcome and at least 40% fewer than required for change in visual function. Change in area involved with retinitis is less efficient because of the relatively large variance of the measure.
Overall, retinitis progression in an involved eye fulfilled the criteria outlined for a surrogate outcome better than change in area involved with retinitis (Table 6). Progression was associated with all measures of decline in visual function and the associations were uniform across all groups of patients; it accounted for, in part, the treatment effect on the clinical outcome of visual field decline; relatively complete data collection was achievable; and the sample size required for a trial was reduced.
There is an obvious relationship between the retinitis measures evaluated and vision—if more retina is destroyed, vision is lost. However, even in this situation, these measures were imperfect surrogates for vision. Progression and change in retinal area involved accounted for only part of the treatment-related change in visual field and did not reflect long-term visual outcomes. Patients without progression lost vision because of other causes, such as retinal detachments. These data suggest that retinitis progression is a reasonable outcome for small trials evaluating the efficacy of new treatments; however, larger trials evaluating the relative efficacy of treatments should include measures of visual acuity and field.
Accepted for publication September 7, 2000.
Supported by cooperative agreements from the National Eye Institute, Bethesda, Md, to the School of Hygiene and Public Health (U10 EY 08057, 1 R03 EY10731-01, NRSA EY07127) and School of Medicine (U10 EY 08052), Johns Hopkins University, Baltimore, Md, and the University of Wisconsin School of Medicine, Madison (U10 EY 08067). Additional support provided by National Center for Research Resources, through General Clinical Research Center grants 5M01 RR 00350 (Baylor College of Medicine, Houston, Tex); 5M01 RR 00035 and 5M01 RR 00722 (Johns Hopkins University); 5M01 RR 05096 (Louisiana State University/Tulane, New Orleans); 5M01 RR 00071 (Mt Sinai Medical Center, New York, NY); 5M01 RR 00047 (New York Hospital-Cornell Medical Center, New York); 5M01 RR 00096(New York University, New York); 5M01 RR 00048 (Northwestern University, Evanston, Ill); 5M01 RR 00865 (University of California, Los Angeles); 5M01 RR 00083(University of California, San Francisco); 5M01 RR 05280 (University of Miami, Miami, Fla); and 5M01 RR 00046 (University of North Carolina, Chapel Hill). Support also provided by the National Institute of Allergy and Infectious Diseases, Bethesda, Md, through cooperative agreements U01 AI 27668 (Johns Hopkins University); U01 AI 27674 (Louisiana State University/Tulane); U01 AI 27669 (Memorial Sloan-Kettering, New York, NY); and U01 AI 25917 (New York Hospital-Cornell Medical Center); U01 AI 27667 (Mount Sinai Medical Center); U01 AI 27665 (New York University); U01 AI 25915 (Northwestern University); U01 AI 27660 (University of California, Los Angeles); U01 AI 27670 (University of California, San Diego); U01 AI 27663 (University of California, San Francisco); and U01 AI 25868 (University of North Carolina). Funding was also provided by Astra Pharmaceutical Products Inc (Westborough, Mass) and Protein Design Laboratories Inc (Mountain View, Calif).
Drugs were provided by Amgen Inc (Thousand Oaks, Calif), Astra Pharmaceutical Products Inc, Bristol-Myers Squibb Co (Princeton, NJ), Burroughs Wellcome Co (Research Triangle Park, NC), Syntex Research (Palo Alto, Calif), and Protein Design Laboratories Inc, Plymouth, Minn.
The membership of the Studies of Ocular Complications of AIDS Research Group is listed in prior articles reporting results of these trials.7,15,16
Corresponding author: Janet T. Holbrook, PhD, MPH, 615 N Wolfe St, Room 5010, Baltimore, MD 21205 (e-mail: firstname.lastname@example.org).