Glassman AR, Stockdale CR, Beck RW, Baker C, Bressler NM, Diabetic Retinopathy Clinical Research Network FT. Evaluation of Masking Study Participants to Intravitreal Injections in a Randomized Clinical Trial. Arch Ophthalmol. 2012;130(2):190-194. doi:10.1001/archophthalmol.2011.387
Author Affiliations: Jaeb Center for Health Research, Tampa, Florida (Mr Glassman, Ms Stockdale, and Dr Beck); Paducah Retinal Center, Paducah, Kentucky (Dr Baker); and Wilmer Eye Institute, The Johns Hopkins University School of Medicine, Baltimore, Maryland (Dr Bressler).
Group Information: The Diabetic Retinopathy Clinical Research Network members are listed in Ophthalmology. 2010;117(6):1064-1077.e35.
Objective To evaluate the success of masking study participants to treatment allocation using sham intravitreal injections.
Methods Eyes were randomized to receive sham injections plus prompt laser, intravitreal ranibizumab injections plus prompt laser, intravitreal ranibizumab injections plus deferred laser, or intravitreal triamcinolone acetonide injections plus prompt laser up to every 16 weeks with sham injections intermittently. All eyes could receive treatment or sham as often as every 4 weeks. Participants with 2 study eyes had 1 eye randomized to sham plus prompt laser and 1 eye randomized to a real injection group. Sham injections were performed by pressing the syringe hub against the conjunctiva to mimic a real injection. Laser treatment was not masked. At the 1-year visit, participants were asked if they believed that the injections received during the study were real, sham, or sometimes real and sometimes sham.
Results Among 423 participants with 1 study eye, the correct assignment was stated by 9.9% of the sham plus prompt laser group, 88.0% of the ranibizumab plus prompt laser group, 89.6% of the unmasked ranibizumab plus deferred laser group, and 44.0% of the triamcinolone plus prompt laser group. Among 112 participants with 2 study eyes, the correct assignment was stated for 24.1% of the sham plus prompt laser eyes.
Conclusions Successful masking of an intravitreal injection can be accomplished when a sham injection procedure carefully mimics a real injection procedure. Masking seems less successful when one eye is receiving a real injection and the other eye is receiving a sham injection or when an individual eye receives both real and sham injections.
Minimizing bias in a randomized clinical trial is a critical consideration during study design. Randomization alone does not ensure an unbiased experiment. An important factor to consider in designing a randomized trial is whether to mask participants to the treatment group assignment. The subjectivity of the outcome, type of treatment, and disease should be considered when determining whether study participants should be masked. For example, visual acuity, which is the primary outcome measure for many ophthalmic trials, is a subjective measurement, whereby the results can be influenced by participant effort. Therefore, participant knowledge of the treatment assignment could bias the measurement. Masking study participants to the treatment group assignment is also important when knowledge of the treatment group could affect a participant's behavior in such a way that it could influence the course of the disease or have an effect on the primary outcome of the trial.1 Although usually desirable, masking of participants is not always feasible and in some cases is impossible.
Numerous randomized clinical trials have recently compared intravitreal injections with standard treatment in several retinal diseases, including diabetic macular edema (DME), age-related macular degeneration, and macular edema from retinal vein occlusions.2- 4 In studies evaluating intravitreal injections, sham injections are sometimes used to attempt to mask the study participant to treatment assignment.5,6 In general, a sham intravitreal injection is a procedure that mimics a real intravitreal injection but does not penetrate the eye. In contrast, some studies use a placebo intravitreal injection, in which an inert substance like saline is injected into the vitreous.7 Although a sham injection may be less effective than a placebo injection at masking a study participant, a sham injection is more often used because it decreases the risk of injection-related complications. Sham intravitreal injections have been used frequently, yet the masking success of the sham injections has not been reported to our knowledge. The Diabetic Retinopathy Clinical Research Network (DRCR.net) conducted a clinical trial evaluating the use of intravitreal ranibizumab or intravitreal triamcinolone acetonide to treat DME in which the success of masking study participants was evaluated. The results of the masking are reported herein.
The success of masking of sham injections was assessed in the DRCR.net trial Laser-Ranibizumab-Triamcinolone for Proliferative Diabetic Retinopathy, which was a phase 3 randomized clinical trial conducted at 52 clinical centers across the United States (http://www.clinicaltrials.gov Identifier: NCT00444600; protocol: http://www.drcr.net; accessed January 1, 2010). Participants with 1 study eye were randomized to 1 of the following 4 treatment groups: (1) sham injection plus prompt focal/grid photocoagulation (sham plus prompt laser group), (2) intravitreal ranibizumab injection (0.5 mg) plus prompt focal/grid photocoagulation (ranibizumab plus prompt laser group), (3) intravitreal ranibizumab injection (0.5 mg) with deferred focal/grid photocoagulation (ranibizumab plus deferred laser group), or (4) intravitreal triamcinolone acetonide injection (4.0 mg) plus prompt focal or grid photocoagulation with triamcinolone injections up to every 16 weeks with sham injections intermittently (triamcinolone plus prompt laser group). All eyes could receive treatment or sham as often as every 4 weeks. Participants with 2 study eyes were randomly assigned to receive ranibizumab plus prompt laser, ranibizumab plus deferred laser, or triamcinolone plus prompt laser in one eye and sham plus prompt laser in the other eye. The details of the trial have been reported previously.8 A prior real intravitreal injection was not an exclusion criterion provided that treatment was at least 4 months before randomization.
To minimize potential bias on measurement of the study outcome, study participants were masked through the 1-year primary outcome except for those with an eye assigned to the ranibizumab plus deferred laser group. This group was not masked because when the study was developed, the investigators believed that a reliable sham laser treatment to mask study participants was infeasible and was unlikely to be successful. DRCR.net-certified examiners of visual acuity (the primary outcome) were masked to treatment allocation for all groups at the 1-year primary outcome visit. Investigators were not masked to treatment assignment.
Study drug or sham injections were given as frequently as every 4-week study visit. When retreatment with a study drug or sham injection was indicated, eyes assigned to 1 of the ranibizumab groups could receive ranibizumab as often as every 4-week study visit; eyes assigned to intravitreal triamcinolone could receive triamcinolone as often as every fourth 4-week study visit (ie, approximately every 16 weeks), with sham injections as often as every 4-week study visit in between triamcinolone injections; eyes assigned to sham plus prompt laser could receive sham injections as often as every 4-week study visit. There were 13 maximally possible sham or intravitreal injections before the 1-year primary outcome visit. The median (25th and 75th percentiles) numbers of injections through the first year were as follows: 11 (8 and 13) sham injections in the sham plus prompt laser group, 8 (6 and 10) ranibizumab injections in the ranibizumab plus prompt laser group, 9 (6 and 11) ranibizumab injections in the ranibizumab plus deferred laser group, and 5 (3 and 7) sham injections and 3 (2 and 4) triamcinolone injections in the triamcinolone plus prompt laser group.
All preinjection and postinjection procedures were identical for sham and intravitreal injections. Both procedures required the use of an eyelid speculum and were preceded by povidone iodine preparation of the conjunctiva directly over the site to receive the injection. For a sham injection, the hub of a syringe without a needle was pressed against the conjunctival surface to simulate the force of an actual injection. Indirect ophthalmoscopy was required after injection to confirm that the central retinal artery was perfused and to assess any complications. The use of antibiotics in the preinjection, peri-injection, or postinjection period was at the discretion of the investigator. The frequency of preinjection and postinjection antibiotic use was similar when a real injection was performed compared with when a sham injection was performed. For real injections vs sham injections, both postinjection and preinjection antibiotics were given 33.3% vs 25.2% of the time, only preinjection antibiotics were given 10.0% vs 11.8% of the time, and only postinjection antibiotics were given 21.2% vs 14.5% of the time. Postinjection intraocular pressure measurement was also at the discretion of the investigator. Intraocular pressure measurement was performed 41.1% of the time with real injections and 34.9% of the time with sham injections.
At the completion of the 1-year primary outcome visit, each study participant was asked a question to assess the effectiveness of the masking technique. Study coordinators or investigators were instructed to read the following question aloud to the participants: “Do you think the injections you have been getting in the [right or left] eye during the study have been: (1) real injections into the eye; (2) sham injection, meaning that a needle has not been injected into the eye; or (3) sometimes real and sometimes sham?” Each study participant was to be instructed to make his or her best guess as to treatment allocation if uncertain. If the study participant refused to choose 1 of the 3 choices, this was recorded. The validity of the masking question has not been established. The masking question was implemented in April 2008; 55 participants with 71 study eyes completed their primary outcome visits before implementation of the masking question and did not have an opportunity to respond, and 38 participants with 46 study eyes did not complete the 1-year primary outcome visit and did not complete the masking question. In addition, 15 participants with 21 study eyes were excluded because the study participant received a nonstudy treatment for DME in at least 1 eye.
Of 565 randomized participants eligible for this analysis, 125 had 2 study eyes, totaling 690 study eyes. The treatment group assignments for participants with 1 study eye are given in Table 1, and those for participants with 2 study eyes are given in Table 2. Overall, 43.9% of eligible participants were women. Eligible participants had a mean (SD) age of 63 (10) years and a mean (SD) visual acuity letter score of 63 (12), which was an approximate Snellen equivalent of 20/63. Of the eligible cohort, 72.6% were of white race/ethnicity, 16.1% were African American, and 8.5% were Hispanic or Latino. One-hundred sixty eyes (23.2%) had received a prior intravitreal injection for DME. Baseline characteristics of the full study cohort have been reported previously.8
At the 1-year primary outcome visit, study participants were asked to indicate which treatments the participant believed that he or she had received. Seventeen study participants with 1 study eye (3.9%) and 13 study participants with 2 study eyes (10.4%) refused to guess the treatment assignment (Table 1 and Table 2). Among 423 participants with 1 study eye who answered the masking question and who had only received the randomized treatment (Table 1), 9.9% of the sham plus prompt laser group believed that a sham injection was always given (meaning 90.1% thought that they received real injections). Participants believed that a real injection was always given in 88.0% of the ranibizumab plus prompt laser group and in 89.6% of the ranibizumab plus deferred laser group (correct response) and that a real injection was sometimes given in 44.0% of the triamcinolone plus prompt laser group (correct response). Across all 4 treatment groups, only 3.4% of participants indicated a belief that sham injections were always given. Correctness of response did not seem to differ for any of the following subgroups within treatment groups (Table 3): age, gender, race/ethnicity, baseline visual acuity, prior laser treatment for DME, and prior intravitreal injection for DME, as well as whether the site was an academic-based or community-based center, whether treatment was given at the 48-week visit (before masking assessment), and whether the participants received 3 or more injections between the 28-week to 48-week visits (inclusive).
Among 112 participants with 2 study eyes (Table 2), 24.1% of study participants believed that a sham injection was always given in the eye that received only sham injections (meaning 75.9% thought that they received real injections). Among 79 participants with 2 study eyes from the masked treatment groups (ie, excluding participants with either eye in the ranibizumab plus deferred laser group), 22.8% of study participants believed that a sham injection was always given in the eye that received only sham injections. For the eye receiving a real injection at least sometimes, the correct assignment was stated for 84.1% of the ranibizumab plus prompt laser group, 87.9% of the ranibizumab plus deferred laser group, and 31.4% of the triamcinolone plus prompt laser group.
Among participants with 1 study eye in the sham plus prompt laser group, a positive visual acuity response to treatment did not significantly affect the percentage of correct responses (P = .63). The participants believed that real injections had been given always or sometimes in the following scenarios: 90.0% of the time when visual acuity improved by 5 or more letters (n = 60), 93.1% of the time when visual acuity changed by 4 or fewer letters (n = 29), and 83.3% of the time when visual acuity worsened by 5 or more letters (n = 12). Therefore, even for 41 participants with no apparent improvement in the sham plus prompt laser group, 37 participants (90.2%) still believed that a real injection had occurred. Similarly, the mean (SD) visual acuity letter score change for an eye in the sham plus prompt laser group was 6.0 (9.4) for participants who thought that real injections had been received and 3.6 (8.1) for participants who thought that only sham injections had been received (P = .45).
Masking study participants to treatment allocation is an important factor to consider when assessing potential bias in a trial. This study evaluated how well a sham intravitreal injection would mask participants as to whether a real or sham injection had been given. The results indicate that few participants believed that they received only sham injections in a randomized trial comparing sham intravitreal injections with real intravitreal injections. Furthermore, in the treatment group that at some visits received a real injection and at other visits received a sham injection, most participants seemed to believe that they had received a real injection when they received a sham injection. Even when visual acuity during the study worsened or did not improve, most participants thought that they had received real injections.
Compared with participants who received sham injections only, participants in the treatment group that received both real and sham injections in the same eye thought less often that real injections had always been given. This may imply that a participant is more likely to identify a sham injection when also receiving a real injection in the same eye. However, the visual disturbances caused by the opaqueness of the triamcinolone may have contributed to this decreased masking. However, because more than half of the participants in the group receiving both real and sham injections believed that real injections had always been given, it seems that masking was successful even in this circumstance.
Across all the treatment groups masked per protocol, results showed that participants with 2 study eyes more often believed that sham injections had been always given in the sham eye than participants in the sham group with only 1 study eye (22.8% vs 9.9%, P = .02). Nevertheless, 77.2% of participants with 2 study eyes thought that the eye receiving only sham injections had received real injections.
Because masking is successful does not mean that it is necessary (ie, that unmasking of study participants will result in bias). The effect of masking intravitreal injections on visual acuity and patient-reported visual function outcome has been evaluated in subfoveal choroidal neovascularization secondary to age-related macular degeneration.9,10 In those studies, the authors showed in some scenarios that visual acuity outcome in an unmasked cohort were similar to those of a matched masked cohort from a subsequent trial. However, it is unknown if the same results would apply to other diseases or other trial designs. In fact, considering the possible effect that treatment group knowledge may have on participant behavior during a study, it seems unlikely that the behavior of a participant with age-related macular degeneration during the study could have much influence on disease progression. In contrast, with diabetes, improving glycemic control could have an effect on DME and subsequently influence related visual acuity outcome. Our study did not assess the potential bias in visual acuity outcome that may occur when study participants receiving intravitreal injections are not masked to treatment assignment.
In summary, this study demonstrated that a sham intravitreal injection can successfully mask study participants to treatment group assignment when a detailed procedure is followed that attempts to carefully mimic a true injection procedure. Masking may be less successful when one eye receives a real injection and the other eye receives a sham injection or when an individual eye receives both real and sham injections.
Correspondence: Adam R. Glassman, MS, Jaeb Center for Health Research, 15310 Amberly Dr, Ste 350, Tampa, FL 33647 (firstname.lastname@example.org).
Submitted for Publication: June 23, 2011; accepted August 3, 2011.
Financial Disclosure: A complete list of all DRCR.net investigator financial disclosures can be found at http://www.drcr.net.
Funding/Support: This study was supported through cooperative agreements EY14231, EY14229, and EY018817 from the National Eye Institute and the National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, US Department of Health and Human Services.
Role of the Sponsor: The funding organization (National Institutes of Health) participated in oversight of the conduct of the study and review of the manuscript but not directly in the design or conduct of the study nor in the collection, management, analysis, or interpretation of the data or in the preparation of the manuscript.