Clinical exclusion criteria. Study entry defined by first visit within 5-year retrospective period during which a set of binocular 24-2 or 30-2 Humphrey visual field (HVF) tests was performed (ie, HVF test used to initially stage the medical record). AMD indicates age-related macular degeneration; ETDRS, Early Treatment of Diabetic Retinopathy Study.
Total annual direct cost of glaucoma treatment per patient by stage.
Total annual direct cost of glaucoma treatment per patient by stage adjusted for realistic medication compliance.
Total direct costs of care reflecting real-world compliance rates and Medicare national average allowances costs.
Percentage of patients with stable staging over time by initial stage. Stage 5 patients have been excluded because no further stage advancement is possible for patients with end-stage disease.
Lee PP, Walt JG, Doyle JJ, Kotak SV, Evans SJ, Budenz DL, Chen PP, Coleman AL, Feldman RM, Jampel HD, Katz LJ, Mills RP, Myers JS, Noecker RJ, Piltz-Seymour JR, Ritch RR, Schacknow PN, Serle JB, Trick GL. A Multicenter, Retrospective Pilot Study of Resource Use and Costs Associated With Severity of Disease in Glaucoma. Arch Ophthalmol. 2006;124(1):12-19. doi:10.1001/archopht.124.1.12
To examine resource consumption and the direct costs of treating glaucoma at different disease severity levels.
Observational, retrospective cohort study based on medical record review.
One hundred fifty-one records of patients with primary open-angle or normal-tension glaucoma, glaucoma suspect, or ocular hypertension (age ≥18 years) were randomly selected from 12 sites in the United States and stratified according to severity based on International Classification of Diseases, Ninth Revision, Clinical Modification codes. Patients had to have been followed up for a minimum of 5 years. Patients with concomitant ocular disease likely to affect glaucoma treatment–related resource consumption were excluded.
Glaucoma severity was assessed and assigned using a 6-stage glaucoma staging system, modified from the Bascom Palmer (Hodapp-Anderson-Parrish) system. Clinical and resource use data were collected from the medical record review. Resource consumption for low-vision care and vision rehabilitation was estimated for patients with end-stage disease based on specialist surveys. For each stage of disease, publicly available economic data were then applied to assign resource valuation and estimate patient-level direct costs from the payer perspective.
Main Outcome Measures
Average annual resource use and estimated total annual direct cost of treatment were calculated at the patient level and stratified by stage of disease. Direct costs by specific resource types, including ophthalmology visits, glaucoma surgeries, medications, visual field examinations, and other glaucoma services, were also assessed.
Direct ophthalmology-related resource use, including ophthalmology visits, glaucoma surgeries, and medication use, increased as disease severity worsened. Average direct cost of treatment ranged from $623 per patient per year for glaucoma suspects or patients with early-stage disease to $2511 per patient per year for patients with end-stage disease. Medication costs composed the largest proportion of total direct cost for all stages of disease (range, 24%-61%).
The study results suggest that resource use and direct cost of glaucoma management increase with worsening disease severity. Based on these findings, a glaucoma treatment that delays the progression of disease could have the potential to significantly reduce the health economic burden of this chronic disease over many years.
Glaucoma affects an estimated 2.2 million Americans older than 40 years, half of whom remain undiagnosed.1,2 The incidence of glaucoma is expected to rise with the growth of the elderly population. Glaucoma costs the US health care system an estimated $2.5 billion annually: $1.9 billion in direct costs and $0.6 billion in indirect costs.3 Several retrospective medical record reviews have considered the aggregate economic burden associated with the management of glaucoma.3- 5 However, few data exist regarding medical resource consumption as a function of varying disease severity in glaucoma, particularly in the treatment of end-stage disease and with the advent of more aggressive treatment patterns and the development of new treatment classes (such as glutamate receptor antagonists as a potential neuroprotective agent). To better understand the value and potential economic impact of such therapy, it is necessary to measure the resource use associated with progressive disease severity. Therefore, this study aimed to test the hypothesis that resource consumption and direct costs increase with progression of disease severity.
Given the objective of assessing resource use and direct costs as a function of disease severity, a glaucoma staging system (GSS) was required to assign severity categories to all study patients. Although numerous systems have been developed, no universally accepted GSS exists to unambiguously categorize the entire range of disease severity for glaucoma suspect, ocular hypertension, normal-tension glaucoma, and primary open-angle glaucoma. To address this issue, a literature search was conducted and a number of existing GSSs were reviewed6- 15 by a panel of 4 glaucoma specialists (R.P.M., D.L.B., P.P.L., R.J.N.) for potential modification into a wider-range staging system. For the purposes of this study, the Bascom Palmer (Hodapp-Anderson-Parrish)16 GSS was considered most appropriate because it allowed for structured severity-stage assignment based on Humphrey visual field (HVF) examination parameters, in a relatively simplified manner, easily applicable to a retrospective medical record review. Based on consensus of our opinion and pilot testing, modifications to the Bascom Palmer GSS were made to allow it to encompass the complete range of disease severity (Table 1).
We recruited 12 sites within the United States. Institutional review board exemptions (10 sites) and approvals (2 sites) were obtained. Eleven of the enrolled sites housed glaucoma specialist practices (10 university practices and 1 community-based), and the 12th site represented a staff-model managed care practice (Henry Ford Health System, Detroit, Mich).
Patient records with continuous follow-up of at least 5 years and with diagnoses codes corresponding to primary open-angle glaucoma (International Classification of Diseases, Ninth Revision, Clinical Modification code 365.11), normal-tension glaucoma (code 365.12), ocular hypertension (code 365.04), or glaucoma suspect (codes 365.00 and 365.01) were randomly selected at each site. Medical records were then qualified based on a minimum of 5 years of both clinical and HVF examination data. Patients with other concomitant ocular disease that could significantly alter resource consumption were excluded (Figure 1). Similarly, patients enrolled in a therapeutic clinical trial were excluded because enrollment in such a trial might have altered resource consumption.
“Study entry” was defined by the date on which the first binocular set of 30-2 or 24-2 HVF test results were documented in the medical record. This set of HVF test results was used to assign an initial stage based on the finalized 6-stage GSS and the patient’s worst eye. For stage 5 (end-stage disease) patients, visual acuity at study entry was also used to determine staging (ie, no hand motion and/or no light perception documented under visual acuity for that eye).
These randomly selected, qualified medical records were staged and sequentially selected for abstraction purposes until a minimum of 2 medical records per stage for stage 0 through stage 4 and 1 medical record for stage 5 were selected (ie, a minimum of 11 records per site).
A Microsoft Access (Microsoft, Redmond, Wash) electronic database was created for data collection purposes and included fields for patient demographic information and medical and ocular history, including glaucoma risk factors, ophthalmologist visits, medications, surgical procedures, and HVF examination data. Fields were also created for relevant clinical examinations and tests and their respective findings, including intraocular pressure assessments and diurnal curves, slitlamp examinations, gonioscopies, optic nerve assessments, retinal and macular examinations, nerve fiber thickness assessments, optic disc photographs, and dilated eye examinations.
A glaucoma specialist with expertise in ophthalmology medical record reviews (P.P.L.) trained 2 medical record abstractors. Training involved a 7-day course covering relevant ophthalmology terminology, patient medical record organization, interpretation of typical medical record diagrams and notation, and an electronic data-entry module. Prior to beginning the study, the trained abstractors completed staging of 2 test medical records, which were later validated by the source documents. The abstractors also demonstrated interrater (>95% agreement), intrarater (>99% agreement), and gold standard (vs the ophthalmologist trainer, at >96% agreement) reliability on test medical records. In addition, periodic monitoring of key abstraction items was performed, with an interrater accuracy of more than 96% relative to the training ophthalmologist and more than 95% between the abstractors.
Low-vision specialists at 7 low-vision care and vision rehabilitation centers in the United States were surveyed over the telephone to capture nonphysician resource use typically consumed by patients with end-stage glaucoma. The survey asked about the typical care patterns of patients with blindness due to glaucoma, including the frequency of reimbursable services, such as low-vision care specialist visits, Goldmann visual field testing, and physical rehabilitation services.
For each member of the cohort, the study period began at the time of study entry and concluded with the most recent visit recorded in the medical record. Data were collected for each visit during the study analytic time horizon and used to calculate the annual resource use on the patient level for each stage of disease. Outcomes assessed included ophthalmologist visits, surgical procedures, medications, HVF test results, and other glaucoma services. A 100% compliance rate was assumed for all prescribed medications in the base model. Since compliance in the real world is not 100%, a sensitivity analysis was performed using a lower rate of compliance. Because compliance data from the medical records was inconsistently documented (though noted when available), we elected to use data from Gurwitz et al,17 who assessed compliance with glaucoma medications in a state managed care organization. Additionally, we performed sensitivity analysis at compliance rates higher and lower than the rate obtained from Gurwitz et al to project the effect of compliance on the total direct cost of treatment and test the robustness of the study findings.
Average resource use outcomes were calculated across all patients within a given stage. Given the natural history of glaucoma, certain patients transitioned to a higher severity category during the study period. If progression was observed, defined by 2 consecutive HVF test results demonstrating a more severe stage than previous HVF test results, the patient was considered to be at the more advanced stage of disease for resource use calculations.
A direct-cost analysis was then performed, using a method similar to previous health economic evaluations of glaucoma treatment.3,4 For each stage of disease, average annual ophthalmology resource use values were multiplied by their respective unit costs to calculate average annual ophthalmology costs on a patient level from a payer perspective. Similarly, for patients with end-stage glaucoma, average annual nonphysician resource use values were multiplied by their respective unit costs and added to the average annual ophthalmology cost for stage 5 patients to calculate the total direct cost of glaucoma from a payer perspective.
Unit costs for visits, surgeries, and other relevant procedures were based on Current Procedural Terminology billing codes and publicly available mean (50th percentile) physician charges in the United States.18,19 Ophthalmology visits without dilated eye examinations were assigned a unit cost based on the level 2 Current Procedural Terminology billing code for eye visits, whereas visits incorporating dilated eye examinations were presumed to represent more comprehensive visits and were assigned a unit cost based on the level 4 Current Procedural Terminology billing code for eye visits. Since mean reimbursements for services may not reflect physician and facility charges, an alternate resource valuation was performed with reimbursements for physician and facility fees based on Medicare national average allowances.20 As compared with commercial insurers, Medicare national average allowances reimbursement may be higher or lower, depending on the type of service or procedure provided.20,21
Unit costs for medications were based on Red Book average wholesale prices.21 A 5-mL bottle prescribed twice daily or its equivalent (eg, a 2.5-mL bottle prescribed once daily) was assumed to represent a standard 1-month supply. Data on medication use were collected at every visit, and patients were assumed in the base model to be 100% compliant unless otherwise noted in the medical record. Because changes in medication regimens were frequent, medication costs were calculated on a daily basis.
Using a generalized linear model with “stage” as an independent variable, total direct cost per patient per stage was predicted, adjusting for age, sex, and follow-up time. Because of its skewed distribution, the square root of cost was used to normalize the distribution. Since data on ethnicity were missing for 40 patients (ie, 26.5% of the total sample), the model was fit without adjusting for ethnicity. The model was refit within the subgroup of patients with complete ethnicity data to evaluate whether the association between cost and stage differed with the exclusion of ethnicity. A second multivariate model was fit in a similar manner using the Medicare national average allowances reimbursement and costs.
Baseline characteristics of the study patients are presented in Table 2. The mean (SD) age of the entire sample (N = 151) at baseline was 66.3 (11.9) years. The average age for patients at stages 0 through 5 was 57.5 years, 64.8 years, 70.1 years, 69.6 years, 67.3 years, and 68.8 years, respectively. There were 86 women (58%). Seventy patients (46%) were white, 33 (22%) were black, 7 (5%) were Asian, and no data were available for the remaining 27%. Among patients with available data on family history of glaucoma (109 [72%]), 53% had a positive family history.
At the patient level, the use of resources increased with greater severity of disease, as presented in Table 3. Follow-up visits to the physician were more frequent as disease severity worsened from stage 0 through stage 4. The average number of visits was slightly lower for stage 5 patients as compared with stage 4 patients in the abstracted medical records; however, while no non–low vision care visits were found for stage 5 patients in this study, this figure does not incorporate non–low vision care visits that may have been made by patients with end-stage disease.
The data also suggest that the intensity or aggressiveness of therapy increases with disease severity. The average number of medications prescribed to patients increased with worsening disease severity through stage 4. Similarly, the average annual number of glaucoma surgeries (both laser and incisional) increased as disease severity worsened from stage 0 to stage 4. In addition, the distribution between laser (trabeculoplasty) and incisional (trabeculectomy) procedures appears to have shifted toward fewer laser procedures and more incisional procedures as disease severity worsened from stage 2 to stage 5. In contrast, the average annual number of HVF tests per patient by stage of disease is more constant, suggesting a relatively stable use of HVF tests across patients in stage 1 through stage 5.
Figure 2 depicts the total annual direct cost of glaucoma treatment by stage at the patient level. For patients who transition between stages, costs are adjusted to reflect annual costs for time in each stage. For patients at stage 0 through stage 4, all direct costs are assumed to be ophthalmology costs and include costs associated with ophthalmologist visits, glaucoma surgeries, HVF testing, medications, and other glaucoma services, such as gonioscopies, optic disc photographs, nerve fiber thickness analysis, and intraocular pressure diurnal testing. These direct ophthalmology costs (based on charges) were estimated at $623 per patient per year for stage 0 patients and increased as disease severity worsened to an estimated $2464 per patient per year for stage 4 patients.
In the case of patients at stage 5, additional costs, namely low-vision care and vision rehabilitation services costs, were added to ophthalmology costs to calculate the total direct cost of treatment. The average nonphysician direct medical cost totaled $511 per patient per year and included the cost of low-vision care specialist visits ($232 per patient per year), non-HVF testing ($30 per patient per year), and physical rehabilitation services ($249 per patient per year), which covered self-care and home management training, community and work reintegration training, and sensory integrative training.Total direct cost (charges) of glaucoma for stage 5 patients was estimated at $2511 per patient per year.
While Figure 2 demonstrates cost under 100% compliance to prescribed drug therapy, Figure 3 (which incorporates medication use patterns from real-world compliance rates estimated by Gurwitz et al17) may be more reflective of actual US practice and patient management patterns. Figure 4 also adjusts for real-world compliance rates and uses Medicare national average allowances reimbursement as opposed to mean physician charges. Overall total direct cost and total direct cost for all stage strata, with the exception of stage 5, were marginally higher with Medicare national average allowances reimbursement as compared with the 50th-percentile physician charges. These results imply that commercial insurers may be slightly underpaying relative to Medicare for glaucoma treatment–related services.
Figure 5 presents the percentage of patients who remain within their initial stage throughout the study period. At least one quarter (25%) of patients at stage 0 to stage 3 progressed to a more advanced stage over the 5-year period. In contrast, patients at stage 4 had a far lower rate of visual field progression (less than 5%) over 5 years of follow-up.
The generalized linear model using mean physician charges indicated that total direct cost per patient per stage was significantly associated with stage (F5,179 = 10.76; P<.001) after controlling for age, sex, and follow-up time. Similar results were found using Medicare national average allowances reimbursement to estimate treatment costs (F5,179 = 10.65; P<.001). The associations persisted in both the mean physician-charge model and the Medicare national average allowances reimbursement model after adjusting for ethnicity (F5,131 = 6.96; P<.001 and F5,131 = 6.74; P<.001, respectively).
In treating a chronic and potentially progressive illness such as glaucoma, one may postulate that as disease severity worsens, increased resource consumption will be prompted by a physician’s desire to slow disease progression. In particular, glaucoma is generally asymptomatic or nonspecific in its symptoms in its earlier phases, resulting in delayed diagnosis and relatively increased medical vigilance throughout the later stages of disease. Results from this study generally support this hypothesis; there is a consistent pattern of increased resource consumption as disease severity worsens from stage 0 to stage 4.
However, direct ophthalmology resource use was lower for stage 5 patients compared with stage 4 patients. This result may be explained by the real-world observation that ophthalmologists have less to offer patients with severe visual impairment and blindness. Patients with end-stage disease typically have failed to adequately respond to conventional ocular antihypertensive medications and may have already undergone surgical procedures with suboptimal results. Because their visual impairment cannot be reversed with medication or surgery, patients with end-stage disease are typically referred to low-vision care and vision rehabilitation centers for further follow-up. When direct nonphysician costs associated with low-vision care and vision rehabilitation resource use are added to the total direct ophthalmology costs for stage 5 patients, the total direct cost of treatment for stage 5 patients ($2511 per patient per year) is in fact slightly higher than the total direct cost of treatment for stage 4 patients ($2464 per patient per year). This inclusion results in a trend of increasing total direct cost of treatment with worsening disease severity across all 6 levels of disease severity (Figure 2).
The study also found that glaucoma medications are major cost drivers in the total direct cost of glaucoma. Assuming 100% compliance, medication costs represent a minimum of 48% of total direct cost at all levels of severity (Table 4). With real-world compliance estimates, medication costs still represent 38% of total costs at stage 5 and at least 44% at stages 1 to 3. Additional scenarios using assumed medication compliance rates lower (35%) and higher (85%) than the realistic compliance rate obtained from Gurwitz et al17 (67.8%) were run through the model. Indeed, even when physician and facility components of direct costs were slightly increased from mean charges to Medicare national average allowances reimbursement and real-world compliance rates were applied, medications still represented approximately one third to one half of total direct cost (Table 4).
Although the overall results from this retrospective study appear to be within the range of similar glaucoma resource use studies,3- 5 a number of methodological issues and study limitations may constrain the generalizability of our findings. These include limitations inherent to the GSS used, limitations associated with the sample size and the sites enrolled, and limitations related to the data tracking and costing methods used.
The GSS developed for this study is based solely on HVF testing for stage 0 through stage 4 disease, thereby restricting both site and patient enrollment based on availability of HVF test data. While this limitation may produce an unrepresentative sample, we believe that automated visual field testing has been the gold standard for the past 10 years and that it provides the single, most useful indicator of disease severity given the wide range of visual field parameters produced by a standard 30-2, 24-2, or 10-2 HVF test. In addition, the use of HVF test parameters for staging purposes allows for precise, quantifiable, and objective staging assignment. Furthermore, the extraction of the necessary HVF test data can easily be learned by medical record abstractors without the intricate understanding of visual field interpretation that would be required in the case of nonautomated visual field data.
Using this GSS, there was less progression from stage 4 to stage 5 disease than between other stages (Figure 5). This could be due to the fact that progressing from stage 4 to stage 5 involves a relatively larger change in the staging criteria used (ie, no HVF testing done at all in that eye and no vision of hand motion and/or light perception in that eye). As such, it is quite possible that stage 4 includes a broader range of patients approaching end-stage disease, and it might be possible or desirable in the future to subdivide stage 4 into 2 or more discrete stages. Because we do not know the nature of the “intervals” between each stage, the low rate of progression detected from stage 4 could well be an artifact of the staging system.
Although the staging system was consistently applied to all patients, study results may be affected by the method of staging patients by their worst, rather than their best, eye. Since patients with 2 blind eyes may have different resource consumption from patients with only one blind eye, resulting stage estimates may be more variable for higher stages of disease than for lower stages. Further, to the extent that the fellow eye also had glaucoma (though no more severe), that would be likely to have increased resource consumption as well. Given the sample size limits, we did not further subdivide the analysis by potentially multiple stages for the fellow eyes. This would be an area for potential future exploration.
Another potential limitation of the GSS is that only visual field deterioration, as opposed to both field and optic nerve deterioration, is used as the criterion for progression. This limitation may be reconciled by a number of factors. First, patient quality of life and visual functioning are more related to the presence of visual field loss and not to the pathophysiology of glaucoma per se.22 Second, indications that the glaucoma had worsened owing to the optic nerve was abstracted from the medical records; no cases were found in which the optic nerve head permanently changed in the absence of a concomitant visual field change. Third, while the Ocular Hypertension Treatment Study23 reported that more than half of progressions from suspect status to glaucoma were based solely on optic nerve findings, these were findings read by an optic nerve head reading center. The physicians in our study did not obtain optic nerve head photographs on every visit; further, the accuracy of physicians to discern optic nerve head progression is generally less than that of a well-trained optic nerve head reading center.23 Most importantly, progression in the Early Manifest Glaucoma Trial occurred with at least some visual field changes in nearly all cases.24
In this study, HVF testing appeared to be performed less frequently in stage 0 patients (0.9 test per patient per year) and remained relatively stable across all other severity categories (1.4-1.5 tests per patient per year). The fact that visual field tests in stages 1 to 5 were obtained on a regular basis and did not vary by stage minimizes the influence of ascertainment bias in detecting progression. On the other hand, the rate at which stage 0 patients were diagnosed as progressing may be an underestimate. However, it is not likely to be an underestimate since the progression rate noted for stage 0 patients in this study far exceeds the rate noted at 5 years in the Ocular Hypertension Treatment Study reports and is consistent with the Ocular Hypertension Treatment Study reports for high-risk patients with suspected glaucoma.25
Other limitations include the limited number of medical records reviewed (151 records) and the fact that the majority of the sites were glaucoma specialist practices. Because this was a pilot study, a sample size sufficient to assess the feasibility of the retrospective methods was required as opposed to a sample size that would be sufficient for extrapolation of results to the entire US population. Given the exploratory nature of this pilot study, academic glaucoma specialist sites were recruited to provide the estimates of maximum levels of care. As such, it is likely that the recruitment of specialized practices introduced a selection bias that limits external validity because glaucoma specialists may be inclined to more vigilantly treat their patients as opposed to general ophthalmologists, resulting in increased resource use and higher treatment costs. Nevertheless, despite these limitations, this study does demonstrate a significantly greater use of resources with worsening glaucoma disease.
Limitations inherent to data collection, tracking, and costing methods should also be considered. First, unless otherwise specified in the medical record, it was assumed that patients were 100% compliant with medications during intervisit intervals, which risks overestimating resource use and costs associated with medication use. In fact, contrary to the results of retrospective studies conducted by Kobelt-Nguyen et al4 and Siebert et al,14 our study suggests that medication costs, as opposed to the costs of surgical procedures and/or visits, compose the majority of total direct costs of treatment. On the other hand, since the completion of these comparable retrospective analyses, a number of highly efficacious and more costly ocular hypotensive medications have gained significant market share, resulting in higher overall medication costs. Furthermore, after adjusting for compliance estimates, medication costs remained a substantial contributor to total direct cost of care, composing 38% to 52% of total cost (Table 4).
Unlike medication costs, which are generally distributed over a period of months or years, surgical costs are incurred at a single point. Yet, in the analysis, surgical costs are represented as average annual costs. For example, an incisional surgery with a unit cost of $5037,14 although incurred at a single point, will be represented as an annual cost of $1007, assuming a study analytic time horizon of 5 years within a given stage of disease. This approach risks the inappropriate estimation of actual surgical costs if the data collection is limited to a short time frame and/or to a relatively small sample size.
The results of this study support the hypothesis that resource use and direct cost of glaucoma management typically increase with worsening disease severity. Although this study appears to validate a GSS for retrospectively assessing and assigning disease severity, a larger evaluation in diverse practice settings would be required to measure GSS validity. To further support the findings from this study, a modeling approach with a societal perspective should be considered. Examining costs from a societal perspective, as opposed to a payer perspective, may have a significant impact on treatment cost, particularly for patients with end-stage disease, because many low-vision care and vision rehabilitation resources are nonreimbursable. Alternatively, sophisticated health policy models may be constructed to allow for investigation of different policy issues and options.26
As advances are made in the development of new therapies, these therapies are likely to be increasingly subject to economic analyses. Our pilot study indicates that delaying progression to later stages of disease is associated with a lower cost of care. As such, new therapies that slow disease progression, such as neuroprotective therapies, may cost more at an earlier stage of care but ultimately are associated with a reduced cost over the course of treatment. Data on the likelihood of progression and the ability of care to delay such progression, as well as understanding the costs of new interventions, will be needed to better understand these issues. This study sets the foreground for further research to better comprehend cost drivers in the treatment of glaucoma.
Correspondence: Paul P. Lee, MD, JD, Duke Eye Center, Erwin Road, Durham, NC 27710 (firstname.lastname@example.org).
Submitted for Publication: March 1, 2004; final revision received May 2, 2005; accepted May 2, 2005.
Financial Disclosure: None.
Funding/Support: This study was supported by an unrestricted grant from Allergan, Inc, Irvine, Calif.
Previous Presentation:Table 1 was presented at the Association for Research in Vision and Ophthalmology Annual Meeting; 2002; Fort Lauderdale, Fla.
Acknowledgment: We wish to thank Mohammad M. Najib, MBA, MPH, PhD, Lisa R. Siegartel, MPH, and Laura M. Katz, MPH, for their extensive statistical assistance; Karina Berenson, MPH, for designing electronic medical record review forms; and Gary Lebovics, BA, for structured medical record abstractions throughout the study.