Accuracy of Kalman Filtering in Forecasting Visual Field and Intraocular Pressure Trajectory in Patients With Ocular Hypertension | Glaucoma | JAMA Ophthalmology | JAMA Network
[Skip to Navigation]
Figure 1.  Forecast of Sample Patient With Ocular Hypertension From the Ocular Hypertension Treatment Study (OHTS) Using the Kalman Filter Model Parameterized Using Patients With Ocular Hypertension (KF-OHTN) Algorithm
Forecast of Sample Patient With Ocular Hypertension From the Ocular Hypertension Treatment Study (OHTS) Using the Kalman Filter Model Parameterized Using Patients With Ocular Hypertension (KF-OHTN) Algorithm

Actual values represent measurements from the OHTS, filtered values are self-corrected past forecasts, and predicted values represent forecast measurements. PI indicates 95% prediction interval (error bars).

Figure 2.  The Proportion of Mean Deviation (MD) Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension
The Proportion of Mean Deviation (MD) Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension

Shorter, fatter violin plots indicate greater concentration and less variation in prediction error compared with longer, thinner plots. The horizontal lines capture values within 0.5 dB (orange solid lines), 1.0 dB (black dotted lines), and 2.5 dB (black dashed and dotted lines) of the actual values. The white plus sign in the middle of each plot represents the mean, and the cyan rectangle indicates the median. KF-HTG indicates Kalman filter parameterized using patients with high-tension glaucoma; KF-OHTN, KF parameterized using patients from the Ocular Hypertension Treatment Study with ocular hypertension; LR1 and LR2, linear regression 1 and 2 models; and PM, personalized mean.

Table 1.  Description of Study Sample
Description of Study Sample
Table 2.  Proportion of Eyes From Patients in the Ocular Hypertension Treatment Study With Forecasts of MD Within 0.5, 1.0, and 2.5 dB and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months for Each of the 5 Forecasting Models
Proportion of Eyes From Patients in the Ocular Hypertension Treatment Study With Forecasts of MD Within 0.5, 1.0, and 2.5 dB and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months for Each of the 5 Forecasting Models
Table 3.  Comparison of the RMSE of the 5 Models at Forecasting MD, PSD, and IOP for All Patients in the Ocular Hypertension Treatment Study at 12, 24, 36, 48, and 60 Months
Comparison of the RMSE of the 5 Models at Forecasting MD, PSD, and IOP for All Patients in the Ocular Hypertension Treatment Study at 12, 24, 36, 48, and 60 Months
Supplement.

eTable 1. Description of Eyes From OHTS Which Progressed Due to Changes in VF and Did Not Progress Due to Changes in VF

eTable 2. Proportion of Eyes From OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models

eTable 3. Proportion of Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 mm Hg, and Beyond 5.0 mm Hg of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models

eTable 4. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Mean Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months into the Future for Each of the Five Forecasting Models

eTable 5. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models

eTable 6. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 dB, and Beyond 5.0 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models

eTable 7. Comparison of the Root Mean Square Error of the 5 Models at Forecasting Key Glaucoma Metrics for Progressed Eyes From OHTS Patients at 12, 24, 36, 48, and 60 Months Into the Future

eTable 8. Characteristics of Patients With Ocular Hypertension Stratified by Whether the KF-OHTN Model Accurately Forecasted Mean Deviation 60 Months Into the Future

eFigure 1. Forecast of Sample Patient With Ocular Hypertension From the OHTS Trial Comparing [A] KF-OHTN With PM and [B] KF-OHTN With LR1

eFigure 2. Violin Plots Showing the Proportion of MD Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension Who Progressed to Open-Angle Glaucoma

1.
Leibowitz  HM, Krueger  DE, Maunder  LR,  et al.  The Framingham Eye Study monograph: an ophthalmological and epidemiological study of cataract, glaucoma, diabetic retinopathy, macular degeneration, and visual acuity in a general population of 2631 adults, 1973-1975.  Surv Ophthalmol. 1980;24(suppl):335-610.PubMedGoogle ScholarCrossref
2.
Weinreb  RN, Aung  T, Medeiros  FA.  The pathophysiology and treatment of glaucoma: a review.  JAMA. 2014;311(18):1901-1911. doi:10.1001/jama.2014.3192PubMedGoogle ScholarCrossref
3.
Gordon  MO, Kass  MA.  The Ocular Hypertension Treatment Study: design and baseline description of the participants.  Arch Ophthalmol. 1999;117(5):573-583. doi:10.1001/archopht.117.5.573PubMedGoogle ScholarCrossref
4.
Lefferts  EJ, Markley  FL, Shuster  MD.  Kalman filtering for spacecraft attitude estimation.  J Guid Control Dyn. 1982;5:417-429. doi:10.2514/3.56190Google ScholarCrossref
5.
Eberle  C, Ament  C.  The unscented Kalman filter estimates the plasma insulin from glucose measurement.  Biosystems. 2011;103(1):67-72. doi:10.1016/j.biosystems.2010.09.012PubMedGoogle ScholarCrossref
6.
Lavieri  MS, Puterman  ML, Tyldesley  S, Morris  WJ.  When to treat prostate cancer patients based on their PSA dynamics.  IIE Trans Healthc Syst Eng. 2012;2:62-77. doi:10.1080/19488300.2012.666631Google ScholarCrossref
7.
Wang  Q, Molenaar  P, Harsh  S,  et al.  Personalized state-space modeling of glucose dynamics for type 1 diabetes using continuously monitored glucose, insulin dose, and meal intake: an extended Kalman filter approach.  J Diabetes Sci Technol. 2014;8(2):331-345. doi:10.1177/1932296814524080PubMedGoogle ScholarCrossref
8.
Schell  GJ, Lavieri  MS, Helm  JE,  et al.  Using filtered forecasting techniques to determine personalized monitoring schedules for patients with open-angle glaucoma.  Ophthalmology. 2014;121(8):1539-1546. doi:10.1016/j.ophtha.2014.02.021PubMedGoogle ScholarCrossref
9.
Garcia  GP, Nitta  K, Lavieri  MS,  et al.  Using Kalman filtering to forecast disease trajectory for patients with normal tension glaucoma.  Am J Ophthalmol. 2019;199:111-119. doi:10.1016/j.ajo.2018.10.012PubMedGoogle ScholarCrossref
10.
Ederer  F, Gaasterland  DE, Sullivan  EK; AGIS Investigators.  The Advanced Glaucoma Intervention Study (AGIS), 1: study design and methods and baseline characteristics of study patients.  Control Clin Trials. 1994;15(4):299-325. doi:10.1016/0197-2456(94)90046-9PubMedGoogle ScholarCrossref
11.
Musch  DC, Gillespie  BW, Lichter  PR, Niziol  LM, Janz  NK; CIGTS Study Investigators.  Visual field progression in the Collaborative Initial Glaucoma Treatment Study: the impact of treatment and other baseline factors.  Ophthalmology. 2009;116(2):200-207. doi:10.1016/j.ophtha.2008.08.051PubMedGoogle ScholarCrossref
12.
Krajewski  L, Ritzman  L, Malhotra  M. Forecasting demand. In:  Operations Management: Processes and Supply Chains. 10th ed. Boston, MA: Pearson; 2013:490-500.
13.
Bhapkar  VP.  A note on the equivalence of two test criteria for hypotheses in categorical data.  J Am Stat Assoc. 1966;61:228-235. doi:10.1080/01621459.1966.10502021Google ScholarCrossref
14.
Kazemian  P, Lavieri  MS, Van Oyen  MP, Andrews  C, Stein  JD.  Personalized prediction of glaucoma progression under different target intraocular pressure levels using filtered forecasting methods.  Ophthalmology. 2018;125(4):569-577. doi:10.1016/j.ophtha.2017.10.033PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    November 14, 2019

    Accuracy of Kalman Filtering in Forecasting Visual Field and Intraocular Pressure Trajectory in Patients With Ocular Hypertension

    Author Affiliations
    • 1Department of Industrial and Operations Engineering, University of Michigan College of Engineering, Ann Arbor
    • 2Department of Ophthalmology and Visual Sciences, University of Michigan Medical School, Ann Arbor
    • 3Center for Eye Policy and Innovation, University of Michigan, Ann Arbor
    • 4Ophthalmology and Visual Sciences, Washington University School of Medicine in St Louis, St Louis, Missouri
    • 5Department of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor
    JAMA Ophthalmol. 2019;137(12):1416-1423. doi:10.1001/jamaophthalmol.2019.4190
    Key Points

    Question  Can Kalman filtering, a machine learning approach, accurately forecast future values of mean deviation, pattern standard deviation, and intraocular pressure for patients with ocular hypertension?

    Findings  In this cohort study, the algorithm forecasts values of mean deviation on perimetry 5 years into the future within 1.0 decibel of the actual value for 61% of all eyes with ocular hypertension and 68% for the subset of eyes in which ocular hypertension eventually progressed to open-angle glaucoma.

    Meaning  This study’s findings indicate that machine learning approaches such as Kalman filtering have the potential to assist clinicians in caring for patients with ocular hypertension and in identifying patients whose ocular hypertension is more likely to progress to glaucoma.

    Abstract

    Importance  Techniques that properly identify patients in whom ocular hypertension (OHTN) is likely to progress to open-angle glaucoma can assist clinicians with deciding on the frequency of monitoring and the potential benefit of early treatment.

    Objective  To test whether Kalman filtering (KF), a machine learning technique, can accurately forecast mean deviation (MD), pattern standard deviation, and intraocular pressure values 5 years into the future for patients with OHTN.

    Design, Setting, and Participants  This cohort study was a secondary analysis of data from patients with OHTN from the Ocular Hypertension Treatment Study, performed between February 1994 and March 2009. Patients underwent tonometry and perimetry every 6 months for up to 15 years. A KF (KF-OHTN) model was trained, validated, and tested to assess how well it could forecast MD, pattern standard deviation, and intraocular pressure at up to 5 years, and the forecasts were compared with results from the actual trial. Kalman filtering for OHTN was compared with a previously developed KF for patients with high-tension glaucoma (KF-HTG) and 3 traditional forecasting algorithms. Statistical analysis for the present study was performed between May 2018 and May 2019.

    Main Outcomes and Measures  Prediction error and root-mean-square error at 12, 24, 36, 48, and 60 months for MD, pattern standard deviation, and intraocular pressure.

    Results  Among 1407 eligible patients (2806 eyes), 809 (57.5%) were female and the mean (SD) age at baseline was 57.5 (9.6) years. For 2124 eyes with sufficient measurements, KF-OHTN forecast MD values 60 months into the future within 0.5 dB of the actual value for 696 eyes (32.8%), 1.0 dB for 1295 eyes (61.0%), and 2.5 dB for 1980 eyes (93.2%). Among the 5 forecasting algorithms tested, KF-OHTN achieved the lowest root-mean-square error (1.72 vs 1.85-4.28) for MD values 60 months into the future. For the subset of eyes that progressed to open-angle glaucoma, KF-OHTN and KF-HTG forecast MD values 60 months into the future within 1 dB of the actual value for 30 eyes (68.2%; 95% CI, 54.4%-82.0%) and achieved the lowest root-mean-square error among all models.

    Conclusions and Relevance  These findings suggest that machine learning algorithms such as KF can accurately forecast MD, pattern standard deviation, and intraocular pressure 5 years into the future for many patients with OHTN. These algorithms may aid clinicians in managing OHTN in their patients.

    Introduction

    Ocular hypertension (OHTN) affects an estimated 3 to 6 million individuals in the United States1 and is a well-established risk factor for open-angle glaucoma (OAG).2 However, according to landmark clinical trials, including the Ocular Hypertension Treatment Study (OHTS),3 few patients with OHTN experience progression to OAG. Although lowering intraocular pressure (IOP) reduces the risk of conversion to OAG,3 common IOP-lowering interventions are associated with adverse effects or are costly. Therefore, if clinicians can more effectively assess which patients with OHTN are at greatest risk for OAG, they can more closely monitor and treat that subset of patients while safely monitoring without treatment many others who have a lower risk.

    Kalman filtering (KF), a machine learning approach, is frequently used in commercial aviation to forecast flight paths from one destination to another.4 Recently, researchers have applied KF to forecast disease trajectory dynamics for patients with chronic diseases.5-8 Kalman filtering initially relies on the disease dynamics of an underlying population to generate its first few forecasts. However, the more KF learns about the unique dynamics of an individual patient, the less it prioritizes information from the population and the more it prioritizes the disease dynamics of the actual patient. This permits KF to generate a personalized forecast of each patient’s disease trajectory and how it changes over time. Kalman filtering has been successfully applied in forecasting disease progression for patients with high-tension glaucoma (HTG)8 and normal-tension glaucoma.9

    Here we study how well KF can forecast values of mean deviation (MD), pattern standard deviation (PSD), and IOP for patients with OHTN, and whether the same model previously created for HTG can effectively forecast progression for patients with OHTN or whether a different KF model is better suited for this new population. Thus, we developed 2 KF models: 1 trained, calibrated, and parameterized using data from patients with HTG (KF-HTG) and 1 using data from patients with OHTN (KF-OHTN). We compared these 2 models with 3 standard forecasting methods to assess which of the 5 prediction algorithms most accurately forecasts visual field global indexes and IOP dynamics up to 60 months into the future for patients with OHTN.

    Methods
    Data Source

    We used data from the OHTS trial.3 The OHTS, performed between February 1994 and March 2009, involved 1636 patients with OHTN from 22 US centers. Patients were randomized to treatment with IOP-lowering medications or observation and were followed up for as long as 15 years by using standardized measurements of tonometry and perimetry at baseline and every 6 months. The trial sought to identify differences in conversion rates from OHTN to OAG between the randomization groups based on evidence of glaucomatous optic neuropathy or visual field loss, which was adjudicated as to cause by an end point committee review. This cohort study, a secondary analysis of data from the OHTS, was approved by the University of Michigan and Washington University institutional review boards, which waived the need for patient informed consent because all data were deidentified before we obtained them.

    Sample Selection

    To be eligible, OHTS participants had baseline IOPs of 21 to 32 mm Hg in both eyes, reliable and normal 30-2 Humphrey visual fields (Carl Zeiss Meditec), and no detectable evidence of glaucoma based on optic nerve evaluation.3 We included the eyes of all persons with 6 or more Swedish interactive threshold algorithm standard or full-threshold Humphrey visual fields and 6 IOP measurements on separate dates over a span of 2 or more years. These 6 measurements ensured that we could forecast using at least 3 warm-up measurements and validate at least 12 months into the future. Patients were censored when they experienced nonglaucomatous visual field loss or underwent any incisional intraocular surgery other than uncomplicated cataract surgery.

    Statistical Analysis

    All statistical analyses were conducted between May 2018 and May 2019 using MatLab, version R2016b (MathWorks). Participant chracteristics were summarized for the entire sample by using means and SDs for continuous variables and frequencies and percentages for categorical variables. Participants with OHTN and HTG were compared using the independent-samples t test for continuous variables and the Pearson χ2 test for categorical variables, with 2-sided P < .05 indicating statistical significance. We also computed 95% Wald CIs for the proportions of patients with predicted values of MD, PSD, and IOP within 0.5, 1.0, and 2.5 dB of the actual value.

    Forecasting MD, PSD, and IOP Values
    Kalman Filter

    When modeling disease trajectories, KF combines population-level and patient-level disease dynamics to produce personalized forecasts of each model variable. Unlike traditional forecasting techniques, KF uses past prediction errors to correct its future estimates, dynamically updating forecasts as new measurements become available. We used KF to forecast future MD, PSD, and IOP values based on their current values and rates of change. Kalman filtering learns each patient’s dynamics before it begins to forecast. Our analyses required 6 previous sets of readings before forecasting. We also performed an ancillary analysis using 3 sets of initial readings to assess whether KFs could effectively forecast future values with fewer initial inputs.

    Data Elements

    We obtained demographic information (ie, sex, age, and race) and MD, PSD, and IOP measurements for each eye beginning at study randomization and ending at last follow-up visit or censoring. We parameterized and trained our KF model (KF-OHTN) using MD and PSD on perimetry and IOP measurements, along with their velocities and accelerations. Because KF requires relatively evenly spaced time intervals between consecutive measurements, we performed linear interpolation when necessary. Little interpolation was required because most patient visits adhered closely to OHTS protocol visit windows.

    Comparison Models
    High-Tension Glaucoma Kalman Filter

    Previously, we parameterized a KF (KF-HTG) using data from the Advanced Glaucoma Intervention Study10 and the Collaborative Initial Glaucoma Treatment Study.8 Briefly, the Advanced Glaucoma Intervention Study studied the efficacy of different treatments for advanced glaucoma, and the Collaborative Initial Glaucoma Treatment Study11 studied patients with early to moderate OAG who were randomized to initial treatment with medication or surgery. Patients from the Advanced Glaucoma Intervention Study and the Collaborative Initial Glaucoma Treatment Study are hereinafter referred to as the HTG group. We assessed how well KF-HTG could forecast MD, PSD, and IOP among patients with OHTN.

    Personalized Mean Model

    The personalized mean (PM) model assumes that MD, PSD, and IOP remain constant at the mean value of the initial 6 readings. For example, consider a patient with initial MD readings of −0.6, −0.9, −1.1, −2.2, −1.0, and −2.0 dB. The PM estimates that MD at 60 months into the future will be −1.3 dB.

    Linear Regression Models

    Linear regression models assume a linear relationship between the variable of interest (eg, MD) for each time period. We created 2 models: a simple linear regression model (LR1) and an econometric forecasting model12 (LR2). Compared with LR1, LR2 has the same slope but instead passes through the most recent observation (ie, LR2 has a different intercept). These models are described in detail elsewhere.9

    For all 5 models, we forecast MD, PSD, and IOP values at 12, 24, 36, 48, and 60 months into the future for all eligible patients with OHTN. Because most patients with OHTN have stable disease over time, we hypothesized that one could adequately forecast future MD, PSD, and IOP values for most patients by assuming no change over time. However, some patients with OHTN ultimately experience progression to OAG, and identifying these patients early is helpful for clinical decision-making. Therefore, a specific aim of our study was to perform a separate analysis investigating the models’ performance on the subset of eyes progressing from OHTN to OAG owing to changes in visual field as determined by the OHTS Endpoint Committee.

    Performance Measures

    We computed MD, PSD, and IOP prediction errors at 12, 24, 36, 48, and 60 months (ie, the difference between the forecast and the value reported in OHTS) as our primary measure of model performance. Because KF-OHTN required entire patient trajectories for parameterization, we used leave-1-out cross-validation to estimate its performance. That is, for each patient with OHTN, we parameterized KF-OHTN by using all eyes in our sample except that patient’s eyes. Then, we forecast glaucoma dynamics for each eye of that patient with KF-OHTN. Because KF-HTG was trained using patients with HTG, and PM, LR1, and LR2 were trained using only past patient readings, leave-1-out cross-validation was unnecessary for these models.

    We analyzed the distribution and magnitude of prediction errors when forecasting 12, 24, 36, 48, and 60 months into the future. To study the distribution of errors, we determined the proportion of MD, PSD, and IOP prediction errors falling within clinically relevant boundaries. For MD and PSD, these thresholds were 0.5, 1.0, and 2.5 dB from the actual value; for IOP, they were 1.0, 2.5, and 5.0 mm Hg from the actual value. We compared prediction error distributions across models by using the Bhapkar test for equality of marginal distributions13 and illustrated these distributions using violin plots. To evaluate prediction error magnitude for each model, we computed root-mean-square error, which estimates the typical deviation between observations and forecasts (eg, for MD, a root-mean-square error equal to 0.8 dB indicates that the actual values will be approximately 0.8 dB greater or less than the forecasts, on average). Lower root-mean-square error values indicate better estimations.

    Outlier Analysis

    We identified eyes of patients for which KF-OHTN forecast MD 2.5 dB or less (ie, accurate) and greater than 2.5 dB (ie, outliers) from the actual value when forecasting 60 months into the future. We separated outliers into those with forecast values 2.5 dB greater than the actual value (ie, high outliers) and lower than the actual value (ie, low outliers). We compared the accurate group, high outliers, and low outliers on baseline MD, PSD, and IOP by using analysis of variance.

    Results
    Study Sample

    From OHTS, there were 2806 eyes from 1407 eligible patients with OHTN. The mean (SD) age at baseline was 57.5 (9.6) years. Among these patients, 809 (57.5%) were female; the racial composition (by self-identification) included 971 white participants (69.0%), 357 black participants (25.4%), and 79 participants (5.6%) classified as another race. Mean (SD) follow-up was 9.8 (3.7) years. Of the 2806 eyes, 2124 (75.7%) had sufficient measurements to forecast 60 months into the future. Overall, 108 eyes (7.7%) were identified by the OHTN Endpoint Committee as progressing from OHTN to OAG based on perimetry results. The mean (SD) time to conversion was 6.8 (3.7) years (Table 1 and eTable 1 in the Supplement).

    We used 601 eyes from 601 patients with HTG to develop KF-HTG. Of these patients, 324 (53.9%) came from the Advanced Glaucoma Intervention Study and 277 (46.1%) from the Collaborative Initial Glaucoma Treatment Study. These patients had a mean (SD) age of 63.1 (10.9) years. Three hundred ten patients (51.6%) were female, 275 (45.8%) were white, 304 (50.6%) were black, and 22 persons (3.7%) reported being of another race. Mean (SD) follow-up was 6.3 (2.8) years. At baseline, the eyes with OHTN and HTG differed significantly on mean (SD) MD (0.0 [1.4] vs −7.5 [5.6] dB), PSD (2.0 [0.5] vs 6.4 [3.7] dB), and IOP (25.2 [3.0] vs 17.7 [4.2] mm Hg) (P < .001 for all) (Table 1).

    Forecasting MD, PSD, and IOP With Kalman Filter Models

    We forecast future MDs, PSDs, and IOPs at 12, 24, 36, 48, and 60 months for each eligible eye with OHTN. Figure 1 illustrates the forecasting of a sample patient from OHTS by using KF-OHTN. Comparisons among KF-OHTN, PM, and LR1 are shown in eFigure 1 in the Supplement. When MD was forecast 60 months into the future, KF-OHTN forecast MD within 0.5 dB for 696 eyes (32.8%; 95% CI, 30.8%-34.8%), within 1.0 dB for 1295 eyes (61.0%; 95% CI, 58.9%-63.0%), and within 2.5 dB for 1980 eyes (93.2%; 95% CI, 92.2%-94.3%). For 144 eyes, (6.8%; 95% CI, 5.7%-7.8%) KF-OHTN estimated an MD greater than 2.5 dB of the actual value. Similarly, KF-HTG estimated an MD within 0.5 dB for 578 eyes (27.2%; 95% CI, 25.3%-29.1%), within 1.0 dB for 1085 eyes (51.1%; 95% CI, 49.0%-53.2%), and within 2.5 dB for 1911 eyes (90.0%; 95% CI, 88.7%-91.2%). The personalized mean model also forecast MD well for many patients, whereas LR1 and LR2 did not (Table 2). The personalized mean model estimated an MD within 0.5 dB of the actual value for 728 eyes (34.3%; 95% CI, 32.3%-36.3%), within 1.0 dB for 1309 eyes (61.6%; 95% CI, 59.6%-63.7%), and within 2.5 dB for 1980 eyes (93.2%; 95% CI, 92.2%-94.3%). The LR1 model estimated an MD within 0.5 dB for 242 eyes (11.4%; 95% CI, 10.0%-12.7%), within 1.0 dB for 484 eyes (22.8%; 95% CI, 21.0%-24.6%), and within 2.5 dB for 1173 eyes (55.2%; 95% CI, 33.1%-57.3%). The LR2 model estimated an MD within 0.5 dB of the actual value for 246 eyes (11.4%; 95% CI, 10.2%-12.9%), within 1.0 dB for 496 eyes (23.4%; 95% CI, 21.6%-25.2%), and within 2.5 dB for 1216 eyes (57.3%; 95% CI, 55.1%-59.4%). Figure 2 compares the distribution of MD prediction errors by using each model when forecasting 12 to 60 months into the future for the entire OHTS sample. Shorter and fatter violin plots indicate greater concentration and less variation in prediction error compared with longer, thinner plots. For PSD, KF-OHTN forecast 1402 eyes (66.0%; 95% CI, 64.0%-68.0%) within 0.5 dB, 1914 eyes (90.1%; 95% CI, 88.8%-91.4%) within 1.0 dB, and 2082 eyes (98.0%; 95% CI, 97.4%-98.6%) within 2.5 dB of the actual value (eTable 2 in the Supplement). For IOP, KF-OHTN forecast within 1.0 mm Hg of the actual value in 560 eyes (26.4%; 95% CI, 24.5%-28.3%), within 2.5 mm Hg of the actual value in 1255 eyes (59.1%; 95% CI, 57.0%-61.2%), and within 5 mm Hg of the actual value in 1854 eyes (87.3%; 95% CI, 85.9%-88.7%), and the KF-HTG forecast 517 eyes (24.3%; 95% CI, 22.5%-26.1%) within 0.5 mm Hg, 1118 eyes (52.6%; 95% CI, 50.5%-54.7%) within 1.0 mm Hg, and 1703 eyes (80.2%; 95% CI, 78.5%-81.9%) within 5 mm Hg of the actual value (eTable 3 in the Supplement).

    We replicated these analyses on the subset of eligible eyes that progressed from OHTN to OAG owing to visual field changes. When MD values were forecast 60 months into the future (44 eyes), KF-OHTN and KF-HTG performed identically: within 0.5 dB of the actual value for 13 eyes (29.5%; 95% CI, 16.0%-43.3%), within 1 dB of the actual value for 30 eyes (68.2%; 95% CI, 54.4%-82.0%) and within 2.5 dB of the actual value for 41 eyes (93.2%; 95% CI, 85.8%-100%). By comparison, PM forecast within 0.5 dB of the actual value in 7 eyes (15.9%; 95% CI, 5.1%-26.7%), within 1 dB in 22 eyes (50.0%; 95% CI, 35.2%-64.8%), and within 2.5 dB in 41 eyes (93.2%; 95% CI, 85.8%-100%) (eTable 4 and eFigure 2 in the Supplement). For this subset of eyes with progression, KF-OHTN and KF-HTG also forecast similarly for PSDs and IOPs 60 months into the future (eTables 5 and 6 in the Supplement). For PSD, KF-OHTN and KF-HTG both forecast within 2.5 dB for all 44 eyes (100%; 95% CI, 100%-100%). For IOP, KF-OHTN forecast within 5 mm Hg for 34 eyes (77.3%; 95% CI, 64.9%-88.7%) and KF-HTG for 29 eyes (65.9%; 95% CI, 51.9%-79.9%).

    As an ancillary analysis, we assessed how well the KF models forecast MD, PSD, and IOP up to 60 months using 3 initial observations as warm-up instead of 6. The use of 3 initial observations resulted in only modest increases in forecasting error (data unreported).

    Comparison of Magnitude of Prediction Errors

    When MD values were forecast 60 months into the future, KF-OHTN had a lower root-mean-square error (1.72) than the other models (1.85-4.28). Similar findings were observed for IOP (root-mean-square error for KF-OHTN, 3.42; for other models, 4.08-8.25). For PSD, the KF-OHTN and PM models achieved the lowest root-mean-square error (1.04 and 1.00, respectively), followed by KF-HTG (1.48) and the LR models (2.01 for LR1 and 2.00 for LR2) (Table 3). For eyes with OHTN that progressed to glaucoma, KF-OHTN (root-mean-square error, 1.20) outperformed PM (1.40), LR1 (3.40), and LR2 (3.28) at forecasting 60 months into the future but did not outperform KF-HTG (1.19) (eTable 7 in the Supplement).

    Outlier Analysis

    Using KF-OHTN, there were 88 high outliers (forecast value was ≥2.5 dB greater than the actual value at 60 months) (61.1%) and 56 low outliers (forecast value was ≥2.5 dB lower than the actual value at 60 months) (38.9%). There were no differences between the accurately forecast group and the 2 outlier groups with respect to mean initial MD, PSD, or IOP (eTable 8 in the Supplement).

    Discussion

    We applied KF, a machine learning technique, to generate personalized forecasts of future MD, PSD, and IOP values for patients with OHTN. A KF model built using a sample of patients with ocular hypertension from OHTS successfully forecast MD within 0.5 dB of the actual value at 60 months for nearly one-third of the patients with OHTN and within 1.0 dB of the actual value for more than 60% of the patients. Although KF-OHTN outperformed the other models on most metrics studied, we found that another KF that was previously parameterized and validated on patients with HTG (KF-HTG)8 also did reasonably well at forecasting disease trajectory for most patients with OHTN. Although the PM model also performed relatively well on the overall sample, it was notably less accurate than the KF models on the small, albeit important subset of patients who experienced progression from OHTN to OAG.

    The simple PM model, which takes the mean of the initial few measurements, performed well when forecasting MD, PSD, and IOP for the entire sample of patients with OHTN. This result is unsurprising because most patients with OHTN never experience progression to OAG. Although the PM model’s simplicity is appealing, it underestimates future MD and PSD values for patients with OHTN who experience progression to OAG because it assumes that future perimetric parameters are similar to their initial values. Because it is unknown a priori which patients with OHTN will eventually develop OAG, our analysis shows the benefits of using KF over models like PM. Furthermore, past studies using other patient populations with more advanced disease have shown that simple models like PM do not forecast progression well.9

    Compared with other forecasting algorithms, KF has the advantage of integrating new data into its forecasts over time. Therefore, although initial forecasts are heavily influenced by the underlying population, KF becomes more influenced by the dynamics of the actual patient as additional measurements are obtained. This likely explains why KF-HTG—which was developed using patients with moderate to severe OAG—accurately estimated the trajectory of many patients with OHTN. We suspect that KF-HTG quickly learned that many patients with OHTN did not experience changes in MD or PSD at IOP levels that were associated with disease progression for many patients with HTG. Likewise, a recently published complementary study9 found that KF-HTG successfully forecast the trajectory of many patients with normal-tension glaucoma, again showing the adaptability of these algorithms. Although additional studies are needed to test and validate our KFs on other glaucoma types and severities, it is possible that a single KF algorithm can accurately forecast disease trajectory across the disease spectrum, from OHTN to advanced OAG. Practically speaking, it would be easier to implement into clinical practice a single KF model that could forecast for all patients with OHTN and OAG instead of selecting from multiple algorithms depending on a patient’s glaucoma type and severity.

    We evaluated the performance of our KF models by using 3 instead of 6 initial observations. Although the use of 6 observations resulted in more accurate forecasts, the improvement was modest. Algorithms that can begin forecasting with fewer measurements are preferable because fewer data points are required before the model can be used, allowing it to forecast for more patients.

    We hope soon to incorporate our KFs into perimeters or electronic health record software to generate automated forecasts delivered in real time to clinicians to aid with clinical decision-making. We have already developed tools involving KFs that clinicians could use to assess the optimal frequency for performing diagnostic testing to monitor OAG8 and identify rates of deterioration of MD at different target IOP levels.14 Similar tools could be created for patients with OHTN based on KF-OHTN.

    Limitations

    Our study has limitations. First, OHTS collected data on the optic nerve appearance to assess for conversion to OAG3; however, because these evaluations were qualitative and not quantitative, they could not be integrated into our KF models. Second, in OHTS, the intervals between study visits were nearly evenly spaced. In clinical practice, timing between visits often varies. Because KF models require evenly spaced measurements, future studies must extend this method to handle irregularly spaced intervals. The precision of KF models may be greater in clinical trials that standardize and monitor the quality of measurements.

    Conclusions

    We trained and validated a KF model to accurately forecast MD, PSD, and IOP values to 60 months. The estimated values from our model aligned with the actual values for most patients we studied. Integrating findings from machine learning algorithms, such as those presented herein, into clinical decision-making may enhance clinicians’ ability to make more informed decisions regarding which patients with OHTN would most benefit from treatment.

    Back to top
    Article Information

    Accepted for Publication: August 24, 2019.

    Corresponding Author: Joshua D. Stein, MD, MS, W.K. Kellogg Eye Center, Department of Ophthalmology and Visual Sciences, University of Michigan, 1000 Wall St, Ann Arbor, MI 48105 (jdstein@med.umich.edu).

    Published Online: November 14, 2019. doi:10.1001/jamaophthalmol.2019.4190

    Author Contributions: Dr Stein had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Lavieri, Liu, Van Oyen, Kass, Stein.

    Acquisition, analysis, or interpretation of data: Garcia, Lavieri, Andrews, Liu, Gordon.

    Drafting of the manuscript: Garcia, Lavieri, Liu, Stein.

    Critical revision of the manuscript for important intellectual content: Garcia, Lavieri, Andrews, Van Oyen, Kass, Gordon.

    Statistical analysis: Garcia, Lavieri, Andrews, Liu.

    Obtained funding: Lavieri, Van Oyen.

    Administrative, technical, or material support: Van Oyen, Gordon, Stein.

    Supervision: Lavieri, Andrews, Kass.

    Conflict of Interest Disclosures: Dr Lavieri reported receiving grants from the National Science Foundation, and Drs Lavieri, Gordon, and Stein reported receiving grants from the National Eye Institute (NEI). No other disclosures were reported.

    Funding/Support: This research was supported by grants EY009307 (Dr Kass), EY009341 (Dr Gordon), and R01EY026641 (Drs Lavieri and Stein) from the NEI.

    Role of the Funder/Sponsor: The NEI had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: David Musch, PhD (University of Michigan), provided access to Collaborative Initial Glaucoma Treatment Study data, and Paul C. Van Veldhuisen, MS (Emmes Corp), provided access to Advanced Glaucoma Intervention Study data. Neither was financially compensated for these contributions.

    References
    1.
    Leibowitz  HM, Krueger  DE, Maunder  LR,  et al.  The Framingham Eye Study monograph: an ophthalmological and epidemiological study of cataract, glaucoma, diabetic retinopathy, macular degeneration, and visual acuity in a general population of 2631 adults, 1973-1975.  Surv Ophthalmol. 1980;24(suppl):335-610.PubMedGoogle ScholarCrossref
    2.
    Weinreb  RN, Aung  T, Medeiros  FA.  The pathophysiology and treatment of glaucoma: a review.  JAMA. 2014;311(18):1901-1911. doi:10.1001/jama.2014.3192PubMedGoogle ScholarCrossref
    3.
    Gordon  MO, Kass  MA.  The Ocular Hypertension Treatment Study: design and baseline description of the participants.  Arch Ophthalmol. 1999;117(5):573-583. doi:10.1001/archopht.117.5.573PubMedGoogle ScholarCrossref
    4.
    Lefferts  EJ, Markley  FL, Shuster  MD.  Kalman filtering for spacecraft attitude estimation.  J Guid Control Dyn. 1982;5:417-429. doi:10.2514/3.56190Google ScholarCrossref
    5.
    Eberle  C, Ament  C.  The unscented Kalman filter estimates the plasma insulin from glucose measurement.  Biosystems. 2011;103(1):67-72. doi:10.1016/j.biosystems.2010.09.012PubMedGoogle ScholarCrossref
    6.
    Lavieri  MS, Puterman  ML, Tyldesley  S, Morris  WJ.  When to treat prostate cancer patients based on their PSA dynamics.  IIE Trans Healthc Syst Eng. 2012;2:62-77. doi:10.1080/19488300.2012.666631Google ScholarCrossref
    7.
    Wang  Q, Molenaar  P, Harsh  S,  et al.  Personalized state-space modeling of glucose dynamics for type 1 diabetes using continuously monitored glucose, insulin dose, and meal intake: an extended Kalman filter approach.  J Diabetes Sci Technol. 2014;8(2):331-345. doi:10.1177/1932296814524080PubMedGoogle ScholarCrossref
    8.
    Schell  GJ, Lavieri  MS, Helm  JE,  et al.  Using filtered forecasting techniques to determine personalized monitoring schedules for patients with open-angle glaucoma.  Ophthalmology. 2014;121(8):1539-1546. doi:10.1016/j.ophtha.2014.02.021PubMedGoogle ScholarCrossref
    9.
    Garcia  GP, Nitta  K, Lavieri  MS,  et al.  Using Kalman filtering to forecast disease trajectory for patients with normal tension glaucoma.  Am J Ophthalmol. 2019;199:111-119. doi:10.1016/j.ajo.2018.10.012PubMedGoogle ScholarCrossref
    10.
    Ederer  F, Gaasterland  DE, Sullivan  EK; AGIS Investigators.  The Advanced Glaucoma Intervention Study (AGIS), 1: study design and methods and baseline characteristics of study patients.  Control Clin Trials. 1994;15(4):299-325. doi:10.1016/0197-2456(94)90046-9PubMedGoogle ScholarCrossref
    11.
    Musch  DC, Gillespie  BW, Lichter  PR, Niziol  LM, Janz  NK; CIGTS Study Investigators.  Visual field progression in the Collaborative Initial Glaucoma Treatment Study: the impact of treatment and other baseline factors.  Ophthalmology. 2009;116(2):200-207. doi:10.1016/j.ophtha.2008.08.051PubMedGoogle ScholarCrossref
    12.
    Krajewski  L, Ritzman  L, Malhotra  M. Forecasting demand. In:  Operations Management: Processes and Supply Chains. 10th ed. Boston, MA: Pearson; 2013:490-500.
    13.
    Bhapkar  VP.  A note on the equivalence of two test criteria for hypotheses in categorical data.  J Am Stat Assoc. 1966;61:228-235. doi:10.1080/01621459.1966.10502021Google ScholarCrossref
    14.
    Kazemian  P, Lavieri  MS, Van Oyen  MP, Andrews  C, Stein  JD.  Personalized prediction of glaucoma progression under different target intraocular pressure levels using filtered forecasting methods.  Ophthalmology. 2018;125(4):569-577. doi:10.1016/j.ophtha.2017.10.033PubMedGoogle ScholarCrossref
    ×