Development and External Validation of a Mortality Prediction Model for Community-Dwelling Older Adults With Dementia

Key Points

Question  What is the accuracy of a novel model for predicting time to death in community-dwelling older adults with dementia?

Findings  We used 2 nationally representative cohorts including a total of 6671 community-dwelling older adults with dementia from 1998 to 2016 (n = 4267) and 2011 to 2019 (n = 2404) to develop and externally validate a mortality prediction model. The final model included readily available clinical predictors (demographics, health factors, functional measures, and chronic conditions) to predict mortality from 1 to 10 years with good discrimination and calibration.

Meaning  Mortality predictions using this prognostic model may help inform conversations between clinicians, patients, and families related to advance care planning and clinical decisions such as cancer screening.

Importance  Estimating mortality risk in older adults with dementia is important for guiding decisions such as cancer screening, treatment of new and chronic medical conditions, and advance care planning.

Objective  To develop and externally validate a mortality prediction model in community-dwelling older adults with dementia.

Design, Setting, and Participants  This cohort study included community-dwelling participants (aged ≥65 years) in the Health and Retirement Study (HRS) from 1998 to 2016 (derivation cohort) and National Health and Aging Trends Study (NHATS) from 2011 to 2019 (validation cohort).

Exposures  Candidate predictors included demographics, behavioral/health factors, functional measures (eg, activities of daily living [ADL] and instrumental activities of daily living [IADL]), and chronic conditions.

Main Outcomes and Measures  The primary outcome was time to all-cause death. We used Cox proportional hazards regression with backward selection and multiple imputation for model development. Model performance was assessed by discrimination (integrated area under the receiver operating characteristic curve [iAUC]) and calibration (plots of predicted and observed mortality).

Results  Of 4267 participants with probable dementia in HRS, the mean (SD) age was 82.2 (7.6) years, 2930 (survey-weighted 69.4%) were female, and 785 (survey-weighted 12.1%) identified as Black. Median (IQR) follow-up time was 3.9 (2.0-6.8) years, and 3466 (81.2%) participants died by end of follow-up. The final model included age, sex, body mass index, smoking status, ADL dependency count, IADL difficulty count, difficulty walking several blocks, participation in vigorous physical activity, and chronic conditions (cancer, heart disease, diabetes, lung disease). The optimism-corrected iAUC after bootstrap internal validation was 0.76 (95% CI, 0.75-0.76) with time-specific AUC of 0.73 (95% CI, 0.70-0.75) at 1 year, 0.75 (95% CI, 0.73-0.77) at 5 years, and 0.84 (95% CI, 0.82-0.85) at 10 years. On external validation in NHATS (n = 2404), AUC was 0.73 (95% CI, 0.70-0.76) at 1 year and 0.74 (95% CI, 0.71-0.76) at 5 years. Calibration plots suggested good calibration across the range of predicted risk from 1 to 10 years.

Conclusions and Relevance  We developed and externally validated a mortality prediction model in community-dwelling older adults with dementia that showed good discrimination and calibration. The mortality risk estimates may help guide discussions regarding treatment decisions and advance care planning.

Dementia is a leading cause of morbidity and mortality worldwide and is associated with increased mortality risk.^1-3 An estimated 6.5 million Americans aged 65 years or older are living with Alzheimer disease and related dementias in 2022.^4,5 The clinical course of individuals with dementia is highly variable, with median survival time from age at diagnosis ranging from 3.3 years to 11.7 years.⁶ Heterogeneity in prognosis has important implications for decisions related to financial planning, advance care planning, long-term care admission, and clinical care. For interventions that have immediate risks or burdens and delayed benefits, such as cancer screening and tight glycemic control in patients with diabetes, life expectancy can help differentiate between patients most likely to benefit vs most likely to be harmed by interventions.⁷ Therefore, accurate estimates of life expectancy in people with dementia are important for providing the foundational data for patients, family, and clinicians to help define patient-centered care goals in their remaining lifetime.^7,8

Mortality prediction models are one tool that can help estimate an individual’s prognosis. Several non–disease-specific mortality indices have been developed among community-dwelling older adults, and they are readily available on websites such as ePrognosis.^9-12 However, their utility may be limited if the populations in which they were developed were not entirely inclusive of individuals with conditions that have a dramatic effect on mortality (eg, dementia or advanced cancer).⁹

Several mortality prediction models have been developed specifically in individuals with dementia.¹³ The most well known is the Advanced Dementia Prognostic Tool (ADEPT), which estimates 6-month mortality in nursing home patients with advanced dementia.^14,15 However, for the approximately 70% of older adults living with dementia in the community, there are currently no well-established mortality models.⁴ This population is more heterogenous in terms of cognition, comorbidities, and daily functioning compared with a nursing home population. Previously published models of community-dwelling older adults with dementia have several limitations, including small sample sizes, outdated data, lack of validation, and poor analytic strategies.¹³

Therefore, the aim of this study was to develop and externally validate a mortality prediction model in 2 nationally representative cohorts of community-dwelling older adults with dementia in the US. Information from this model can be used to facilitate discussions regarding clinical care and advance care planning.

This study’s reporting was guided by the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines (eTable 1 in the Supplement).¹⁶

We used data from the Health and Retirement Study (HRS) between 1998 and 2016, a nationally representative survey of US adults aged 51 years or older in which participants were interviewed every 2 years.¹⁷ For individuals who could not participate in the interview (eg, physical and/or cognitive reasons), HRS performed proxy interviews typically with a spouse or other family member to help reduce attrition bias.¹⁸ We included information from both sample respondents and proxy interviews.

We included community-dwelling older adults aged 65 years or older with probable dementia ascertained by a previously validated algorithm (eMethods in the Supplement).¹⁹ This algorithm has high accuracy compared with gold-standard dementia diagnoses from the Aging, Demographics, and Memory Study.^19-21 Participants were included from the date of interview when predicted probability of dementia exceeded 0.5.

For external validation, we used the National Health and Aging Trends Study (NHATS), a nationally representative longitudinal study of Medicare beneficiaries aged 65 years or older from 2011 to 2019.²² We included individuals classified as having probable dementia based on defined cutoffs (eMethods in the Supplement).²³

The primary outcome was all-cause mortality. Information on mortality in HRS was collected until May 31, 2019, in multiple ways to ensure accurate and complete ascertainment. We primarily used information from the HRS Tracker File, which ascertains mortality through exit interviews with next of kin, and the National Death Index (NDI). This method has been shown to be highly accurate and effectively complete.²⁴ To further increase accuracy, we searched Medicare data files (Denominator File, Master Beneficiary Summary File, and Inpatient files) and the NDI.

Candidate predictor variables were identified through previously published systematic reviews of risk factors for mortality in individuals with dementia and performing an independent review of mortality models for community-dwelling individuals with dementia (eMethods in the Supplement).^6,13,25-29 All candidate predictors were collected at the time of the HRS interview wave when dementia probability was greater than 0.5. Detailed definitions of all candidate variables are provided in eTable 2 in the Supplement.

Briefly, predictors included demographic factors, such as age category (65-69, 70-74, 75-79, 80-84, 85-89, ≥90 years), sex, and marital status. Health and/or behavioral factors included body mass index (BMI, calculated as weight in kilograms divided by height in meters squared) category (<18.5, 18.5 to <25, 25 to <30, and ≥30), smoking status (never smoker, former smoker, current smoker), and participation in vigorous physical activity (hardly ever/never vs any vigorous physical activity). Functional status was assessed by count of difficulties or dependencies with activities of daily living (ADL, including dressing, bathing, eating, getting in/out of bed, and toileting with range 0-5), and instrumental activities of daily living (IADLs, including preparing meals, managing medications, managing money, grocery shopping, and using telephone with range 0-5). In general, difficulty and dependency with individual ADLs/IADLs were defined by self or proxy report of difficulty performing the task or needing help with the task, respectively. Self-reported comorbidities included cancer (excluding minor skin cancer), diabetes, heart disease, lung disease, hypertension, and stroke. Definitions of these predictors largely overlapped in NHATS (eTable 3 in the Supplement).

We used Cox proportional hazards regression with all candidate predictors to develop the model. To create a parsimonious model that can more efficiently be used in clinical practice, we performed variable selection using backward selection. Because this process may lead to a final set of predictors that is unstable, we combined the backward selection process with bootstrapping and multiple imputation.^30-33 Because some variables had incomplete baseline data, we first performed multiple imputation (MI) assuming the data were missing at random to create 20 imputed data sets. We created 30 bootstrap samples for each of the 20 imputed data sets. We performed a backward selection process with a P value of <.05 for elimination in each of the imputed data sets within each bootstrap sample. Variables that were selected in more than 60% of the 20 imputed data sets (≥12 times) and more than 50% of the 30 bootstrap samples (≥15 times) were included. We explored different percentage thresholds based on previous research without any significant changes in predictors included.^30,34 To account for the complex survey design of the HRS, we incorporated survey weights and strata at each stage. The final model coefficients were obtained by pooling results across imputations. To evaluate the effect of proxy responses, we included a proxy variable as a candidate predictor and separately evaluated models in proxy and nonproxy respondents. Because the results were not substantially different with either method, we reported the pooled model without an indicator for proxy status. We also performed a sensitivity analysis only including incident cases of probable dementia.

Model performance was assessed through discrimination, calibration, and the Brier score. For discrimination, we calculated the integrated area under the receiver operating characteristic curve (iAUC), which averages all available AUC statistics over time.^35-37 We also reported time-specific values at clinically relevant time points of 1, 2, 5, and 10 years. Calibration refers to the agreement between observed and predicted probability of mortality. This was assessed graphically at fixed time points.³⁸ As an overall measure of model performance, we calculated the integrated Brier score, a scoring rule affected by both discrimination and calibration, with lower scores indicating better model accuracy.³⁹

We assessed internal validity via bootstrapping to quantify any optimism in model performance. We repeated the entire modeling process (including variable selection) in 30 bootstrapped samples and averaged the individual iAUC values to obtain an optimism-corrected iAUC. We externally validated the final prediction model by examining model performance in NHATS. Because the length of follow-up time in NHATS did not allow us to provide estimates at 10 years, we present time-specific AUC values and calibration plots at 1, 2, and 5 years. To present the final model, we have provided β coefficients and baseline survival at specific time points to facilitate calculation of individual predictions. We also created a web application to input individual patient characteristics to obtain mortality risk estimates. Statistical analyses were performed using SAS (version 9.4; SAS Institute, Inc), Stata (version 17.0; StataCorp, LLC), and R statistical software (version 4.05, R Project for Statistical Computing).

Of the 43 398 participants aged 65 years or older in the HRS between 1998 and 2016, we included 4267 participants with probable dementia (eFigure 1 in the Supplement). The mean (SD) age when participants were classified as having dementia was 82.2 (7.6) years (Table 1). In the HRS cohort, 2930 (survey-weighted 69.4%) were female, and 785 (survey-weighted 12.1%) identified as Black. The amount of missing information was generally less than 1%. The median (IQR) follow-up time was 3.9 (2.0-6.8) years in the overall cohort, 3.5 (1.6-6.2) years among those who died, and 5.2 (3.0-8.6) years among those who did not die by end of follow-up. A total of 3466 participants (81.2%) died by the end of follow-up.

The external validation cohort from NHATS included 2404 individuals with probable dementia (eFigure 2 in the Supplement). Demographics were similar to HRS, although a greater proportion of individuals had functional impairments and were Black (33% in NHATS vs. 12% in HRS) (Table 1). Kaplan-Meier survival curves are shown in eFigures 3 and 4 in the Supplement. The histogram of the prognostic index showed decreased case-mix heterogeneity in NHATS (eFigure 5 in the Supplement). By the end of follow-up, 1426 participants (59.3%) had died.

eTable 4 in the Supplement shows the unadjusted hazard ratios for all candidate predictors. The final variables following variable selection were age category, sex, BMI category, smoking status, number of ADL dependencies, number of IADL difficulties, difficulty walking several blocks, participation in vigorous physical activity, and chronic conditions including diabetes, heart disease, cancer, and lung disease. Table 2 displays the multivariable-adjusted hazard ratios and β coefficients for variables in the model.

The iAUC prior to internal validation was 0.76 (95% CI, 0.75-0.76) (Table 3). After bootstrap internal validation, the optimism-corrected iAUC was 0.76 (95% CI, 0.75-0.76), suggesting good discrimination. Overall, AUC values tended to increase over time (eFigure 6 in the Supplement). For example, the optimism-corrected, time-specific AUC at 1 year was 0.73 (95% CI, 0.70-0.75), 0.75 (95% CI, 0.73-0.77) at 5 years, and 0.84 (95% CI, 0.82-0.85) at 10 years. Figure 1 displays the predicted vs observed risk of death at the 1-, 2-, 5-, and 10-year time points across deciles of predicted risk. This showed that the model was well calibrated with good agreement between observed and predicted risk. The integrated Brier score averaged from 20 imputations was 0.10. Results from the sensitivity analysis only including incident probable dementia cases showed similar model coefficients and performance (eResults 1, eTables 5 and 6, and eFigure 7 in the Supplement).

On external validation in NHATS, the calibration slope was 0.92 (95% CI, 0.84-1.00). The time-specific AUC values were 0.73 (95% CI, 0.70-0.76) at 1 year, 0.72 (95% CI, 0.70-0.75) at 2 years, and 0.74 (95% CI, 0.71-0.76) at 5 years (Table 3). Calibration was similar to HRS (Figure 1).

Users can input individual patient characteristics on ePrognosis (https://eprognosis.ucsf.edu/dementia.php) to obtain mortality risk estimates (eFigures 8 and 9 in the Supplement). eResults 2 in the Supplement shows how to manually compute mortality risk. For illustration, we selected 10 participants at random in each decile of predicted risk and display their baseline characteristics and median predicted time to death (Figure 2). For example, patient G was a woman in her 80s with a BMI of 19.5, never smoker, 3 ADL dependencies, 4 IADL difficulties, and walking difficulty. Her median (IQR) time to death was 3.8 (1.9-6.3) years with predicted mortality risk of 12% at 1 year, 64% at 5 years, and 94% at 10 years.

We developed and externally validated a mortality prediction model in community-dwelling older adults with dementia that can be used to estimate mortality risk across a time frame of 1 to 10 years. We included variables that are readily obtainable in clinical practice, including demographics, functional status, health and/or behavioral factors, and comorbidities. Measures of model performance, including discrimination and calibration, were good and generally exceeded the discrimination in previous dementia mortality prediction models.

Our model provides accurate prognostic estimates for a heterogenous population of community-dwelling older adults with dementia that may be used to enhance conversations with patients and families and inform decision-making. Several clinical guidelines related to cancer screening and management of chronic diseases now encourage the use of life expectancy when making treatment decisions.^40-43 Given the broad range of mortality risk in this cohort, providing these estimates in a clinical context can help frame screening and treatment decisions by targeting interventions to those most likely to benefit.⁴⁴ Similarly, they can help families prepare for the future in terms of advance care planning and financial planning. Nonetheless, we recognize that the mortality estimates from our model are only one of many factors that go into decision-making for older adults with dementia and should be incorporated within the broader context of patient and family goals and preferences.

The final predictors in our mortality model represent variables commonly used in prediction models involving community-dwelling older adults and those previously identified as predictive of mortality among individuals with dementia.^6,9,25,26 In line with previous studies, older age had a strong effect on mortality, with female sex being a protective factor.^13,25 A BMI lower than 18.5 compared with 18.5 to 25.0 was associated with increased mortality risk, which is consistent with previous studies.^10,45-47 We included several functional measures (ADL dependencies, difficulties with cognitively demanding IADLs, difficulty walking several blocks, and participation in vigorous physical activity), which have been consistently shown to influence prognostic estimates in mortality indices, likely because they represent the end effect of chronic diseases.^48,49

Given that this was a community-dwelling cohort covering a wide range of dementia severity, our focus was to estimate prognosis over years rather than months. Therefore, we did not include predictors such as pressure ulcers or time spent in bed that are often included in models assessing hospice eligibility in those with severe dementia. The final predictors in our model suggest that prognosis in a community-dwelling cohort is often driven by comorbid disease.

A 2021 systematic review and our review of mortality models in persons with dementia identified many models at high risk of bias (ROB) owing to methodologic limitations, including small sample sizes, lack of internal and/or external validation, and not reporting model calibration (eMethods and eTables 7-9 in the Supplement).¹³ A few mortality models are worth noting in comparison to our model. The model by Haaksma et al⁵⁰ developed in Sweden presents 3-year survival probabilities from a simplified model that includes age, sex, Mini-Mental State Exam score, Charlson Comorbidity Index (CCI), and dementia type. The model by van de Vorst et al⁵¹ developed in the Netherlands presented 1-year mortality risk using age, sex, setting of care, and modified CCI. The concordance statistics ranged from 0.69 to 0.72, which is somewhat lower than the findings in our study. Advantages of our model include longer follow-up duration, greater applicability for individuals in the US, and inclusion of functional measures that might explain improved model performance.

Notable models in US populations include Newcomer et al (2003), Armstrong et al (2022), and several longitudinal Grade of Membership (L-GoM) models.^52-58 Newcomer et al⁵² included participants enrolled in the Medicare Alzheimer Disease Demonstration Evaluation. The primary limitations are that the data came from 1989 to 1994, and the article did not easily allow calculations of individual mortality risk. Armstrong et al⁵⁸ used data from the National Alzheimer Coordinator Center (NACC) and reported 5-year survival probabilities for individuals with different types of dementia. A possible limitation is the generalizability of the NACC population. The L-GoM models offer a flexible approach to modeling future trajectories across multiple end points, and the most recent update demonstrated accurate mortality predictions based on a single comprehensive visit.⁵³ Disadvantages include unclear generalizability (developed using approximately 250 participants in the predictors 1 [1989-2001] and predictors 2 [1997-2007] cohorts at 3 academic research centers) and a large number of variables (79 clinical signs/symptoms).⁵⁹

This study has several strengths. We included a large cohort of community-dwelling individuals with dementia to predict mortality with good discrimination and calibration up to 10 years. The final variables in the model are readily obtainable and can be input into an online calculator to obtain prognostic estimates. We performed an external validation in NHATS and found similar discrimination and calibration.

This study has a few limitations. First, participants were classified as having dementia using a validated algorithm that may be subject to misclassification. Although this algorithm has shown high accuracy in validation studies (eg, AUC of 0.93 in training data and 0.84 in validation data), the overall accuracy is reduced in certain subgroups, such as racial and ethnic minorities and less-educated individuals.^19,20,60 Second, information on dementia etiology (eg, Alzheimer disease or vascular disease) or dementia severity was not available. We also chose not to include cognitive tests as candidate predictors, instead relying on cognitively demanding IADLs as a proxy of cognitive status. We did this because cognitive tests were only available for self- (nonproxy) responses, and it is often challenging to guide patients with dementia through cognitive testing in routine clinical settings. The fact that our model performance was an improvement on previous models that included measures assessing cause and severity of dementia suggests that our model can still provide predictions that inform decision-making without this data. Third, the follow-up duration in NHATS precluded external validation at 10 years. Given that model performance after internal and external validation was relatively similar at 5 years, it is likely that our internal validation results from HRS at 10 years will remain accurate. However, future validation studies in other cohorts with longer follow-up times should be performed.

In this cohort study, we developed and externally validated a mortality model with good discrimination and calibration that can be used to estimate mortality risk up to 10 years in community-dwelling older adults with dementia. Final predictors in the model are readily obtainable in clinical practice, including demographics, behavioral/health factors, functional status, and comorbidities. Estimates can be used by clinicians, patients, and families to help aid in discussions around clinical management of medical comorbidities, preventive screening, and advance care planning.

Accepted for Publication: August 6, 2022.

Published Online: September 26, 2022. doi:10.1001/jamainternmed.2022.4326

Corresponding Author: William James Deardorff, MD, 490 Illinois St, Floor 08, San Francisco, CA 94158 (william.deardorff@ucsf.edu).

Author Contributions: Drs Deardorff, Lee, and Smith had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Smith and Lee contributed equally as co-senior authors.

Concept and design: Deardorff, Barnes, Smith, Lee.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Deardorff.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Deardorff, Boscardin.

Obtained funding: Smith, Lee.

Administrative, technical, or material support: Covinsky, Lee.

Supervision: Boscardin, Mitchell, Smith, Lee.

Conflict of Interest Disclosures: Dr Deardorff reported grants from National Institute on Aging T32-AG000212 during the conduct of the study. Dr Langa reported personal fees from the University of California, San Francisco (UCSF) as a consultant for this project during the conduct of the study; grants from the National Institutes of Health (NIH)/National Institute on Aging (NIA) related to epidemiology of dementia; and grants from Alzheimer's Association outside the submitted work; and served as an expert witness for a legal case related to cognitive decline and decisional capacity. Dr Covinsky reported grants from NIA during the conduct of the study. Dr Whitlock reported grants from National Center for Advancing Translational Sciences (KL2TR001870) during the conduct of the study; grants from NIA (R03AG059822), grants from Foundation for Anesthesia Education and Research, grants from UCSF Department of Anesthesia & Perioperative Care, and nonfinancial support from NIA (P30AG044281) outside the submitted work. Dr Lee reported grants from NIA (R01AG057751) during the conduct of the study and grants from VA HSR&D IIR 15-434 outside the submitted work. No other disclosures were reported.

Funding/Support: This work was supported by NIA (T32-AG000212 to Dr Deardorff, R01AG057751 to Drs Smith and Lee, K24AG068312 to Dr Smith, K24AG066998 to Dr Lee, P01AG066605 to Drs Covinsky, Smith, Lee, Jeon, and Boscardin, and P30AG044281 to Drs Covinsky, Smith, Lee, Jeon, and Boscardin) and the National Center for Advancing Translational Sciences (KL2TR001870 to Dr Whitlock). The Health and Retirement Study is funded by NIA grant (U01AG009740) and performed at the Institute for Social Research, University of Michigan. The National Health and Aging Trends Study was produced and distributed by https://www.nhats.org/ with funding from the NIA (grant number U01AG32947).

Role of the Funder/Sponsor: The funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr Covinsky is Associate Editor of JAMA Internal Medicine, but he was not involved in any of the decisions regarding review of the manuscript or its acceptance.