If populations from multiple countries were included in 1 study, all populations were counted once (unweighted). Of 302 included studies, 153 (51%) included participants from North America and 140 (46%) from Europe. Asia, Australia, South America, and Africa were less represented with 33 (11%), 9 (3%), 5 (2%), and 0 studies including participants from these continents, respectively.
Blue dotted lines represent the average age across all population-based studies; orange dotted lines represent the average age across clinic-based studies. The mean age of patients with dementia was 8.8 years higher in population-based studies than in clinic-based studies.
Percentages reflect the percentage of studies of a certain study type among all studies using the biomarker. Studies that used multiple biomarkers contributed to each of those biomarkers. None of the included studies conducted in a nursing home or registry setting used any biomarkers. CSF indicates cerebrospinal fluid; MRI, magnetic resonance imaging; PET, positron emission tomography.
eAppendix 1. Search strategy
eFigure 1. Flow diagram of study inclusion
eTable 1. Representation of the most investigated study populations
eTable 2. Aim of studies by study setting
eTable 3. Design of studies by study setting
eFigure 2. Geographical diversity of affiliation of the lead authors from 302 included studies on dementia published in 2018 and 2019
eTable 4. Numbers of studies reporting dementia subtypes among all studies and by study setting
eTable 5. Characteristics of study populations and the number of studies reporting those characteristics in postmortem studies
eTable 6. Use of biomarkers across all study settings
eTable 7. Altmetric scores of studies by study characteristics
eTable 8. Number of studies reporting characteristics of the study population
eAppendix 2. Studies included in the systematic review with characteristics of studies and study populations
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Mooldijk SS, Licher S, Wolters FJ. Characterizing Demographic, Racial, and Geographic Diversity in Dementia Research: A Systematic Review. JAMA Neurol. 2021;78(10):1255–1261. doi:10.1001/jamaneurol.2021.2943
To what extent does contemporary dementia research reflect patient characteristics in routine clinical practice?
In this systematic review of 302 studies on dementia published in 2018 and 2019, most studies originated from North America or Europe, including a median (interquartile range) of 89% (78-97) White participants. Patients with dementia in clinic-based studies were younger at time of diagnosis than those in population-based studies (mean difference 8.8 years; 95% CI, 7.3-10.2) years younger at time of diagnosis.
Generalizability of contemporary dementia research could benefit from greater racial and geographical diversity, and inclusion of patients within an age range that is more representative of routine practice.
For informed decision making on diagnosis and treatment of dementia, physicians and their patients rely on the generalizability of evidence from published studies to clinical practice. However, it is uncertain whether everyday care of elderly patients with dementia is sufficiently captured in contemporary research.
To systematically review contemporary dementia research in terms of study and patient characteristics in order to assess generalizability of research findings.
PubMed was searched for dementia studies published in the top 100 journals in the fields of neurology and neuroscience, geriatrics, psychiatry, and general medicine between September 1, 2018, and August 31, 2019. Two reviewers extracted study characteristics, including setting, number of participants, age at diagnosis, and use of biomarkers.
Among 513 identified studies, 211 (41%) included fewer than 50 individuals with dementia and were excluded. The remaining 302 studies included a median (interquartile range) of 214 patients (98-628) with a mean (SD) age at diagnosis of 74.1 years (8.0). Age at diagnosis differed with study setting. Patients in the 180 clinic-based studies had a mean (SD) age of 71.8 (6.4) years at time of diagnosis compared with 80.6 (4.7) years among patients in the 79 population-based studies (mean difference, 8.8 years; 95% CI, 7.3-10.2). Use of magnetic resonance imaging, positron emission tomography imaging, and cerebrospinal fluid imaging was mostly done in clinic-based studies (80% to 96%) and consequently in relatively young patients (mean [SD] age, 71.6 [5.1] years). A longitudinal design was more common in population-based studies than in clinic-based studies (82 % vs 40%). Most studies originated from North America and Europe (89%), including almost exclusively White participants (among 74 studies [22%] reporting on ethnicity: median [interquartile range], 89% [78-97]). The 3 most studied cohorts represented 21% of all included study populations.
Conclusions and Relevance
Contemporary dementia research is limited in terms of racial and geographic diversity and draws largely from clinic-based populations with relatively young patients. Greater inclusivity and deeper phenotyping in unselected cohorts could improve generalizability as well as diagnosis and development of effective treatments for all patients with dementia.
Physicians must often translate scientific evidence on diagnosis and care into the best decisions in the consultation room. However, the generalizability of published studies to the clinic largely depends on whether the particular group of patients has been sufficiently studied in clinical research. Lack of generalizability or external validity is a cause for major concern in clinical trials,1 including trials for dementia.2-4 However, in the absence of disease-modifying interventions, most research and clinical decision-making are observational in nature, querying etiology, diagnostic strategies, and prognosis. Lack of generalizability threatens these studies as well as clinical trials when biological underpinning and disease course vary with patient characteristics, as is very common with dementia.5 In 2004, it was suggested that the representativeness of observational dementia research may also be limited,6 but to our knowledge, no published studies have investigated this systematically in a contemporary setting.
Clinical guidelines for assessment of suspected dementia do not recommend extensive investigation with cerebrospinal fluid (CSF), for example, or even brain magnetic resonance imaging (MRI) (instead of computed tomography [CT]) in routine clinical practice.7-10 Although sensible from the viewpoint of efficient and cost-effective care, this could easily lead to research populations that are not representative of the wider population with the disease. It could exclude elderly individuals, who constitute the largest percentage of patients with cognitive decline but often do not attend specialized memory clinics. Unlike in younger patients with dementia, who mostly present with distinct clinical phenotypes, the variety of symptoms that comes with accumulation of various pathologies in old age11,12 influences diagnostic yield and treatment efficacy. Similarly, sex and race or ethnicity differences in biology or presentation, owing to, for instance, variation in genetic makeup and comorbidity, can have an important impact on diagnosis, prognostic risk stratification, and treatment effects. As the prevalence of dementia is expected to increase most in Asia and Africa,13,14 inclusiveness is pivotal, but actual geographic and racial representativeness of study populations are not clearly established.15 We therefore systematically reviewed the contemporary literature to determine the extent to which dementia research is representative of the wider population with dementia and identified patient, study, and disease characteristics that may help refine generalizability.
On December 17, 2019, we conducted a systematic search of the literature in PubMed for studies including patients with dementia published in any language between September 1, 2018, and August 31, 2019. We selected all studies published in the top 100 journals, ranked by 2018 impact factor, in the following InCites (Clarivate Analytics) categories: clinical neurology, neurosciences, geriatrics and gerontology, psychiatry, neuroimaging, and general and internal medicine. A list of included journals and the complete search syntax are presented in eAppendix 1 in the Supplement. We included any study including patients with dementia for extraction of study details by 2 reviewers (S.S.M. and F.J.W.). The study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.16 This review was not registered, as it did not consider health outcomes on the individual patient level.
We imported all retrieved records into an EndNote version X9 (Clarivate Analytics) library. A single reviewer (F.J.W.) screened all articles for eligibility, using the following inclusion criteria: original research articles that included patients with a dementia diagnosis either established at baseline or during follow-up.
Study characteristics were independently extracted from the identified reports by 2 investigators (S.S.M. and F.J.W.), with discrepancies resolved by consensus discussion and adjudicated by a third investigator (S.L.). The extracted information included affiliation of the first author; geographical location of the study’s source population; study setting (eg, clinic based, population based, or nursing home); study design (eg, longitudinal or cross-sectional); study aim (based on the research question); and use of imaging (eg, MRI, positron emission tomography [PET]), genetic, cerebrospinal fluid, or blood biomarkers. We further extracted various patient characteristics for included study participants, namely the number of patients with dementia and their sex, race, and age at diagnosis; in postmortem studies, age of death, sex, and dementia subtype were included. After determining the number of participants with dementia in each study, we proceeded with the extraction of further details in studies with 50 individuals or more with dementia. This number was chosen for reasons of efficiency as well as capturing in more detail studies with typically greater precision. Finally, we looked up corresponding publicity scores (ie, the Altmetric score) of all included articles on November 3, 2020.17 Altmetric scores are quantitative indices that are considered complementary to traditional, citation-based metrics to indicate the amount of attention an article receives based on, for example, public policy documents, mainstream media coverage, and mentions on social networks.
When a single publication described multiple study populations (eg, discovery and replication or coordinated analyses across different cohorts), these were pooled for analyses unless relevant distinctions in study setting, design, or aims merited separate analyses. First, we determined the number of participants across studies. We then compared study and patients characteristics between different settings for all studies including 50 or more patients.
We calculated the mean age at diagnosis of dementia for clinic-based and population-based studies based on summary measures as reported by the individual studies (mean or median age) and compared these between study settings using the t test. As we anticipated that clinic-based studies would more often include patients with dementia subtypes that may occur at a relatively younger age, such as dementia with Lewy bodies, frontotemporal dementia, and Creutzfeldt-Jakob disease, we performed a sensitivity analysis of clinic-based studies without inclusion of dementia with Lewy bodies, frontotemporal dementia, and Creutzfeldt-Jakob disease. For postmortem studies, we extracted age at death, as age at diagnosis was not reported in most studies. Likewise, for patients in nursing homes, the age was reflective of time at inclusion rather than time at diagnosis.
We compared the distributions of sex and race between study settings. Because average life expectancy in the global population is longer for women than for men, we stratified the sex analyses by the average age. Finally, we log transformed Altmetric scores because of their skewed distribution and subsequently assessed differences by study characteristics using linear regression, adjusted for the time between publication and collection of the scores. Analyses were done using R version 3.6.3 (the R Foundation). A 2-tailed P value of less than 0.05 was considered significant.
Among a total 955 articles, we identified 513 eligible studies of individuals with dementia (eFigure 1 in the Supplement). Most ineligible studies were either preclinical (148) or included dementia-free individuals only (161). Of 513 included studies, 211 (41%) included fewer than 50 individuals. The remaining 302 studies represented 329 distinct study populations (eAppendix 2 in the Supplement), with a median (interquartile range [IQR]) of 214 (98-628) participants with dementia (Figure 1).
Among the included studies, most had an observational design (315 of 329; 96%), whereas 14 studies (4%) reported results of clinical trials. Most studies were clinic based (55%) or population based (24%), followed by studies based on postmortem examinations (17%), studies situated in nursing homes (2%), registry-based studies (1%), and a combination of clinic-based and population-based studies (4%). The 3 most investigated (agglomerated) study populations (ie, the Alzheimer’s Disease Neuroimaging Initiative, the Religious Orders Study and Memory and Aging Project, and the National Alzheimer’s Coordinating Center) jointly represented 21% of all study populations (eTable 1 in the Supplement). Characteristics of the included studies by study setting are presented in the Table.
Of the 329 study populations for which we extracted detailed data, 156 (47%) aimed to unravel dementia etiology and 96 (29%) to improve detection or diagnosis, while only a minority focused on disease management (7%), prognosis (5%), or treatment (5%) (eTable 2 in the Supplement). Population-based studies were mainly aimed at unraveling disease etiology, whereas clinic-based studies primarily investigated diagnosis and early detection. In terms of design, the overall numbers of cross-sectional and longitudinal studies were nearly identical (161 vs 159), but a longitudinal design was twice as common in population-based cohorts than in clinic-based studies (82% vs 40%) (eTable 3 in the Supplement). This difference between clinic-based studies and population cohorts was independent of differences in study aims (such as detection or diagnosis and etiology).
Most of the included populations originated from either North America (153 of 302; 51%) or Europe (140 of 302; 46%), jointly accounting for 89% of all populations (Figure 2). Other continents were represented as follows: 33 studies (11%) from Asia, 9 studies (3%) from Australia, and 5 studies (2%) from South America; none of the included studies described patients with dementia from Africa. We observed a similar geographical pattern for the affiliation of the lead author of the included articles (eFigure 2 in the Supplement). Accordingly, in 22% of the studies that reported on race, a median (IQR) 89% (78-97) were White (Table).
The mean (SD) age at diagnosis of dementia across all studies was 74.1 years (8.0). Age at diagnosis differed by study setting. Patients in the 180 clinic-based studies had a mean (SD) age of 71.8 (6.4) years at time of diagnosis compared with 80.6 (4.7) years among patients in the 79 population-based studies (mean difference, 8.8 years; 95% CI, 7.3-10.2) (Figure 3). This difference was broadly unchanged when excluding clinic-based studies of dementia with Lewy bodies, frontotemporal dementia, and Creutzfeldt-Jakob disease (7.7 years; 95% CI 6.2-9.2; P < .001) (eTable 4 in the Supplement) and when excluding studies of early-onset dementia (6.9 years; 95% CI, 5.3-8.5; P < .001). Accordingly, in postmortem studies, mean (SD) age at death was higher in population-based studies than in clinic-based studies (eTable 5 in the Supplement). As expected, patients in nursing homes were oldest (mean [SD] age, 84.8 [4.7] years) (Table). In terms of study design, patients in cross-sectional studies were younger than those in longitudinal studies (3.7 years; 95% CI, 2.0-5.5; P < .001) (eTable 3 in the Supplement).
A median (IQR) 54% (45-62) of all patients in contemporary dementia research were women, somewhat higher in population-based studies (64% [52-70]) and nursing homes (69% [65-72]) than in clinic-based studies (50% [44-59]) and postmortem studies (53% [44-59]) (Table). This difference in sex distribution seemed at least in part attributable to age differences between the study settings, as clinic-based studies that included patients with a mean age 80 years or older included more women (median, 65%; IQR, 61-68; n = 11) than clinic-based studies of patients aged 75 to 80 years (median, 51%; IQR, 45-61; n = 33), 70 to 75 years (median, 49%; IQR, 44-55; n = 47), or younger than 70 years (median 49%; IQR, 43-54; n = 48).
Brain imaging was the most commonly applied biomarker for dementia, with 75 of 329 studies (23%) using MRI, and 41 of 329 studies (12%) using PET. Of studies using PET, 16 used fluorodeoxyglucose PET, 27 amyloid PET, and 4 tau PET. CSF was assessed in 48 of 329 studies (15%), genotyping in 65 of 329 studies (20%), and plasma markers in 29 of 329 (9%). Of all studies using imaging or CSF data, 80% to 96% came from clinic-based patient populations (Figure 4; eTable 6 in the Supplement). Consequently, patients that underwent brain imaging or CSF investigation were relatively young at time of diagnosis (mean [SD] age, 71.6 [5.1] years) and included fewer women (49%) in comparison with the overall age and sex distribution from population-based studies.
Study sample size was associated with publicity scores, with more media attention for the larger studies compared with the smaller studies (with 250 participants as the cutoff; difference in log-transformed scores [β], 0.46; 95% CI, 0.16-0.76; P = .003) (eTable 7 in the Supplement). Publicity scores also differed by study setting: population-based studies received more attention than clinic-based studies (β, 0.70; 95% CI, 0.35-1.06; P < .001), whereas studies based in nursing homes received less publicity compared with clinic-based studies (β, −1.57; 95% CI, −2.65 to −0.49; P = .004).
In this comprehensive evaluation of contemporary research on patients with dementia published in leading journals in clinical medicine from September 1, 2018, to August 31, 2019, we observed poor representativeness of the wider patient population with dementia. Only 1 in 5 studies reported on participants’ race, including almost exclusively White participants residing in North America and Europe. Age at diagnosis of dementia was on average 8.8 years lower in clinic-based studies compared with population-based studies, and phenotyping by means of MRI, PET, and CSF was therefore mostly limited to relatively young patients.
While internal validity (ie, minimizing bias) is crucial to acquiring truthful findings, the usefulness of the clinical research findings in many studies is threatened if the results cannot reasonably be applied to the broader group of patients in routine practice.1 A lack of racial and geographic diversity and underrepresentation of older persons has been described in other medical research fields, such as cancer and cardiovascular medicine, in particular regarding participant inclusion in clinical trials.18-21 In this review, observational studies comprised 96% of the included studies. Our findings imply that nonrepresentativeness of patients hampers the external validity of much of contemporary observational dementia research, as has previously been suggested with respect to clinical trials.2,4,6 This has important implications for the applicability of results by clinicians outside specialized referral centers, which often diagnose, treat, and monitor substantially different patient populations. Generally speaking, generalizability will be affected by any factor that substantially affects outcome, including study setting, patient selection and characteristics, and outcome measures. For example, absolute risk estimates from prediction models and the diagnostic value of specific tests often translate poorly to populations with very different background risk. While relative risk estimates may be more stable across populations than absolute risks,22 substantial discrepancies in relative risk between clinic-based populations and unselected populations have recently been reported.23 Statistical methods, such as inverse probability weighting, may alleviate concerns to some extent, provided sufficient information about the source population.24
In this review of dementia research, older patients were underrepresented. Younger patients often present with distinct clinical phenotypes, owing to the predominance of a single type of brain pathology, be it hallmarks of Alzheimer disease, vascular brain injury, alpha-synuclein, or prions. By contrast, in the average 80-year-old brain, multiple types of pathology have accumulated and jointly contribute to a dementia phenotype with a conglomerate of symptoms that precludes a single etiological diagnosis.12,25 The median age at diagnosis of dementia in the population is older than 80 years, and mixed pathology is therefore present in most elderly patients with dementia.26-29 While the study of highly selected younger individuals can be useful to our understanding of specific mechanisms,30 it also carries certain pitfalls. For example, hippocampal atrophy is a specific sign of Alzheimer disease in younger patients but is much more common in elderly individuals, even in the absence of cognitive impairment.28 Similarly, amyloid pathology is increasingly common in elderly individuals, in whom it often does not correlate with cognitive performance.31 We observed that biomarkers and diagnostic tools, such as PET imaging, CSF sampling, and even brain MRI, were sparsely used in studies of older participants with dementia. Therefore, conclusions based on studies of younger patients, in particular relating to diagnosis and prognosis, may not be readily translatable into best practice for most older dementia patients. Indeed, it was recently found that findings from a clinic-based sample often do not align with results from the same study among a more representative population.23 Broader inclusiveness of clinic-based studies and more in-depth study of biomarkers in unselected populations may help to ensure generalizability to patient populations beyond specialized referral centers.32
Over the coming decades, dementia prevalence is expected to increase most in low- and middle-income countries such that by 2050, 68% of all patients with dementia will be living in low- and middle-income countries, mainly Africa, Asia, and South America.13,14 Yet 9 of 10 studies in this review primarily included patients from North America or Europe, including on average 89% White individuals (in the 22% of studies that reported on race and ethnicity). The projected increases in dementia prevalence are largely owing to shifts in demography and an increasing prevalence of dementia risk factors.33 In many instances, however, it remains unknown if risk factors are associated with similar increases in risk across populations with various genetic makeup. Whether the genetic predisposition for dementia differs by race also remains unclear.34,35 Other differences that warrant cross-cultural dementia research relate to awareness, recognition (including stigmatization), and (available means for) diagnosis and disease management.34,36 It should be noted that only 1 in 5 studies reported on race. The racial diversity in other studies was likely similar, given their geographical location and overlap in population with studies that did report these distributions. Standardized reporting of race and ethnicity of study participants may promote inclusiveness in future studies, for example by implementing this information in internationally recognized guidelines for reporting studies. This applies to studies of dementia as well as cognitive aging in the prediagnostic phase.37
The veracity of any research finding depends on validity (ie, the absence of bias) and precision (ie, power). Our observation that over 40% of studies included fewer than 50 patients with dementia triggers caution about the precision of findings in much of contemporary evidence. Insufficient sample sizes are associated with a reduced chance of detecting true effects as well as a lower likelihood that statistically significant results in fact reflect a true effect.38 In addition, underpowered studies prompt publication bias owing to selective reporting of statistically significant results. Although smaller studies can provide valuable insight—for instance, in case of a qualitative design—and various smaller studies included in this review undoubtedly have made important contributions to the literature, adequately powered studies are pivotal to avoid spurious findings.
Although we believe our results accurately reflect the most informative contemporary dementia research, certain limitations should be considered when interpreting our results. First, detailed characteristics of studies with smaller sample sizes (fewer than 50 participants with dementia) were not extracted. Based on our observations among the studies with 50 or more participants, it can be expected that smaller studies were mostly clinic-based studies among relatively younger patients. Second, restricting our search to the more renowned medical journals may have biased our results to the inclusion of research from institutions with a longer tradition of publishing in medical journals in the English language, thereby underestimating the overall contribution to dementia research from South America, Africa, or Asia. Nevertheless, our findings are in line with previously noted underrepresentation of low- and middle-income countries in population-based research,36 extending this to other settings as well. Third, not all articles reported data on race, sex, and age at diagnosis, calling for more consistent reporting of important population characteristics, for example by using international reporting guidelines. Fourth, age at diagnosis compared with the general population in the included reports may be underestimated by earlier diagnosis owing to either sensitive diagnostic tools in specialized memory clinics or routinely applied cognitive screening in population-based studies, but this is unlikely to have affected the difference between study settings.
External validity of contemporary dementia research is hampered by lack of racial and geographic diversity. Elderly patients are particularly underrepresented in clinic-based studies, notably with regard to the development and application of biomarkers. Increasing inclusiveness and in-depth study of biomarkers in well-defined, unselected populations ensures transportability to patient populations beyond specialized referral centers.
Accepted for Publication: June 23, 2021.
Published Online: September 7, 2021. doi:10.1001/jamaneurol.2021.2943
Corresponding Author: Frank J. Wolters, MD, PhD, Department of Epidemiology, Erasmus University Medical Center, Doctor Molewaterplein 40, 3015GD, Rotterdam, the Netherlands (firstname.lastname@example.org).
Author Contributions: Dr Wolters had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Wolters.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Mooldijk.
Critical revision of the manuscript for important intellectual content: Licher, Wolters.
Statistical analysis: Mooldijk.
Supervision: Licher, Wolters.
Conflict of Interest Disclosures: None reported.
Additional Contributions: We thank Reffat A. Segufa, MD, Erasmus University Medical Center, for her valuable contribution to the literature search and M. Kamran Ikram, MD, PhD, Erasmus Medical Center, for his thoughtful comments on an earlier version of the manuscript. These individuals received no compensation, financial or otherwise, for their contributions.
Create a personal account or sign in to: