Background
Apathy is a common and significant problem in patients with dementia, regardless of its cause. Observations about frontosubcortical circuit syndromes indicate that apathy may have affective, behavioral, or cognitive manifestations.
Objectives
To explore whether the apathy manifested in frontotemporal dementia (FTD), with its predominantly anterior brain neuropathologic features, differs from the apathy in dementia of Alzheimer type (DAT), with its predominantly hippocampal- and temporoparietal-based neuropathologic features, and to determine whether other behavioral disturbances reported in frontosubcortical circuit syndromes correlate with apathy.
Design
Analyses included individual items within Neuropsychiatric Inventory subscale items. Items of the apathy/indifference subscale were designated by consensus as affective (lacking in emotions), behavioral (inactive, chores abandoned), or cognitive (no interest in the activities of others). Proportions of correlated nonapathy Neuropsychiatric Inventory items were calculated.
Setting
Several neurology specialty clinics contributed to our data set.
Participants
A total of 92 participants with FTD and 457 with DAT.
Main Outcome Measures
The Neuropsychiatric Inventory was analyzed.
Results
Apathy was more prevalent in patients with FTD than in those with DAT, but when present, the specific apathy symptoms associated with both types of dementia were rarely restricted to 1 of the 3 domains of apathy. Dysphoria concurrent with apathy was unique to the DAT group and negatively correlated in the FTD group. Participants with affective apathy more frequently copresented with an orbital frontosubcortical syndrome in FTD (impulsivity and compulsions). Affective apathy also copresented with uncooperative agitation, anger, and physical agitation in both types of dementia.
Conclusions
Apathy is common in patients with FTD and DAT, although it is more common in those with FTD. When present, it usually involves changes in affect, behavior, and cognition. It is associated with behaviors that have previously been shown to affect patient safety, independence, and quality of life.
Apathy is increasingly being recognized as a common and clinically significant neurobehavioral syndrome. Previously reported prevalence of apathy in patients with clinically diagnosed dementia of Alzheimer type (DAT), the most common type of dementia among elderly individuals, ranges from 36% to 88%.1 A similar prevalence of apathy has also been reported in frontotemporal dementia (FTD; 60%-90%).2-4 Although physicians recognize the prevalence of apathy in patients with dementia, little has been reported about the pathophysiology, characteristics, and behavioral associations of apathy.
Robert et al5 and Marin6 have proposed that symptoms of apathy may be separable. Affective apathy can manifest as symptoms of indifference or lack of empathy. A second domain of apathy is behavioral apathy, which manifests as indolence and a requirement for prompts to initiate physical activity. Cognitive apathy refers to inactivation of goal-directed cognitive activity manifested, for example, by the requirement of assistance in the initiation of mental activity or speech. The pattern of symptoms that develop in an individual may depend on which brain regions have been affected by the neurodegenerative process. We and others propose that each of these domains of apathy derives from dysfunction of frontosubcortical circuits.7,8 We hypothesized that apathy in patients with FTD would include all domains (affective, behavioral, and cognitive) because of its impact on the superior medial frontal cortex and that apathy in patients with DAT would have mainly affective features because of damage focused on limbic structures. In addition, patients with DAT show frontal lobe cognitive dysfunction, a finding that opens the range of possible findings to include cognitive apathy.
Beyond the apathy domains manifested by groups, we also hypothesized that nonapathy behavioral correlates ascribed to frontosubcortical circuit syndromes would differ between the 2 types of dementia: in FTD, impulsivity and compulsions would increase in the presence of apathy, and in DAT, dysphoria would increase when apathy is present. Symptoms of irritability or agitation that are common to both types of dementia might not correlate with apathy at all.9
The informant-based Neuropsychiatric Inventory (NPI) has established validity and reliability in clinical trials for symptomatic treatment of dementia and consists of 12 behavioral subscales.10 The criterion for diagnosing apathy with the NPI is any positive response to 1 or more apathy/indifference subscale items. We used NPI data to characterize features of apathy in FTD vs DAT and then used the NPI subscales that refer to hypothesized differences in impulsivity, compulsions, mood disturbance, irritability, and agitation to characterize the associations with apathy in each type of dementia.
Participant identification and recruitment
Data for this study were extracted from the existing databases of collaborators at clinics that serve patients with dementia: Baycrest, Sunnybrook Health Sciences Centre, University of California at Los Angeles, and University of California at San Francisco. Ethics committees at all institutions approved the protocols for collection of data. Participants or their substitute decision makers provided informed consent for inclusion in the site databases. Participant data were included if the participant had (1) a clinical diagnosis of dementia due to FTD according to consensus criteria11 or DAT as defined by National Institute for Neurological and Communicative Disorders and Stroke–Alzheimer's Disease and Related Disorder Association criteria12 and (2) full NPI responses from an informant. If the NPI had been administered with regard to the same participant on more than 1 occasion, we included only the earliest NPI scores, with the aim of increasing the possibility that data were taken before the use of any psychotropic medications. Duration of illness was defined as the time since the report of first symptoms. We identified 92 eligible participants with FTD and 457 with DAT.
Categorization of apathy symptoms
Three behavioral neurologists (Jon Ween, MD, T.W.C., and M.F.), 3 neuropsychologists (Brian Levine, PhD, Morris Moscovitch, PhD, and D.S.), and 3 geriatric psychiatrists (Nathan Herrmann, MD, Robert Madan, MD, and R.V.R.) categorized items on the apathy/indifference subscale of the NPI into affective, behavioral, cognitive, or generic (≥2) domains based on the definitions stated herein. Agreement by any 6 or more of the raters on an item identified the categorization used in our comparative analysis.
Analysis of variance was used to compare means of continuous patient characteristic variables between the FTD and DAT groups. We used χ2 tests to compare sex distribution, the proportion of patients with apathy between the FTD and DAT groups, and behavioral variant FTP and primary progressive aphasia between the FTD subgroups.13
To examine the propensity of participants to present with apathy symptoms from multiple domains, we used ordinal logistic regression to compare the number of apathy domains (ie, affective, behavioral, and cognitive) endorsed for the participant between the FTD and DAT groups, covarying for the total number of apathy subscale items. This approach allowed us to explore whether patients with different types of dementia were equally likely to show symptoms in a single domain or spread across multiple domains.
Using our hypothesis that affective apathy would be exhibited more frequently in DAT patients than in FTD patients, participants in each dementia group were classified as exhibiting no apathy, affective apathy, or nonaffective apathy. Helmert contrasts were used to compare each level of a factor with the combined average of subsequent factor levels.14 Specifically, we used a pair of Helmert contrasts (1) to compare participants who endorsed any of the 8 apathy items with those with no apathy and (2) to compare participants who endorsed an item in the affective apathy domain with participants with nonaffective apathy. Analysis of variance was used to compare the total NPI score (excluding apathy items) between the 2 dementia groups and the 3 apathy subgroups to examine breadth of behavioral disturbance in these patients.
To examine associations between occurrence of apathy and selected nonapathy NPI items, we further narrowed the data set of nonapathy subscale items to those with a relative frequency greater than 6% (>6 of the 92 FTD patients) in either the DAT or FTD group. An initial series of logistic regression models was run to estimate the effects of the dementia group and apathy subgroup on each of the remaining 47 nonapathy NPI items. We bundled the 47 nonapathy items into the groups of behavioral disturbances of interest stated in the hypotheses (impulsivity, compulsions, dysphoria, anger, and agitation) based on similarity in the patterns of logistic regression coefficients and on clinical interpretation of the items. Twelve clusters were designated, as indicated in Table 1.
Logistic regression was used to compare the occurrence of a behavioral disturbance in each of the 12 behavioral clusters as a function of the dementia and apathy subgroup (no apathy at all, affective apathy, and nonaffective apathy). The 2 Helmert contrasts were included in the regression as were interactions between the dementia group and each of the Helmert contrasts. If neither interaction was found to be statistically significant, they were both removed from the model, and parameters were reestimated. All hypothesis tests reported in the “Results” section were performed at an α level of .05.
Of the 92 participants with FTD, 53 had behavioral variant FTP and 39 had primary progressive aphasia. Table 2 lists the patient characteristics and the Mini-Mental State Examination (MMSE) scores of the FTD and DAT groups. Durations of illness, education levels, and MMSE scores were not available for all eligible participants. Although the groups were not matched for age, education level, or MMSE or total NPI scores, no statistically significant differences were found among groups for any of these variables.
Table 3 gives the prevalence of psychotropic drug classes most commonly prescribed to this sample. Serotonergic agents (including selective serotonin reuptake inhibitors and trazodone hydrochloride) were more commonly prescribed to patients with FTD, whereas cholinesterase inhibitors were more frequently used by patients with DAT at the time of NPI administration. Antipsychotic medication use was not common for either group.
Apathy, defined as any positive response to items on the NPI apathy/indifference subscale, was reported in 66 of 92 patients (72%) in the FTD group statistically significantly more frequently than in the DAT group (255/457 or 56%; χ2 test, P = .001). Among the FTD patients, 42 patients with behavioral variant FTP (79%) and 24 patients with primary progressive aphasia (62%) had apathy (χ2 test, P = .06).
Apathy domains in ftd and dat
The raters had an agreement of 6 or more of 9 on the designation of only 4 NPI apathy items as affective, behavioral, or cognitive apathy domains. “Other apathy” from the NPI apathy/indifference subscale was too general a category and therefore was not included in this analysis.
Large percentages of the apathetic participants (100% in the FTD group and 92.5% in DAT group) demonstrated behavioral apathy (either decreased spontaneous activity or decreased pursuit of baseline interests). Considerable overlap was found among apathy domains as we had designated them. Forty-six percent of FTD patients with apathy and 19% of DAT patients with apathy showed concurrent affective, behavioral, and cognitive apathy symptoms. Results of an ordinal logistic regression showed that the number of affective, behavioral, or cognitive apathy domains endorsed increased with total NPI score (P<.001), but no statistically significant difference was found in the number of affective, behavioral, or cognitive domains endorsed when FTD was compared with DAT.
Small segments of the apathetic DAT sample (10 participants) endorsed only apathy items outside those described herein: decreased spontaneous conversation, loss of interest in family or friends, the dropping of former interests, or the lack of pursuit of novel stimuli. The FTD and DAT groups exhibited similar rank orders for frequencies of the 7 NPI apathy items reported by informants (Figure). Analysis of variance showed that participants with apathy had higher total NPI scores (apathy subscale frequency × severity score excluded) than those without apathy (t543 = 7.60, P < .001), and participants with affective apathy had higher total NPI scores (apathy subscale frequency × severity score excluded) than those without affective apathy (includes no apathy, presence of behavioral and/or cognitive apathy, and/or apathy not categorized as affective, behavioral, or cognitive domain; t543 = 5.61, P < .001). Participants with FTD had higher total NPI scores (apathy subscale frequency × severity score excluded) than those with DAT (t543 = 2.54, P = .01).
Behavioral associates of apathy
The first Helmert contrast in the logistic regression model revealed that the presence of apathy was associated with higher proportions of anorexia (20% vs 5%), symptoms within the anxiety cluster (38% vs 19%), rummaging behavior (13% vs 9%), change of eating habits (45% vs 14%), childish behavior (20% vs 9%), and delusions of theft (9% vs 4%; Table 1).
Behavioral clusters associated with affective apathy
Behavioral associates of affective apathy (the second Helmert contrast) were only observed when the first Helmert contrast was also significant. Both Helmert contrasts applied to the following behavioral disturbances for affective apathy vs nonaffective apathy vs without apathy: uncooperative agitation (48% vs 37% vs 18%), anger (42% vs 32% vs 23%), insensitive behavior (29% vs 14% vs 5%), compulsions (63% vs 46% vs 22%), and impulsivity and/or euphoria (51% vs 33% vs 17%). The presence of apathy, especially affective apathy, was associated with a polarization toward the more extreme mood and behavioral disturbances that characterize each type of dementia.
Behavioral associates of the groups
As expected from the clinical diagnostic criteria for these types of dementia, logistic regression revealed that participants with FTD showed statistically significantly higher proportions of compulsions (55% vs 32%), impulsive behavior (44% vs 23%), changes of eating habits (49% vs 20%), and childish behavior (23% vs 9%) than participants with DAT. Participants with DAT had higher proportions than patients with FTD of the dysphoria cluster (42% vs 27%) and delusions of theft (14% vs 2%).
In the case of the dysphoria cluster, we identified a group × Helmert 1 interaction. In patients with DAT, those with apathy had a higher proportion of tears or sobbing, acting sad, or feeling like a failure, feeling discouraged, anticipating no future, feeling like a burden to family, or feeling suicidal than those without apathy (50% vs 22%). The opposite held in the FTD group: those with apathy had a lower proportion of the dysphoric symptoms than those who had FTD without apathy (24% vs 31%).
In the case of physical agitation, both Helmert contrasts were significant and were a group × Helmert 2 interaction. A significantly higher proportion of participants with DAT and affective apathy was reported to be physically agitated compared with participants who had DAT and nonaffective apathy (16% vs 6%). On the other hand, a lower proportion of participants with FTD and affective apathy was physically agitated than participants with FTD with nonaffective apathy (8% vs 17%).
Power calculations entailed the revisiting of our logistic regression model with 3 variables: group (FTD vs DAT) and the 2 Helmert contrasts. Power of the hypothesis tests for a binary predictor variable in logistic regression is a function of α level, sample size, effect size, event rate, and distribution of predictor variable.15 Hypothesis tests with our sample of 549 participants were performed at an α level of .05. Of the 549 participants in this study, 92 (17%) had FTD and 321 (59%) had apathy. Among the 321 participants with apathy, 114 (36%) had affective apathy. The baseline event rate for the included behavioral disturbances ranged from 2% to 25%.
Odds ratios for the significant group effects reported herein ranged from 2.90 to 4.46 and 1 significant odds ratio less than 1 (0.457) for dysphoria. The power for these hypothesis tests exceeded 88% for all but dysphoria, which had a power of 66%.
Odds ratios for the significant comparisons of participants with apathy with those without ranged from 1.88 to 4.60. The power for these hypothesis tests exceeded 88% for all but 2 comparisons: physical agitation, which had a 3.9% baseline event rate, an odds ratio of 2.65, and a power of 71%, and euphoria, which had a 6% baseline event rate, an odds ratio of 1.88 (the lowest significant odds ratio), and a power of 49%.
Odds ratios for the significant comparisons of participants with affective apathy vs those with nonaffective apathy ranged from 1.76 to 2.44. The power for these comparisons ranged from 54% to 79%.
Apathy is rarely the sole behavioral change occurring in patients with FTD or DAT. With apathy, patients with FTD and DAT are likely to be less active, but the behavioral disturbances present are more abnormal. That apathy is prevalent in dementia, whether the origin is probable DAT or FTD, is not a new finding,16,17 but the higher association of apathy with depressive features in DAT calls attention to the negative mood disruption in DAT patients with apathy in contrast to FTD patients, who are more likely to have apathy without any accompanying depression. We had hypothesized that in DAT, limbic neurodegeneration might sway apathy toward the affective domain. Concomitant depression supported this hypothesis, although, as described herein, apathy in both patients with DAT and those with FTD was infrequently restricted to the affective domain, which implies the simultaneous involvement of several frontosubcortical circuits.
As opposed to previous work which proposed that apathy and disinhibition in FTD patients are mutually exclusive,18 our study indicates that apathy is not a benign behavioral disturbance in dementia: when present, it is more likely for a patient to also manifest behaviors that are difficult to manage, such as impulsivity, socially embarrassing behaviors, or irritability, lability, and resistance to care. In addition, emotional blunting is often considered a separate symptom from behavioral or cognitive apathy,16 but our study shows that the 3 commonly co-occur. Our sample predominantly constituted individuals with early-onset illness (mean age in the 60s) and those who were fairly early in the course of their illness (mean, 4 years). One might expect this sample to have biased the results toward the hypothesized polarizations between DAT and FTD patients or even between apathetic FTD and disinhibited FTD patients, yet we were unable to elicit such distinctions. Davis and Tremont19 report that apathy in dementia does not contribute to caregiver perception of burden, but our study highlights the fact that apathy often co-occurs with other behaviors that add to caregiver distress.
The symptoms of apathy manifested in this study did not differ based on the origin of the dementia, but the unpacking of all responses to the NPI revealed that apathy was accompanied in FTD by behavioral disturbances that conform to an orbital frontosubcortical syndrome. The findings in DAT patients with apathy defied our assumption that a damaged limbic circuit would result in affective apathy alone; behavioral and cognitive apathy domains were endorsed in DAT as frequently as in FTD, which implies the additional involvement of at least the right dorsolateral prefrontosubcortical circuit. The pathologic changes of FTD and DAT overlap considerably in location, so it is entirely possible that each may affect much of the same circuitry. Amyloid imaging studies20 have shown burden in the frontal lobes early in DAT, and Royall et al21 emphasized that the diagnosis of DAT is probably not made until after the disease has affected the frontal systems.
Mourik et al22 report a similar prevalence of apathy in patients with FTD but do not find correlations of apathy with the use of other NPI subscales. Differences from the present findings may be due to longer duration of illness among their sample (mean, 6.7 years) and use of subscales of the NPI instead of specific subscale items. As disease progresses, the patients become less able to manifest socially inappropriate impulses and behaviors. Our analysis of the NPI data allows for the individual apathy symptoms present to carry more weight as symptoms of illness and may therefore be more sensitive to caregiver report.
Interrater recategorization of apathy items facilitated exploration of our hypotheses that apathy in FTD patients would include all domains (affective, behavioral, and cognitive) because of its impact on the superior medial frontal cortex and that apathy in DAT patients would have mainly affective features because of damage focused on limbic structures. In the recategorization, half the apathy subscale items did not achieve consensus by the raters. Our process was arbitrary, and the anatomical substrates of the apathy domains are certainly speculative. To raise the issue of apathy being associated with depression circumvents the assignation of how one might observe “loss of interest” as being characteristic of apathy vs of depression. It is a manifestation of both conditions.
We confirm that apathy is more common in patients with FTD than in those with DAT. When present, apathy usually involves changes in affect, behavior, and cognition. It is associated with the co-occurrence of major mood or behavioral disturbances that may not be anticipated if physicians focus too closely on the apathy itself.
Correspondence: Tiffany W. Chow, MD, Baycrest Rotman Research Institute, Brain Health Complex, 3560 Bathurst St, Eighth Floor, Toronto, Ontario M6A 2E1, Canada (tchow@rotman-baycrest.on.ca).
Accepted for Publication: September 17, 2008.
Author Contributions:Study concept and design: Chow, Cummings, and Lam. Acquisition of data: Chow, Lam, Black, Miller, and Freedman. Analysis and interpretation of data: Chow, Binns, Stuss, and van Reekum. Drafting of the manuscript: Chow and Binns. Critical revision of the manuscript for important intellectual content: Chow, Binns, Cummings, Lam, Black, Miller, Freedman, Stuss, and van Reekum. Statistical analysis: Chow and Binns. Obtained funding: Chow, Black, and Miller. Administrative, technical, and material support: Chow, Cummings, Lam, Black, Miller, and Freedman. Study supervision: Chow, Black, Stuss, and van Reekum.
Financial Disclosure: Dr Cummings owns the copyright to the NPI.
Funding/Support: This work was funded by National Institute on Aging grant F32 AG022802 (Dr Chow); the Saul A. Silverman Family Foundation, as part of a Canada International Scientific Exchange Program project, and Canadian Institutes of Health Research grant 53267 (Dr Freedman); University of Toronto Dean's Fund for New Faculty grant 457494 (Dr Chow); an endowment from the Sam and Ida Ross Memory Clinic (Drs Chow and Freedman); Canadian Institutes of Health Research grants MT-12853 and MRC-GR-14974 and James S. McDonnell Foundation grant 21002032 (Dr Stuss); and Canadian Institutes of Health Research grant 13129 (Drs Black and Stuss and Ms Lam).
2.Chow
TWMiller
BLBoone
KMishkin
FCummings
J Frontotemporal dementia classification and neuropsychiatry.
Neurologist 2002;8
(4)
263- 269
PubMedGoogle ScholarCrossref 3.Levy
MLMiller
BLCummings
JLFairbanks
LACraig
A Alzheimer disease and frontotemporal dementias: behavioral distinctions.
Arch Neurol 1996;53
(7)
687- 690
PubMedGoogle ScholarCrossref 4.Chow
TW Treatment approaches to symptoms associated with frontotemporal degeneration.
Curr Psychiatry Rep 2005;7
(5)
376- 380
Google ScholarCrossref 5.Robert
PHClairet
SBenoit
M
et al. The apathy inventory: assessment of apathy and awareness in Alzheimer's disease, Parkinson's disease and mild cognitive impairment.
Int J Geriatr Psychiatry 2002;17
(12)
1099- 1105
PubMedGoogle ScholarCrossref 6.Marin
RS Apathy: a neuropsychiatric syndrome.
J Neuropsychiatry Clin Neurosci 1991;3
(3)
243- 254
PubMedGoogle Scholar 7.Stuss
DTvan Reekum
RMurphy
K Differentiation of states and causes of apathy. Borod
J
The Neuropsychology of Emotion. New York, NY Oxford University Press2000;340- 363
Google Scholar 8.Levy
RDubois
B Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits.
Cereb Cortex 2006;16
(7)
916- 928
PubMedGoogle ScholarCrossref 9.Srikanth
SNagaraja
AVRatnavalli
E Neuropsychiatric symptoms in dementia-frequency, relationship to dementia severity and comparison in Alzheimer's disease, vascular dementia and frontotemporal dementia.
J Neurol Sci 2005;236
(1-2)
43- 48
PubMedGoogle ScholarCrossref 10.Cummings
JLMega
MGray
KRosenberg-Thompson
SCarusi
DAGornbein
J The Neuropsychiatric Inventory: comprehensive assessment of psychopathology in dementia.
Neurology 1994;44
(12)
2308- 2314
PubMedGoogle ScholarCrossref 11.Neary
DSnowden
JSGustafson
L
et al. Frontotemporal lobar degeneration: a consensus on clinical diagnostic criteria.
Neurology 1998;51
(6)
1546- 1554
PubMedGoogle ScholarCrossref 12.McKhann
GDrachman
DFolstein
MKatzman
RPrice
DStadlan
EM Mental and clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of the Department of Health and Human Services Task Force on Alzheimer's Disease.
Neurology 1984;34
(7)
939- 944
PubMedGoogle ScholarCrossref 13.McKhann
GMAlbert
MSGrossman
MMiller
BDickson
DTrojanowski
JQWork Group on Frontotemporal Dementia and Pick's Disease, Clinical and pathological diagnosis of frontotemporal dementia: report of the Work Group on Frontotemporal Dementia and Pick's Disease.
Arch Neurol 2001;58
(11)
1803- 1809
PubMedGoogle ScholarCrossref 14.Harville
DA Matrix Algebra From a Statistician's Perspective. New York, NY Springer Publishing1997;
16.Mendez
MFLauterbach
ECSampson
SMANPA Committee on Research, An evidence-based review of the psychopathology of frontotemporal dementia: a report of the ANPA Committee on Research.
J Neuropsychiatry Clin Neurosci 2008;20
(2)
130- 149
PubMedGoogle ScholarCrossref 17.Hollingworth
PHamshere
MLMoskvina
V
et al. Four components describe behavioral symptoms in 1,120 individuals with late-onset Alzheimer's disease.
J Am Geriatr Soc 2006;54
(9)
1348- 1354
PubMedGoogle ScholarCrossref 18.Snowden
JSBathgate
DVarma
ABlackshaw
AGibbons
ZCNeary
D Distinct behavioural profiles in frontotemporal dementia and semantic dementia.
J Neurol Neurosurg Psychiatry 2001;70
(3)
323- 332
PubMedGoogle ScholarCrossref 19.Davis
JDTremont
G Impact of frontal systems behavioral functioning in dementia on caregiver burden.
J Neuropsychiatry Clin Neurosci 2007;19
(1)
43- 49
PubMedGoogle ScholarCrossref 20.Klunk
WEEngler
HNordberg
A
et al. Imaging brain amyloid in Alzheimer's disease with Pittsburgh Compound-B.
Ann Neurol 2004;55
(3)
306- 319
PubMedGoogle ScholarCrossref 21.Royall
DRPalmer
RMulroy
AR
et al. Pathological determinants of the transition to clinical dementia in Alzheimer's disease.
Exp Aging Res 2002;28
(2)
143- 162
PubMedGoogle ScholarCrossref 22.Mourik
JCRosso
SMNiermeijer
MFDuivenvoorden
HJvan Swieten
JCTibben
A Frontotemporal dementia: behavioral symptoms and caregiver distress.
Dement Geriatr Cogn Disord 2004;18
(3-4)
299- 306
PubMedGoogle ScholarCrossref