[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.211.168.204. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1.
Performance on the California Verbal Learning Test Total Recall for All Participants by Group
Performance on the California Verbal Learning Test Total Recall for All Participants by Group

The California Verbal Learning Test scores were significantly worse in athletes compared with control participants (P = .002), athletes with mild cognitive impairment (MCI) compared with athletes without MCI (P < .001), and athletes with MCI compared with control participants (P < .001). The horizontal line in the middle of each box indicates the median; the top and bottom borders of each box mark the 75th and 25th percentiles, respectively; the whiskers above and below each box indicate the 90th and 10th percentiles, respectively; and the circles are data for individuals.

Figure 2.
Hippocampal Volume as a Function of Age for Control Participants and Athletes with Grade 3 Concussions
Hippocampal Volume as a Function of Age for Control Participants and Athletes with Grade 3 Concussions

A, Left hippocampal volume. B, Right hippocampal volume. Average hippocampal volume was compared between athletes with grade 3 concussions and control participants at the 20th, 40th, 60th, and 80th age percentiles across the age distribution. The hippocampal means at each age percentile were estimated by group-specific regressions. Error bars indicate the standard error of the mean and significant P values are displayed directly above each comparison between conditional means at the 40th, 60th, and 80th percentiles.

Figure 3.
Hippocampal Volume as a Function of Age for Retired Athletes With and Without Grade 3 Concussions
Hippocampal Volume as a Function of Age for Retired Athletes With and Without Grade 3 Concussions

Average hippocampal volume was compared between athletes with grade 3 (G3) concussions and athletes without G3 concussions at the 20th, 40th, 60th, and 80th age percentiles across the age distribution. The hippocampal means at each age percentile were estimated by group-specific regressions. Error bars indicate standard error of the mean and significant P values are displayed directly above each comparison between conditional means at the 40th, 60th, and 80th percentiles.

Figure 4.
Scatterplot of Raw Total CVLT Scores as a Function of Age
Scatterplot of Raw Total CVLT Scores as a Function of Age

Average California Verbal Learning Test (CVLT) scores were compared between athletes and control participants at the 20th, 40th, 60th, and 80th age percentiles across the age spectrum. The mean scores at each age percentile were estimated by group-specific regressions. Error bars indicate standard error of the mea, and significant P values are displayed directly above each comparison between conditional means at the 40th, 60th, and 80th percentiles.

Table.  
Demographic Characteristics
Demographic Characteristics
1.
Hart  J  Jr, Kraut  MA, Womack  KB,  et al.  Neuroimaging of cognitive dysfunction and depression in aging retired National Football League players: a cross-sectional study. JAMA Neurol. 2013;70(3):326-335.
PubMedArticle
2.
Iverson  GL, Brooks  BL, Collins  MW, Lovell  MR.  Tracking neuropsychological recovery following concussion in sport. Brain Inj. 2006;20(3):245-252.
PubMedArticle
3.
Klein  M, Houx  PJ, Jolles  J.  Long-term persisting cognitive sequelae of traumatic brain injury and the effect of age. J Nerv Ment Dis. 1996;184(8):459-467.
PubMedArticle
4.
Kraus  MF, Susmaras  T, Caughlin  BP, Walker  CJ, Sweeney  JA, Little  DM.  White matter integrity and cognition in chronic traumatic brain injury: a diffusion tensor imaging study. Brain. 2007;130(pt 10):2508-2519.
PubMedArticle
5.
Geary  EK, Kraus  MF, Pliskin  NH, Little  DM.  Verbal learning differences in chronic mild traumatic brain injury. J Int Neuropsychol Soc. 2010;16(3):506-516.
PubMedArticle
6.
Johnson  VE, Stewart  JE, Begbie  FD, Trojanowski  JQ, Smith  DH, Stewart  W.  Inflammation and white matter degeneration persist for years after a single traumatic brain injury. Brain. 2013;136(pt 1):28-42.
PubMedArticle
7.
Levin  HS, Gary  HE  Jr, Eisenberg  HM,  et al.  Neurobehavioral outcome 1 year after severe head injury: experience of the Traumatic Coma Data Bank. J Neurosurg. 1990;73(5):699-709.
PubMedArticle
8.
DeKosky  ST, Abrahamson  EE, Ciallella  JR,  et al.  Association of increased cortical soluble abeta42 levels with diffuse plaques after severe brain injury in humans. Arch Neurol. 2007;64(4):541-544.
PubMedArticle
9.
Franz  G, Beer  R, Kampfl  A,  et al.  Amyloid beta 1-42 and tau in cerebrospinal fluid after severe traumatic brain injury. Neurology. 2003;60(9):1457-1461.
PubMedArticle
10.
Roberts  GW, Gentleman  SM, Lynch  A, Murray  L, Landon  M, Graham  DI.  β amyloid protein deposition in the brain after severe head injury: implications for the pathogenesis of Alzheimer’s disease. J Neurol Neurosurg Psychiatry. 1994;57(4):419-425.
PubMedArticle
11.
Blasko  I, Jellinger  K, Kemmler  G,  et al.  Conversion from cognitive health to mild cognitive impairment and Alzheimer’s disease: prediction by plasma amyloid beta 42, medial temporal lobe atrophy and homocysteine. Neurobiol Aging. 2008;29(1):1-11.
PubMedArticle
12.
Du  AT, Schuff  N, Amend  D,  et al.  Magnetic resonance imaging of the entorhinal cortex and hippocampus in mild cognitive impairment and Alzheimer’s disease. J Neurol Neurosurg Psychiatry. 2001;71(4):441-447.
PubMedArticle
13.
Convit  A, De Leon  MJ, Tarshish  C,  et al.  Specific hippocampal volume reductions in individuals at risk for Alzheimer’s disease. Neurobiol Aging. 1997;18(2):131-138.
PubMedArticle
14.
Omalu  BI, Hamilton  RL, Kamboh  MI, DeKosky  ST, Bailes  J.  Chronic traumatic encephalopathy (CTE) in a National Football League player: case report and emerging medicolegal practice questions. J Forensic Nurs. 2010;6(1):40-46.
PubMedArticle
15.
McKee  AC, Stern  RA, Nowinski  CJ,  et al.  The spectrum of disease in chronic traumatic encephalopathy. Brain. 2013;136(pt 1):43-64.
PubMedArticle
16.
Devanand  DP, Pradhaban  G, Liu  X,  et al.  Hippocampal and entorhinal atrophy in mild cognitive impairment: prediction of Alzheimer disease. Neurology. 2007;68(11):828-836.
PubMedArticle
17.
Chetelat  G, Desgranges  B, de la Sayette  V,  et al.  Dissociating atrophy and hypometabolism impact on episodic memory in mild cognitive impairment. Brain. 2003;126(pt 9):1955-1967.
PubMedArticle
18.
Vanderploeg  RD, Donnell  AJ, Belanger  HG, Curtiss  G.  Consolidation deficits in traumatic brain injury: the core and residual verbal memory defect. J Clin Exp Neuropsychol. 2014;36(1):58-73.
PubMedArticle
19.
Singh  R, Meier  TB, Kuplicki  R,  et al.  Relationship of collegiate football experience and concussion with hippocampal volume and cognitive outcomes. JAMA. 2014;311(18):1883-1888.
PubMedArticle
20.
Maller  JJ, Reglade-Meslin  C.  Longitudinal hippocampal and fornix changes after traumatic brain injury: observations from traditional structural magnetic resonance imaging. J Neurol Neurophysiol. 2014;5(1):1-8.
21.
Himanen  L, Portin  R, Isoniemi  H, Helenius  H, Kurki  T, Tenovuo  O.  Cognitive functions in relation to MRI findings 30 years after traumatic brain injury. Brain Inj. 2005;19(2):93-100.
PubMedArticle
22.
Senathi-Raja  D, Ponsford  J, Schönberger  M.  Impact of age on long-term cognitive function after traumatic brain injury. Neuropsychology. 2010;24(3):336-344.
PubMedArticle
23.
Strain  J, Didehbani  N, Cullum  CM,  et al.  Depressive symptoms and white matter dysfunction in retired NFL players with concussion history. Neurology. 2013;81(1):25-32.
PubMedArticle
24.
Didehbani  N, Munro Cullum  C, Mansinghani  S, Conover  H, Hart  J  Jr.  Depressive symptoms and concussions in aging retired NFL players. Arch Clin Neuropsychol. 2013;28(5):418-424.
PubMedArticle
25.
Grundman  M, Petersen  RC, Ferris  SH,  et al; Alzheimer’s Disease Cooperative Study.  Mild cognitive impairment can be distinguished from Alzheimer disease and normal aging for clinical trials. Arch Neurol. 2004;61(1):59-66.
PubMedArticle
26.
Albert  MS, DeKosky  ST, Dickson  D,  et al.  The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):270-279.
PubMedArticle
27.
Quality Standards Subcommittee, American Academy of Neurology.  Practice parameter: the management of concussion in sports (summary statement). report of the Quality Standards Subcommittee. Neurology. 1997;48(3):581-585.
PubMedArticle
28.
Loring  DW, Martin  RC, Meador  KJ, Lee  GP.  Psychometric construction of the Rey-Osterrieth Complex Figure: methodological considerations and interrater reliability. Arch Clin Neuropsychol. 1990;5(1):1-14.
PubMedArticle
29.
Kaplan E, Goodglass H, Weintraub S, Segal O. Boston Naming Test. Philadelphia, PA: Lea & Febiger; 1983.
30.
Kraut  MA, Kremen  S, Segal  JB, Calhoun  V, Moo  LR, Hart  J  Jr.  Object activation from features in the semantic system. J Cogn Neurosci. 2002;14(1):24-36.
PubMedArticle
31.
Patenaude  B, Smith  SM, Kennedy  DN, Jenkinson  M.  A Bayesian model of shape and appearance for subcortical brain segmentation. Neuroimage. 2011;56(3):907-922.
PubMedArticle
32.
Lucarelli  RT, Peshock  RM, McColl  R,  et al.  MR imaging of hippocampal asymmetry at 3T in a multiethnic, population-based sample: results from the Dallas Heart Study. AJNR Am J Neuroradiol. 2013;34(4):752-757.
PubMedArticle
33.
Morey  RA, Petty  CM, Xu  Y,  et al.  A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes. Neuroimage. 2009;45(3):855-866.
PubMedArticle
34.
Whitwell  JL, Crum  WR, Watt  HC, Fox  NC.  Normalization of cerebral volumes by use of intracranial volume: implications for longitudinal quantitative MR imaging. AJNR Am J Neuroradiol. 2001;22(8):1483-1489.
PubMed
35.
Zhang  Y, Brady  M, Smith  S.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans Med Imaging. 2001;20(1):45-57.
PubMedArticle
36.
Bigler  ED, Blatter  DD, Anderson  CV,  et al.  Hippocampal volume in normal aging and traumatic brain injury. AJNR Am J Neuroradiol. 1997;18(1):11-23.
PubMed
37.
Vythilingam  M, Luckenbaugh  DA, Lam  T,  et al.  Smaller head of the hippocampus in Gulf War-related posttraumatic stress disorder. Psychiatry Res. 2005;139(2):89-99.
PubMedArticle
38.
Chen  KH, Chuah  LY, Sim  SK, Chee  MW.  Hippocampal region-specific contributions to memory performance in normal elderly. Brain Cogn. 2010;72(3):400-407.
PubMedArticle
39.
Ariza  M, Serra-Grabulosa  JM, Junqué  C,  et al.  Hippocampal head atrophy after traumatic brain injury. Neuropsychologia. 2006;44(10):1956-1961.
PubMedArticle
40.
Ibarretxe-Bilbao  N, Ramírez-Ruiz  B, Tolosa  E,  et al.  Hippocampal head atrophy predominance in Parkinson’s disease with hallucinations and with dementia. J Neurol. 2008;255(9):1324-1331.
PubMedArticle
41.
Buffalo  EA, Reber  PJ, Squire  LR.  The human perirhinal cortex and recognition memory. Hippocampus. 1998;8(4):330-339.
PubMedArticle
42.
Iverson  GL, Echemendia  RJ, Lamarre  AK, Brooks  BL, Gaetz  MB.  Possible lingering effects of multiple past concussions. Rehabil Res Pract. 2012;2012:316575.
PubMed
43.
MacKenzie  JD, Siddiqi  F, Babb  JS,  et al.  Brain atrophy in mild or moderate traumatic brain injury: a longitudinal quantitative analysis. AJNR Am J Neuroradiol. 2002;23(9):1509-1515.
PubMed
44.
Lehman  EJ, Hein  MJ, Baron  SL, Gersic  CM.  Neurodegenerative causes of death among retired National Football League players. Neurology. 2012;79(19):1970-1974.
PubMedArticle
45.
Randolph  C.  Is chronic traumatic encephalopathy a real disease? Curr Sports Med Rep. 2014;13(1):33-37.
PubMed
46.
Nemetz  PN, Leibson  C, Naessens  JM,  et al.  Traumatic brain injury and time to onset of Alzheimer’s disease: a population-based study. Am J Epidemiol. 1999;149(1):32-40.
PubMedArticle
47.
Robbins  CA, Daneshvar  DH, Picano  JD,  et al.  Self-reported concussion history: impact of providing a definition of concussion. Open Access J Sports Med. 2014;5:99-103.
PubMedArticle
48.
Llewellyn  T, Burdette  GT, Joyner  AB, Buckley  TA.  Concussion reporting rates at the conclusion of an intercollegiate athletic career. Clin J Sport Med. 2014;24(1):76-79.
PubMedArticle
49.
Kerr  ZY, Marshall  SW, Guskiewicz  KM.  Reliability of concussion history in former professional football players. Med Sci Sports Exerc. 2012;44(3):377-382.
PubMedArticle
Original Investigation
July 2015

Imaging Correlates of Memory and Concussion History in Retired National Football League Athletes

Author Affiliations
  • 1Center for BrainHealth, School of Behavioral and Brain Sciences, The University of Texas at Dallas, Dallas
  • 2Department of Psychiatry, University of Texas Southwestern Medical Center, Dallas
  • 3Department of Neurology and Neurotherapeutics, University of Texas Southwestern Medical Center, Dallas
  • 4Department of Radiology, Johns Hopkins School of Medicine, Baltimore, Maryland
JAMA Neurol. 2015;72(7):773-780. doi:10.1001/jamaneurol.2015.0206
Abstract

Importance  To our knowledge, this is the first study to show an association between concussion, cognition, and anatomical structural brain changes across the age spectrum in former National Football League athletes.

Objective  To assess the relationship of hippocampal volume, memory performance, and the influence of concussion history in retired National Football League athletes with and without mild cognitive impairment (MCI).

Design, Setting, and Participants  This retrospective cohort study assessed differences between groups, mean hippocampal volumes, and memory performance by computing age quintiles based on group-specific linear regression models corrected for multiple comparisons for both athletes and control participants. The study was conducted starting in November 2010 and is ongoing at a research center in the northern region of Texas. This current analysis was conducted from October 9, 2013, to August 21, 2014. Participants included 28 retired National Football League athletes, 8 of whom had MCI and a history of concussion, 21 cognitively healthy control participants, and 6 control participants with MCI without concussion.

Main Outcomes and Measures  Hippocampal volume, age, California Verbal Learning Test scores, and the number of grade 3 (G3) concussions. In addition, the number of games played was examined as an objective variable pertaining to football history.

Results  The mean (SD) age was 58.1 (13) years for the 28 former athletes and 59.0 (12) years for the 27 control participants. Retired athletes with concussion history but without cognitive impairment had normal but significantly lower California Verbal Learning Test scores compared with control participants (mean [SD], 52.5 [8] vs 60.24 [7]; P = .002); those with a concussion history and MCI performed worse (mean [SD], 37 [8.62]) compared with both control participants (P < .001) and athletes without memory impairment (P < .001). Among the athletes, 17 had a G3 concussion and 11 did not. Older retired athletes with at least 1 G3 concussion had significantly smaller bilateral hippocampal volumes compared with control participants at the 40th age percentile (left, P = .04; right, P = .03), 60th percentile (left, P = .009; right, P = .01), and 80th percentile (left, P = .001; right, P = .002) and a smaller right hippocampal volume compared with athletes without a G3 concussion at the 40th percentile (P = .03), 60th percentile (P = .02), and 80th percentile (P = .02). Athletes with a history of G3 concussion were more likely to have MCI (7 of 7) compared with retired athletes without a history of G3 concussion (1 of 5) older than 63 years (P = .01). In addition, the left hippocampal volume in retired athletes with MCI and concussion was significantly smaller compared with control participants with MCI (P = .03).

Conclusion and Relevance  Prior concussion that results in loss of consciousness is a risk factor for increased hippocampal atrophy and the development of MCI. In individuals with MCI, hippocampal volume loss appears greater among those with a history of concussion.

Introduction

Quiz Ref IDChronic neurobehavioral sequelae from concussions range from good recovery to cognitive dysfunction and mood symptoms to postconcussive syndromes.1 While most individuals recover completely within days or weeks of a concussion, a small subset demonstrates residual neurobehavioral changes.2 One of the most common deficits following traumatic brain injury (TBI) is a disturbance in episodic memory, which may persist years after injury,35 although the potential mechanism for this is poorly understood (eg, white matter dysfunction, inflammation, and aging effects).6,7

Traumatic brain injury has also been reported as a risk factor for dementia, including Alzheimer disease (AD) and chronic traumatic encephalopathy later in life. The presence of abnormal β-amyloid 42 and phosphorylated tau protein deposition after acute injury8,9 links TBI with a pathology similar to AD. These protein depositions in TBI are more prevalent in the medial temporal lobe,10 associated with medial temporal lobe atrophy,11 and characteristic of AD in the absence of TBI.12 Atrophy in this region is also present in patients with mild cognitive impairment (MCI)13 and is indicative of progression from MCI to AD.14Quiz Ref ID While chronic traumatic encephalopathy is defined pathologically by cortical tau deposition and has been seen in the context of repetitive TBI, the clinical characterization is incomplete.15,16

The association of concussion in the subsequent development of memory dysfunction with hippocampal atrophy remains poorly understood. The goal of this study was to assess the relationship of memory performance with hippocampal volume coupled with the influence of concussion history in retired National Football League (NFL) athletes with and without a diagnosis of MCI. We studied this in conjunction with assessments of concussion history, duration of time in the NFL, and magnetic resonance imaging–derived measures of hippocampal volume because the latter measure can be associated with episodic memory impairment.17,18

In a recent study, smaller hippocampi were found in collegiate athletes with concussion in contrast to athletes without a history of concussion.19 Hippocampal volume loss and expansion of the temporal horns are often seen in TBI and frequently correlate with episodic memory impairment.20,21 This reduction in cognitive ability is speculated to progress at a more sudden rate than typically seen in normal aging, suggesting that effects of TBI superimpose deleteriously on the aging process.3,22 Therefore, we sought to understand the relationship between hippocampal volume loss and episodic memory performance in retired players across the age distribution.

Based on the findings that hippocampal volume loss is evident in both TBI and MCI, we also assessed the integrity of the hippocampus among former players diagnosed as having MCI compared with a control MCI group without a history of concussion. We hypothesized that hippocampal loss would be greater in those with a history of concussion.

Methods

The current study was conducted from October 9, 2013, to August 21, 2014. We recruited 40 retired NFL athletes from a local gathering of retired NFL athletes living in the northern region of Texas from meetings of the NFL Athletes Association local chapter, through local advertising, and by word of mouth. Two former athletes who showed clinical symptoms consistent with AD and 1 athlete diagnosed as having nonamnestic MCI and showing an isolated impairment in naming were excluded. Nine additional former athletes were excluded owing to a diagnosis of depression and 1 was excluded owing to excessive head movement during scanning. More information on the athletes with depression is described elsewhere.23,24

All participants provided written informed consent in accordance with the Declaration of Helsinki. The institutional review boards of the University of Texas Southwestern Medical Center and the University of Texas at Dallas approved the study protocols and consent forms.

Our final sample comprised 28 former athletes, 8 of whom were clinically diagnosed as having amnestic MCI25 with a concussion history. The remaining 20 had no memory impairment as determined by neurological and neuropsychological evaluation. Of the 20 athletes without memory impairment, 4 reported cognitive symptoms that occurred after concussion; however, these 4 were not progressively declining and did not meet clinical criteria for a neurodegenerative disorder. They exhibited static word-finding deficits, did not meet clinical criteria for MCI,26 and were not impaired on formal memory assessment.

Former NFL athletes ranged in age from 36 to 79 years (mean [SD], 58.1 [13] years), education ranged from 15 to 18 years (mean [SD], 16.5 [0.9] years), and estimated IQ ranged from 92 to 126 (mean [SD], 111 [10]). Experience in the NFL ranged from 2 to 15 years (mean [SD], 8.9 [4.2] years). Nineteen athletes were white and 9 were African American. Concussion history was obtained from self-reports (Table) and classified using the 1997 American Academy of Neurology practice parameter guidelines for grading concussion.27 A maximum number for grade 1 concussions was set at 10 for 2 participants who reported too many to count. Concussion severity ranged from brief periods of confusion to loss of consciousness for several hours. All but 3 former athletes experienced at least 1 concussion (mean [SD], 3.85 [3.47] concussions).

Twenty-one cognitively healthy control participants with no history of concussion or past participation in college or professional football were recruited as a comparison group for the retired athletes. The comparison group was matched for education and IQ and had no history of mental illness, cognitive complaints, or neurological disorders. The healthy control group ranged in age from 41 to 77 years (mean [SD], 59.0 [12] years) and education spanned from 12 to 20 years (mean [SD], 15.9 [2.4] years). Two control participants were African American and 19 were white.

An age- and sex-matched MCI group with no history of concussion (n = 6) was recruited from the Alzheimer’s Disease Center at The University of Texas Southwestern Medical Center for comparison with athletes with a history of concussion diagnosed as having MCI. Participants with MCI were diagnosed via multidisciplinary consensus conference and ranged in age from 55 to 77 years (mean [SD], 68.0 [8.3] years).26

Four neuropsychological tests of episodic memory and word retrieval were selected from our original cohort described by Hart et al1 because they showed differences between former athletes and control participants. The California Verbal Learning Test (CVLT) Second Edition and the Rey-Osterrieth Complex Figure Test were used to evaluate verbal and nonverbal episodic memory, respectively. The following 3 primary scores from the CVLT were used to assess verbal memory: total t score from the 5 learning trials as well as short- and long-delay free-recall z scores. The Rey-Osterrieth Complex Figure Test was scored according to the criteria by Loring et al28 and long-delay recall was used as the primary score. In addition to episodic memory tasks, the standard 60-item Boston Naming Test and the Semantic Object Retrieval Test were included as assessments of semantic processing.2931

High-resolution 3-dimensional magnetization-prepared rapid-acquisition gradient-echo images were collected in the sagittal plane using a standard SENSE 8-channel head coil. All images were acquired on a 3-T scanner (Phillips) with 120 slices; repetition time, 8.2 milliseconds; echo time, 3.8 milliseconds; 1.0-mm slice thickness; field of view, 256 mm2; and voxel dimension, 1.0 × 1.0 × 1.0 mm.

We used a semiautomated method that combined the Functional Magnetic Resonance Imaging of the Brain (FMRIB) integrated registration and segmentation tool from the FMRIB software library with manual editing.31 The image data were segmented into cerebral spinal fluid, white matter, and gray matter and warped to standard space. Registrations were carried out with a 2-stage affine registration involving the FMRIB linear image registration tool from the FMRIB software library. In common space, a mask for the hippocampus was modeled for each participant using a Bayesian probabilistic approach. A hippocampal surface mesh was then reconstructed from modeled data provided by the Center for Morphometric Analysis and back projected to the original participant space. Manual edits were applied when necessary on visual inspection of the results of the automated process.32,33 The same individual blinded to demographic information but not group classification manually edited all hippocampi, and intrarater reliability was calculated using Cronbach α.

Hippocampal volumes were then corrected for head size by dividing each participant’s total intracranial volume.34 Each participant’s total intracranial volume was calculated using the FMRIB automated segmentation tool from the FMRIB software library and by totaling each tissue volume to quantify head size and account for cortical atrophy.35 A forward/backward stepwise linear regression was performed to identify significant variables associated with hippocampal volume in athletes. We used the Bayesian information criterion to screen the set of variables and choose those with the best predictive ability. For this model, age, concussion history, selected CVLT scores, Rey-Osterrieth Complex Figure Test delayed recall, Boston Naming Test total scores, and Semantic Object Retrieval Test retrieval scores were assessed. Variables that did not survive our variable selection were not included in further analyses.

Neuropsychological performance was compared between retired athletes and control participants via independent-sample t tests. Differences in hippocampal volumes between athletes with MCI and control participants with MCI were assessed by 2-sample t tests. Hippocampal volumes were also compared across the age distribution for both athletes and control participants by first imposing the following group-specific linear regression models:

yij = μi|X + εij

where:

μi|X = ai + biX

was the mean hippocampal volume for group i at age X and the j = 1,…,ni subject-level errors were assumed independent.

To assess the interaction between age and concussion history, we used a method that evaluated hippocampal volume across the age distribution. To better match the control group to athletes, we added the control participants with MCI to the control group. This was done because some of our athletes had MCI and we wanted to counterbalance the effect of MCI. Specifically, participants were divided into age quintiles, with equal representation for both control participants and athletes demarcated at different percentile (p) intervals across the following age distribution: X = x(p) for (p) = 20th percentile, 40th percentile, 60th percentile, and 80th percentile. Using each group-specific regression line, we plotted each age percentile to estimate the hippocampal volume or conditional mean μ̂i|X for each group. We then computed a t statistic on ν = 48 df as:

Image description not available.

where the standard error (SE) was a function of the variances and covariance of the estimates:

Image description not available.

and each term under the root was conditional on X = x(p). We adjusted the P values for the 4 comparisons across the age distribution using a false discovery rate correction.

Results

Retired athletes without MCI and control participants were not significantly different in IQ, education, or age. Likewise, the MCI groups with and without concussion history did not differ in age or education. Using Bayesian information criterion criteria, only the following 3 variables were found to be predictors of hippocampal volume bilaterally in athletes without memory impairment: age, CVLT total score, and grade 3 (G3) concussions. For the remainder of this article, we focus exclusively on these 3 variables, with 1 additional variable to represent time spent in the NFL. To examine the contributions of variables identified by the Bayesian information criterion screen, we categorized all athletes regardless of cognitive status as to whether the athlete reported experiencing a G3 concussion (n = 17) or not (n = 11). We added 1 additional variable, the number of games played in the NFL, as a measure of career duration because this was a more refined metric and highly correlated with years in the NFL.

Former athletes with concussion history but without cognitive impairment performed in the healthy range but obtained lower scores on the CVLT (mean [SD], 52.5 [8]) than control participants (mean [SD], 60.24 [7]) (P = .002). Athletes with a concussion history and MCI performed worse on the CVLT (mean [SD], 37 [8.62]) compared with both control participants (P < .001) and athletes without memory impairment (P < .001). No difference was found between control participants with MCI and athletes with MCI on the CVLT. The distribution of scores for the groups along with individual scores is shown in Figure 1.

Interrater reliability ratings for the manual modifications after the automated process were high in both hippocampi for 3 randomly selected athletes (left, mean = 0.93; right, mean = 0.99). Former athletes without a G3 concussion showed similar hippocampal volumes compared with control participants across the age distribution as assessed by conditional averages from the group-specific regression models tested at each age percentile. However, former athletes with at least 1 G3 concussion had lower mean hippocampal volumes bilaterally compared with control participants at the 40th age percentile (left, P = .04; right, P = .03), 60th percentile (left, P = .009; right, P = .01), and 80th percentile (left, P = .001; right, P = .002) (Figure 2). Former athletes with a G3 concussion also had significantly lower hippocampal volumes on the right compared with former athletes without a G3 concussion at the 40th percentile (P = .03), 60th percentile (P = .02), and 80th percentile (P = .02) (Figure 3). Similar findings were seen in the left hippocampus but only at a trend level after correction for multiple comparisons.

To assess whether the number of games played would show similar findings, we dichotomized all athletes using a median split to maintain consistency with our prior analysis (median, 120 games; 13 athletes with <120 games and 15 athletes with ≥120 games). Former athletes with a career of 120 or more games in the NFL had lower mean left hippocampal volumes compared with control participants for the 60th age percentile (P = .02) and 80th percentile (P = .001). Athletes with fewer than 120 games played in the NFL were not significantly different from control participants in hippocampal volume, and no difference was seen in hippocampal volume between the 2 athlete groups.

A similar analysis was performed for raw CVLT scores. Raw scores were used in place of t scores because the latter were already stratified for age and the focus of this analysis was the relationship between memory performance and age. No difference was found between athlete groups subdivided by either history of G3 concussion or number of games played for any age percentile. Therefore, when compared with control participants, all former athletes were combined into 1 group, excluding those with MCI because part of the clinical criteria for the diagnosis of MCI is verbal memory impairment and we did not want to bias our regression line. These athletes performed worse compared with control participants at the 40th age percentile (P = .005), 60th percentile (P = .003), and 80th percentile (P < .001) (Figure 4).

Quiz Ref IDAthletes with a history of concussion and MCI had a lower mean left hippocampal volume compared with control participants with MCI (P = .03), but the difference for the right hippocampus was nonsignificant (P = .08).

We also examined the association between MCI in those with a history of concussion and the presence of at least 1 G3 concussion among those older than 63 years (the minimum age at which we observed an athlete to have developed MCI) with a Fisher exact test. All of the retired athletes older than 63 years with a G3 concussion (7 of 7) were diagnosed as having MCI and only 1 other former athlete was diagnosed as having MCI but did not have a G3 concussion (1 of 5). A significant association was found between G3 concussion and MCI (P = .01), but no relationship was found between the number of games played and MCI. Additionally, athletes with more games played were unlikely to have a G3 concussion compared with those who played fewer games.

Discussion

In our sample of retired NFL athletes, we demonstrated that a history of concussion with loss of consciousness was associated with reduced hippocampal volume and lower verbal memory performance later in life. To our knowledge, this is the first study that has evaluated quantitatively the relationships between hippocampal volume and memory performance with concussion severity in retired NFL athletes. For older age groups, former athletes in our study had lower verbal memory scores compared with control participants, but only athletes with a G3 concussion showed significantly smaller hippocampal volumes on the right. Overall, the regression line for athletes without a G3 concussion was between the control regression line and the regression line for athletes with a G3 concussion. Additionally, former athletes with MCI had a smaller left hippocampus compared with control participants with MCI, suggesting that concussion history increases the risk of medial temporal atrophy in conjunction with abnormal pathology. A similar result was seen after reclassifying athletes based on the number of games played, which revealed that athletes who participated in more games had significantly smaller hippocampal volumes compared with control participants. However, history of a G3 concussion was associated with a clinical diagnosis of MCI, whereas the number of games played was not.

Quiz Ref IDOur cognitive battery included 2 tests of episodic memory, but only verbal memory was found to be a significant predictor of hippocampal volume, which is consistent with previous studies showing hippocampal volume loss associated with verbal rather than visual memory.3640 The hippocampus is just 1 region in the medial temporal lobe designated as important in episodic memory. Other structures, such as the parahippocampus and entorhinal cortex, have also been implicated.41 We focused solely on the hippocampus because of the plethora of data suggesting its susceptibility to traumatic events, but it is possible that the observed verbal memory deficit resulted from the disconnection of the hippocampus or other medial temporal lobe structures.

In this cohort we used a method that evaluated hippocampal volume across age distribution to assess the interaction between age and concussion history because we hypothesized that athletes with a history of concussion would exhibit patterns of unhealthy cognitive aging that would be more apparent in older groups. The analysis by quintiles demonstrated this accentuated age-related decline both anatomically and behaviorally, with significant differences found only in older groups. This abrupt aging trajectory has been reported in other TBI populations compared with matched control participants but has not been shown in former athletes with a history of concussion.22

One criterion for G3 concussion is loss of consciousness, which has been associated with poorer long-term outcomes,42,43 including an increased risk of developing dementia.44 Athletes who incurred G3 concussions demonstrated marked reductions in hippocampal volumes and lower CVLT scores. In TBI, changes in hippocampal volume may not be apparent until approximately 100 days following injury.36 Further studies are needed to ascertain the temporal component in the pathophysiological changes that occur in conjunction with aging in those who have experienced a G3 concussion.

The actual number of impacts an individual experiences throughout an NFL career is unquantifiable, but the number of games played in the NFL is an objective measure that we used as a proxy for contact exposure. Dichotomizing our athlete group in terms of games played did yield a significant difference in unilateral hippocampal volume compared with control participants, but no significant difference was found between many vs fewer games played. In addition, no relationship was found between games played and MCI or G3 concussions. We infer from these data that G3 concussions and the number of games played explain different sources of variance, with G3 concussions being more associated with cognitive decline across time.

Control participants with MCI and athletes with MCI both had the same clinical diagnosis but may have had different types or degrees of pathology.25 The prevalence of dementia in former football players has been suggested to be higher than in the general population,44 but this is a controversial issue.45 Although it is unclear for concussion, more serious TBI is a risk factor for developing dementia and can be associated with slightly earlier dementia onset.46 Our data suggested that 2 key factors in this process are G3 concussion and age. After age 63 years, athletes with a G3 concussion were significantly more likely to have MCI. The likely interpretation is that in MCI without a history of concussion, hippocampal volume loss is neurodegenerative based, with age as the key factor. In those with a history of repetitive head trauma and G3 concussion, while initial behavioral deficits may not be evident at the time of injury, increasing age leads to development of clinically relevant deficits. An alternative account is that the combination of age and previous disruptions from concussion leads to a unique process resulting in degeneration with greater hippocampal volume loss.

The association of age with developing degenerative disease could be clarified with future longitudinal studies. Knowledge of hippocampal volumes prior to head trauma would also benefit the analyses. We cannot deny the possibility that our participants with concussion and MCI were predisposed for developing MCI with smaller hippocampi prior to involvement in the NFL because this cannot be determined without pretrauma imaging. Although our data were limited in addressing this issue, the association between G3 concussions and diagnosis of MCI in a region vulnerable to head impacts bolsters the possibility that this could represent a degenerative response elicited from a G3 concussion.

There have been concerns regarding the accuracy of self-reported concussions without more substantial evidence beyond subjective recall. Studies have noted both underreporting (eg, lack of understanding about concussion) and exaggeration of head impacts potentially driven by media and/or litigation influence.47,48 One study showed that reevaluation of concussion history yielded moderate reliability after classifying responses into 3 categories (0, 1-2, and ≥3) based on total reported concussions.49 Categorizing variables can potentially improve the reliability of retrospective subjective scores, and we used a similar technique that avoided most issues with self-reported concussions. Specifically, we focused on state of consciousness, which was a binary response (“Were you ever knocked unconscious from a concussion?”). We compared the responses of all athletes who returned for our longitudinal assessment and tested the reliability on this aspect of concussion. All 25 athletes reported the same answer they gave at the initial assessment, validating that a retrospective report of G3 concussions assessed in this manner appeared to be a reliable metric, particularly in lieu of objective documentation of concussion, which is rarely available.

Conclusions

Our findings suggest that a remote history of concussion with loss of consciousness is associated with both later-in-life decreases in hippocampal volume and memory performance in retired NFL football players. The number of games played was negatively associated with hippocampal size but not as strongly as a history of G3 concussion because these changes in anatomical structure and cognitive function only become evident with aging. Quiz Ref IDOur findings further show that a history of G3 concussion in athletes with MCI was associated with greater hippocampal volume loss compared with control participants with MCI. Prospective longitudinal studies after a G3 concussion would add further insight to the mechanism of MCI development in these populations.

Back to top
Article Information

Corresponding Author: C. Munro Cullum, PhD, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390-9044 (munro.cullum@utsouthwestern.edu).

Accepted for Publication: February 12, 2015.

Published Online: May 18, 2015. doi:10.1001/jamaneurol.2015.0206.

Author Contributions: Mr Strain and Dr Spence had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Strain, Womack, Conover, Hart, Kraut, Cullum.

Acquisition, analysis, or interpretation of data: Strain, Womack, Didehbani, Spence, Hart, Kraut, Cullum.

Drafting of the manuscript: Strain, Hart, Kraut, Cullum.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Strain, Spence, Hart.

Obtained funding: Hart.

Administrative, technical, or material support: Didehbani, Conover, Hart,

Study supervision: Womack, Spence, Hart, Kraut, Cullum.

Conflict of Interest Disclosures: None reported.

Funding/Support: This project was supported by the BrainHealth Institute for Athletes at the Center for BrainHealth, a research center at The University of Texas at Dallas, and grant P30Ag12300-19 from the National Institute on Aging.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the retired National Football League athletes who participated in the study and David Kennedy, PhD, the Center for Morphometric Analysis, for training data for the Functional Magnetic Resonance Imaging of the Brain integrated registration and segmentation tool. None of the individuals received financial compensation for their contributions.

References
1.
Hart  J  Jr, Kraut  MA, Womack  KB,  et al.  Neuroimaging of cognitive dysfunction and depression in aging retired National Football League players: a cross-sectional study. JAMA Neurol. 2013;70(3):326-335.
PubMedArticle
2.
Iverson  GL, Brooks  BL, Collins  MW, Lovell  MR.  Tracking neuropsychological recovery following concussion in sport. Brain Inj. 2006;20(3):245-252.
PubMedArticle
3.
Klein  M, Houx  PJ, Jolles  J.  Long-term persisting cognitive sequelae of traumatic brain injury and the effect of age. J Nerv Ment Dis. 1996;184(8):459-467.
PubMedArticle
4.
Kraus  MF, Susmaras  T, Caughlin  BP, Walker  CJ, Sweeney  JA, Little  DM.  White matter integrity and cognition in chronic traumatic brain injury: a diffusion tensor imaging study. Brain. 2007;130(pt 10):2508-2519.
PubMedArticle
5.
Geary  EK, Kraus  MF, Pliskin  NH, Little  DM.  Verbal learning differences in chronic mild traumatic brain injury. J Int Neuropsychol Soc. 2010;16(3):506-516.
PubMedArticle
6.
Johnson  VE, Stewart  JE, Begbie  FD, Trojanowski  JQ, Smith  DH, Stewart  W.  Inflammation and white matter degeneration persist for years after a single traumatic brain injury. Brain. 2013;136(pt 1):28-42.
PubMedArticle
7.
Levin  HS, Gary  HE  Jr, Eisenberg  HM,  et al.  Neurobehavioral outcome 1 year after severe head injury: experience of the Traumatic Coma Data Bank. J Neurosurg. 1990;73(5):699-709.
PubMedArticle
8.
DeKosky  ST, Abrahamson  EE, Ciallella  JR,  et al.  Association of increased cortical soluble abeta42 levels with diffuse plaques after severe brain injury in humans. Arch Neurol. 2007;64(4):541-544.
PubMedArticle
9.
Franz  G, Beer  R, Kampfl  A,  et al.  Amyloid beta 1-42 and tau in cerebrospinal fluid after severe traumatic brain injury. Neurology. 2003;60(9):1457-1461.
PubMedArticle
10.
Roberts  GW, Gentleman  SM, Lynch  A, Murray  L, Landon  M, Graham  DI.  β amyloid protein deposition in the brain after severe head injury: implications for the pathogenesis of Alzheimer’s disease. J Neurol Neurosurg Psychiatry. 1994;57(4):419-425.
PubMedArticle
11.
Blasko  I, Jellinger  K, Kemmler  G,  et al.  Conversion from cognitive health to mild cognitive impairment and Alzheimer’s disease: prediction by plasma amyloid beta 42, medial temporal lobe atrophy and homocysteine. Neurobiol Aging. 2008;29(1):1-11.
PubMedArticle
12.
Du  AT, Schuff  N, Amend  D,  et al.  Magnetic resonance imaging of the entorhinal cortex and hippocampus in mild cognitive impairment and Alzheimer’s disease. J Neurol Neurosurg Psychiatry. 2001;71(4):441-447.
PubMedArticle
13.
Convit  A, De Leon  MJ, Tarshish  C,  et al.  Specific hippocampal volume reductions in individuals at risk for Alzheimer’s disease. Neurobiol Aging. 1997;18(2):131-138.
PubMedArticle
14.
Omalu  BI, Hamilton  RL, Kamboh  MI, DeKosky  ST, Bailes  J.  Chronic traumatic encephalopathy (CTE) in a National Football League player: case report and emerging medicolegal practice questions. J Forensic Nurs. 2010;6(1):40-46.
PubMedArticle
15.
McKee  AC, Stern  RA, Nowinski  CJ,  et al.  The spectrum of disease in chronic traumatic encephalopathy. Brain. 2013;136(pt 1):43-64.
PubMedArticle
16.
Devanand  DP, Pradhaban  G, Liu  X,  et al.  Hippocampal and entorhinal atrophy in mild cognitive impairment: prediction of Alzheimer disease. Neurology. 2007;68(11):828-836.
PubMedArticle
17.
Chetelat  G, Desgranges  B, de la Sayette  V,  et al.  Dissociating atrophy and hypometabolism impact on episodic memory in mild cognitive impairment. Brain. 2003;126(pt 9):1955-1967.
PubMedArticle
18.
Vanderploeg  RD, Donnell  AJ, Belanger  HG, Curtiss  G.  Consolidation deficits in traumatic brain injury: the core and residual verbal memory defect. J Clin Exp Neuropsychol. 2014;36(1):58-73.
PubMedArticle
19.
Singh  R, Meier  TB, Kuplicki  R,  et al.  Relationship of collegiate football experience and concussion with hippocampal volume and cognitive outcomes. JAMA. 2014;311(18):1883-1888.
PubMedArticle
20.
Maller  JJ, Reglade-Meslin  C.  Longitudinal hippocampal and fornix changes after traumatic brain injury: observations from traditional structural magnetic resonance imaging. J Neurol Neurophysiol. 2014;5(1):1-8.
21.
Himanen  L, Portin  R, Isoniemi  H, Helenius  H, Kurki  T, Tenovuo  O.  Cognitive functions in relation to MRI findings 30 years after traumatic brain injury. Brain Inj. 2005;19(2):93-100.
PubMedArticle
22.
Senathi-Raja  D, Ponsford  J, Schönberger  M.  Impact of age on long-term cognitive function after traumatic brain injury. Neuropsychology. 2010;24(3):336-344.
PubMedArticle
23.
Strain  J, Didehbani  N, Cullum  CM,  et al.  Depressive symptoms and white matter dysfunction in retired NFL players with concussion history. Neurology. 2013;81(1):25-32.
PubMedArticle
24.
Didehbani  N, Munro Cullum  C, Mansinghani  S, Conover  H, Hart  J  Jr.  Depressive symptoms and concussions in aging retired NFL players. Arch Clin Neuropsychol. 2013;28(5):418-424.
PubMedArticle
25.
Grundman  M, Petersen  RC, Ferris  SH,  et al; Alzheimer’s Disease Cooperative Study.  Mild cognitive impairment can be distinguished from Alzheimer disease and normal aging for clinical trials. Arch Neurol. 2004;61(1):59-66.
PubMedArticle
26.
Albert  MS, DeKosky  ST, Dickson  D,  et al.  The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):270-279.
PubMedArticle
27.
Quality Standards Subcommittee, American Academy of Neurology.  Practice parameter: the management of concussion in sports (summary statement). report of the Quality Standards Subcommittee. Neurology. 1997;48(3):581-585.
PubMedArticle
28.
Loring  DW, Martin  RC, Meador  KJ, Lee  GP.  Psychometric construction of the Rey-Osterrieth Complex Figure: methodological considerations and interrater reliability. Arch Clin Neuropsychol. 1990;5(1):1-14.
PubMedArticle
29.
Kaplan E, Goodglass H, Weintraub S, Segal O. Boston Naming Test. Philadelphia, PA: Lea & Febiger; 1983.
30.
Kraut  MA, Kremen  S, Segal  JB, Calhoun  V, Moo  LR, Hart  J  Jr.  Object activation from features in the semantic system. J Cogn Neurosci. 2002;14(1):24-36.
PubMedArticle
31.
Patenaude  B, Smith  SM, Kennedy  DN, Jenkinson  M.  A Bayesian model of shape and appearance for subcortical brain segmentation. Neuroimage. 2011;56(3):907-922.
PubMedArticle
32.
Lucarelli  RT, Peshock  RM, McColl  R,  et al.  MR imaging of hippocampal asymmetry at 3T in a multiethnic, population-based sample: results from the Dallas Heart Study. AJNR Am J Neuroradiol. 2013;34(4):752-757.
PubMedArticle
33.
Morey  RA, Petty  CM, Xu  Y,  et al.  A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes. Neuroimage. 2009;45(3):855-866.
PubMedArticle
34.
Whitwell  JL, Crum  WR, Watt  HC, Fox  NC.  Normalization of cerebral volumes by use of intracranial volume: implications for longitudinal quantitative MR imaging. AJNR Am J Neuroradiol. 2001;22(8):1483-1489.
PubMed
35.
Zhang  Y, Brady  M, Smith  S.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans Med Imaging. 2001;20(1):45-57.
PubMedArticle
36.
Bigler  ED, Blatter  DD, Anderson  CV,  et al.  Hippocampal volume in normal aging and traumatic brain injury. AJNR Am J Neuroradiol. 1997;18(1):11-23.
PubMed
37.
Vythilingam  M, Luckenbaugh  DA, Lam  T,  et al.  Smaller head of the hippocampus in Gulf War-related posttraumatic stress disorder. Psychiatry Res. 2005;139(2):89-99.
PubMedArticle
38.
Chen  KH, Chuah  LY, Sim  SK, Chee  MW.  Hippocampal region-specific contributions to memory performance in normal elderly. Brain Cogn. 2010;72(3):400-407.
PubMedArticle
39.
Ariza  M, Serra-Grabulosa  JM, Junqué  C,  et al.  Hippocampal head atrophy after traumatic brain injury. Neuropsychologia. 2006;44(10):1956-1961.
PubMedArticle
40.
Ibarretxe-Bilbao  N, Ramírez-Ruiz  B, Tolosa  E,  et al.  Hippocampal head atrophy predominance in Parkinson’s disease with hallucinations and with dementia. J Neurol. 2008;255(9):1324-1331.
PubMedArticle
41.
Buffalo  EA, Reber  PJ, Squire  LR.  The human perirhinal cortex and recognition memory. Hippocampus. 1998;8(4):330-339.
PubMedArticle
42.
Iverson  GL, Echemendia  RJ, Lamarre  AK, Brooks  BL, Gaetz  MB.  Possible lingering effects of multiple past concussions. Rehabil Res Pract. 2012;2012:316575.
PubMed
43.
MacKenzie  JD, Siddiqi  F, Babb  JS,  et al.  Brain atrophy in mild or moderate traumatic brain injury: a longitudinal quantitative analysis. AJNR Am J Neuroradiol. 2002;23(9):1509-1515.
PubMed
44.
Lehman  EJ, Hein  MJ, Baron  SL, Gersic  CM.  Neurodegenerative causes of death among retired National Football League players. Neurology. 2012;79(19):1970-1974.
PubMedArticle
45.
Randolph  C.  Is chronic traumatic encephalopathy a real disease? Curr Sports Med Rep. 2014;13(1):33-37.
PubMed
46.
Nemetz  PN, Leibson  C, Naessens  JM,  et al.  Traumatic brain injury and time to onset of Alzheimer’s disease: a population-based study. Am J Epidemiol. 1999;149(1):32-40.
PubMedArticle
47.
Robbins  CA, Daneshvar  DH, Picano  JD,  et al.  Self-reported concussion history: impact of providing a definition of concussion. Open Access J Sports Med. 2014;5:99-103.
PubMedArticle
48.
Llewellyn  T, Burdette  GT, Joyner  AB, Buckley  TA.  Concussion reporting rates at the conclusion of an intercollegiate athletic career. Clin J Sport Med. 2014;24(1):76-79.
PubMedArticle
49.
Kerr  ZY, Marshall  SW, Guskiewicz  KM.  Reliability of concussion history in former professional football players. Med Sci Sports Exerc. 2012;44(3):377-382.
PubMedArticle
×