[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.
CONSORT Diagram of Participant Flowchart
CONSORT Diagram of Participant Flowchart
Figure 2.
Graphic Representation of Trial Method
Graphic Representation of Trial Method
Table 1.  
Study Measures
Study Measures
Table 2.  
Sample Characteristics of Children at Enrollment
Sample Characteristics of Children at Enrollment
Table 3.  
Outcome Comparisons At 6, 12, and 24 Months
Outcome Comparisons At 6, 12, and 24 Months
1.
Prior  M, Sanson  A, Smart  D, Oberklaid  F.  Reading disability in an Australian community sample.  Aust J Psychol. 1995;47(1):32-37. doi:10.1080/00049539508258766.Google ScholarCrossref
2.
4226.0—Aspects of literacy: profiles and perceptions, Australia, 1996. Australian Bureau of Statistics website. http://www.abs.gov.au/ausstats/abs@.nsf/mf/4226.0. Released April 30, 1997. Accessed January 29, 2016.
3.
Westwood  P.  What Teachers Need to Know About Learning Difficulties. Camberwell, Victoria, Australia: ACER Press; 2008.
4.
Gorman  JC.  Emotional Disorders and Learning Disabilities in the Elementary Classroom: Interactions and Interventions. Thousand Oaks, CA: Corwin Press; 2001.
5.
Dyck  MJ, Hay  D, Anderson  M, Smith  LM, Piek  J, Hallmayer  J.  Is the discrepancy criterion for defining developmental disorders valid?  J Child Psychol Psychiatry. 2004;45(5):979-995.PubMedGoogle ScholarCrossref
6.
Whitehurst  GJ, Lonigan  CJ.  Child development and emergent literacy.  Child Dev. 1998;69(3):848-872.PubMedGoogle ScholarCrossref
7.
Shaywitz  SE, Shaywitz  BA, Fletcher  JM, Escobar  MD.  Prevalence of reading disability in boys and girls: results of the Connecticut Longitudinal Study.  JAMA. 1990;264(8):998-1002.PubMedGoogle ScholarCrossref
8.
Lyon  GR, Fletcher  JM, Shaywitz  SE,  et al. Rethinking learning disabilities. In: Finn  CE  Jr, Rotherham  AJ, Hokanson  CR  Jr, eds.  Rethinking Special Education for a New Century. Progressive Policy Institute; 2001:259-287.
9.
Committee on Integrating the Science of Early Childhood Development; Board on Children, Youth, and Families; National Research Council.  From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press; 2000.
10.
Doyle  O, Harmon  CP, Heckman  JJ, Tremblay  RE.  Investing in early human development: timing and economic efficiency.  Econ Hum Biol. 2009;7(1):1-6.PubMedGoogle ScholarCrossref
11.
Gathercole  SE, Alloway  TP.  Practitioner review: short-term and working memory impairments in neurodevelopmental disorders: diagnosis and remedial support.  J Child Psychol Psychiatry. 2006;47(1):4-15.PubMedGoogle ScholarCrossref
12.
Gathercole  SE, Alloway  TP.  Working Memory and Learning: A Practical Guide for Teachers. London, England: Sage; 2008.
13.
Miyake  A, Friedman  NP, Emerson  MJ, Witzki  AH, Howerter  A, Wager  TD.  The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: a latent variable analysis.  Cogn Psychol. 2000;41(1):49-100.PubMedGoogle ScholarCrossref
14.
St Clair-Thompson  HL, Gathercole  SE.  Executive functions and achievements in school: shifting, updating, inhibition, and working memory.  Q J Exp Psychol (Hove). 2006;59(4):745-759.PubMedGoogle ScholarCrossref
15.
Gathercole  SE, Brown  L, Pickering  SJ.  Working memory assessments at school entry as longitudinal predictors of National Curriculum attainment levels.  Educ Child Psychol. 2003;20(3):109-122.Google Scholar
16.
Gathercole  SE, Alloway  TP, Willis  C, Adams  AM.  Working memory in children with reading disabilities.  J Exp Child Psychol. 2006;93(3):265-281.PubMedGoogle ScholarCrossref
17.
Geary  DC, Hoard  MK, Bailey  DH.  Fact retrieval deficits in low achieving children and children with mathematical learning disability.  J Learn Disabil. 2012;45(4):291-307.PubMedGoogle ScholarCrossref
18.
Beck  SJ, Hanson  CA, Puffenberger  SS, Benninger  KL, Benninger  WB.  A controlled trial of working memory training for children and adolescents with ADHD.  J Clin Child Adolesc Psychol. 2010;39(6):825-836.PubMedGoogle ScholarCrossref
19.
Green  CT, Long  DL, Green  D,  et al.  Will working memory training generalize to improve off-task behavior in children with attention-deficit/hyperactivity disorder?  Neurotherapeutics. 2012;9(3):639-648.PubMedGoogle ScholarCrossref
20.
Grunewaldt  KH, Løhaugen  GC, Austeng  D, Brubakk  AM, Skranes  J.  Working memory training improves cognitive function in VLBW preschoolers.  Pediatrics. 2013;131(3):e747-e754.PubMedGoogle ScholarCrossref
21.
Hardy  KK, Willard  VW, Allen  TM, Bonner  MJ.  Working memory training in survivors of pediatric cancer: a randomized pilot study.  Psychooncology. 2013;22(8):1856-1865.PubMedGoogle ScholarCrossref
22.
Holmes  J, Gathercole  SE, Dunning  DL.  Adaptive training leads to sustained enhancement of poor working memory in children.  Dev Sci. 2009;12(4):F9-F15.PubMedGoogle ScholarCrossref
23.
Klingberg  T, Fernell  E, Olesen  PJ,  et al.  Computerized training of working memory in children with ADHD—a randomized, controlled trial.  J Am Acad Child Adolesc Psychiatry. 2005;44(2):177-186.PubMedGoogle ScholarCrossref
24.
Holmes  J, Gathercole  SE.  Taking working memory training from the laboratory into schools.  Educ Psychol (Lond). 2014;34(4):440-450. PubMedGoogle ScholarCrossref
25.
Cortese  S, Ferrin  M, Brandeis  D,  et al; European ADHD Guidelines Group (EAGG).  Cognitive training for attention-deficit/hyperactivity disorder: meta-analysis of clinical and neuropsychological outcomes from randomized controlled trials.  J Am Acad Child Adolesc Psychiatry. 2015;54(3):164-174.PubMedGoogle ScholarCrossref
26.
Peijnenborgh  JCAW, Hurks  PM, Aldenkamp  AP, Vles  JSH, Hendriksen  JGM.  Efficacy of working memory training in children and adolescents with learning disabilities: a review study and meta-analysis [published online April 17, 2015].  Neuropsychol Rehabil. doi:10.1080/09602011.2015.1026356.PubMedGoogle Scholar
27.
Spencer-Smith  M, Klingberg  T.  Benefits of a working memory training program for inattention in daily life: a systematic review and meta-analysis.  PLoS One. 2015;10(3):e0119522.PubMedGoogle ScholarCrossref
28.
Westerberg  H, Klingberg  T.  Changes in cortical activity after training of working memory—a single-subject analysis.  Physiol Behav. 2007;92(1-2):186-192.PubMedGoogle ScholarCrossref
29.
Astle  DE, Barnes  JJ, Baker  K, Colclough  GL, Woolrich  MW.  Cognitive training enhances intrinsic brain connectivity in childhood.  J Neurosci. 2015;35(16):6277-6283.PubMedGoogle ScholarCrossref
30.
Melby-Lervåg  M, Hulme  C.  Is working memory training effective? a meta-analytic review.  Dev Psychol. 2013;49(2):270-291.PubMedGoogle ScholarCrossref
31.
Dunning  DL, Holmes  J, Gathercole  SE.  Does working memory training lead to generalized improvements in children with low working memory? a randomized controlled trial.  Dev Sci. 2013;16(6):915-925.PubMedGoogle Scholar
32.
Schwaighofer  M, Fischer  F, Bühner  M.  Does working memory training transfer? a meta-analysis including training conditions as moderators.  Educ Psychol. 2015;50(2):138-166. doi:10.1080/00461520.2015.1036274.Google ScholarCrossref
33.
Shipstead  Z, Redick  TS, Engle  RW.  Is working memory training effective?  Psychol Bull. 2012;138(4):628-654.PubMedGoogle ScholarCrossref
34.
Roberts  G, Quach  J, Gold  L,  et al.  Can improving working memory prevent academic difficulties? a school based randomised controlled trial.  BMC Pediatr. 2011;11:57.PubMedGoogle ScholarCrossref
35.
Roberts  G, Quach  J, Mensah  F,  et al.  Schooling duration rather than chronological age predicts working memory between 6 and 7 years: Memory Maestros Study.  J Dev Behav Pediatr. 2015;36(2):68-74.PubMedGoogle ScholarCrossref
36.
Alloway  T, Gathercole  S, Pickering  S.  Automated Working Memory Assessment. London, England: The Psychological Corporation; 2007.
37.
Cogmed Coaching Manual: Cogmed Working Memory Training [computer program]. Version 1.0.7. Sydney, Australia: Pearson Australia; 2009.
38.
Wilkinson  GS, Robertson  GJ.  Wide Range Achievement Test 4 Professional Manual. Lutz, FL: Psychological Assessment Resources; 2006.
39.
von Bastian  CC, Oberauer  K.  Effects and mechanisms of working memory training: a review.  Psychol Res. 2014;78(6):803-820.PubMedGoogle ScholarCrossref
40.
Sterne  JA, White  IR, Carlin  JB,  et al.  Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls.  BMJ. 2009;338:b2393.PubMedGoogle ScholarCrossref
41.
6227.0.55.003—Education and work, Australia—additional data cubes, May 2013. Australian Bureau of Statistics website. http://www.abs.gov.au/ausstats/abs@.nsf/mf/6227.0.55.003. Released November 29, 2013. Accessed January 29, 2016.
42.
Wake  M, Tobin  S, Girolametto  L,  et al.  Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let’s Learn Language cluster randomised controlled trial.  BMJ. 2011;343:d4741.PubMedGoogle ScholarCrossref
43.
Dahlin  KIE.  Working memory training and the effect on mathematical achievement in children with attention deficits and special needs.  J Educ Learning. 2013;2(1):118. doi:10.5539/jel.v2n1p118.Google ScholarCrossref
44.
Holmes  J, Gathercole  SE, Place  M, Dunning  DL, Hilton  KA, Elliott  JG.  Working memory deficits can be overcome: impacts of training and medication on working memory in children with ADHD.  Appl Cogn Psychol. 2010;24(6):827-836. doi:10.1002/acp.1589.Google ScholarCrossref
45.
Mezzacappa  E, Buckner  JC.  Working memory training for children with attention problems or hyperactivity: a school-based pilot study.  School Ment Health. 2010;2(4):202-208. doi:10.1007/s12310-010-9030-9.Google ScholarCrossref
46.
Gray  SA, Chaban  P, Martinussen  R,  et al.  Effects of a computerized working memory training program on working memory, attention, and academics in adolescents with severe LD and comorbid ADHD: a randomized controlled trial.  J Child Psychol Psychiatry. 2012;53(12):1277-1284.PubMedGoogle ScholarCrossref
47.
Egeland  J, Aarlien  AK, Saunes  BK.  Few effects of far transfer of working memory training in ADHD: a randomized controlled trial.  PLoS One. 2013;8(10):e75660.PubMedGoogle ScholarCrossref
48.
Klingberg  T.  Training and plasticity of working memory.  Trends Cogn Sci. 2010;14(7):317-324.PubMedGoogle ScholarCrossref
Original Investigation
May 2, 2016

Academic Outcomes 2 Years After Working Memory Training for Children With Low Working MemoryA Randomized Clinical Trial

Author Affiliations
  • 1Population Health, Murdoch Childrens Research Institute, Centre for Community Child Health, Royal Children’s Hospital, Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia
  • 2Population Health, Murdoch Childrens Research Institute, Graduate School of Education, University of Melbourne, Melbourne, Victoria, Australia
  • 3School of Psychological Sciences, Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Melbourne, Victoria, Australia
  • 4Clinical Sciences, Murdoch Childrens Research Institute, Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia
  • 5Medical Research Council Cognition and Brain Sciences Unit, Cambridge, England
  • 6Deakin Health Economics, Deakin University, Geelong, Victoria, Australia
  • 7Data Science, Murdoch Childrens Research Institute, Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia
  • 8Melbourne Graduate School of Education, University of Melbourne, Melbourne, Victoria, Australia
  • 9Australian Council for Educational Research, Camberwell, Victoria, Australia
 

Copyright 2016 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.

JAMA Pediatr. 2016;170(5):e154568. doi:10.1001/jamapediatrics.2015.4568
Abstract

Importance  Working memory training may help children with attention and learning difficulties, but robust evidence from population-level randomized controlled clinical trials is lacking.

Objective  To test whether a computerized adaptive working memory intervention program improves long-term academic outcomes of children 6 to 7 years of age with low working memory compared with usual classroom teaching.

Design, Setting, and Participants  Population-based randomized controlled clinical trial of first graders from 44 schools in Melbourne, Australia, who underwent a verbal and visuospatial working memory screening. Children were classified as having low working memory if their scores were below the 15th percentile on either the Backward Digit Recall or Mister X subtest from the Automated Working Memory Assessment, or if their scores were below the 25th percentile on both. These children were randomly assigned by an independent statistician to either an intervention or a control arm using a concealed computerized random number sequence. Researchers were blinded to group assignment at time of screening. We conducted our trial from March 1, 2012, to February 1, 2015; our final analysis was on October 30, 2015. We used intention-to-treat analyses.

Intervention  Cogmed working memory training, comprising 20 to 25 training sessions of 45 minutes’ duration at school.

Main Outcomes and Measures  Directly assessed (at 12 and 24 months) academic outcomes (reading, math, and spelling scores as primary outcomes) and working memory (also assessed at 6 months); parent-, teacher-, and child-reported behavioral and social-emotional functioning and quality of life; and intervention costs.

Results  Of 1723 children screened (mean [SD] age, 6.9 [0.4] years), 226 were randomized to each arm (452 total), with 90% retention at 1 year and 88% retention at 2 years; 90.3% of children in the intervention arm completed at least 20 sessions. Of the 4 short-term and working memory outcomes, 1 outcome (visuospatial short-term memory) benefited the children at 6 months (effect size, 0.43 [95% CI, 0.25-0.62]) and 12 months (effect size, 0.49 [95% CI, 0.28-0.70]), but not at 24 months. There were no benefits to any other outcomes; in fact, the math scores of the children in the intervention arm were worse at 2 years (mean difference, −3.0 [95% CI, −5.4 to −0.7]; P = .01). Intervention costs were A$1035 per child.

Conclusions and Relevance  Working memory screening of children 6 to 7 years of age is feasible, and an adaptive working memory training program may temporarily improve visuospatial short-term memory. Given the loss of classroom time, cost, and lack of lasting benefit, we cannot recommend population-based delivery of Cogmed within a screening paradigm.

Trial Registration  anzctr.org.au Identifier: ACTRN12610000486022

Introduction

Low academic achievement is a major public health issue because it is prevalent,1,2 alters the life opportunities of individuals via poorer mental and physical health and financial hardship,3-6 and threatens the economic and societal functioning of nations. By the time it becomes evident, low academic achievement is often entrenched, and it may be too late to intervene.1,7,8 Preventing low academic achievement is a public health priority,9,10 but solutions remain elusive.

Novel interventions that can be widely applied can be attractive, even in the absence of evidence of long-term benefits and cost-effectiveness. One such approach is “brain training” to improve working memory, a cognitive function responsible for temporarily storing and manipulating information needed to support learning.11-13 Children with low working memory often fail classroom activities12 and are at high risk of low academic achievement.14,15 For example, more than 90% of children 6 to 7 years of age with reading difficulties have low working memory,16 and children with mathematical difficulties are more likely than their peers to have low working memory,17 and this association persists after adjusting for IQ.11 Children at educational risk can be identified at school entry on the basis of low working memory scores.15

Commercially available computerized programs to improve working memory or cognition are widely implemented, despite a lack of supporting evidence. The cognitive training program “Elevate” was ranked by Apple as the best iPhone app in 2014, and the cognitive training sector has been forecast to profit by more than $500 million internationally in 2015. Administered to tens of thousands of clients each year, the Cogmed Working Memory Training program (Pearson) is the most widely used and evaluated working memory training program. Randomized clinical trials have shown the benefits to working memory for children with attention-deficit/hyperactivity disorder18,19 and for other clinical and nonclinical populations.20-22 The benefits of training may transfer to other cognitive domains,23 academic functioning,22,24 and behavior.25-27 Cognitive training may induce neuroplasticity, such as increased activation in the frontal and parietal areas of the brain,28 and increased connectivity of key components of the attentional control network at rest.29

While there is evidence for short-term working memory enhancement following Cogmed,25,30 there are no rigorous randomized clinical trials with large samples and long-term follow-up, nor is there any consistent evidence of long-term transfer effects on educational attainment.26,30-32 Furthermore, the economic and opportunity costs of implementation of a working memory training program as a population-level prevention strategy are unknown, calling the effectiveness of working memory training programs into question.33

To our knowledge, we report the first large-scale population-based randomized clinical trial of a working memory intervention for children 6 to 7 years of age screened as having low working memory. We aimed to (1) determine the efficacy of Cogmed, at 12 and 24 months, on reading, spelling, and mathematics (our primary outcome measured at 12 months) and a broad range of secondary outcomes for these children (the intervention arm) compared with children receiving the usual classroom education (the control arm), and (2) evaluate the costs and benefits.

Box Section Ref ID

Key Points

  • Question: Compared with usual classroom teaching, does a computerized adaptive working memory intervention program (Cogmed) improve long-term academic outcomes in children 6 to 7 years of age who were determined to have low working memory after a population screening?

  • Findings: Although there was a temporary benefit to visuospatial short-term memory, our population-based randomized controlled clinical trial showed that there was no improvement in reading, spelling, or mathematics in the intervention group compared with the control group at 12 and 24 months after randomization.

  • Meaning: Given the loss of classroom time, the cost, and the lack of a lasting benefit, we cannot recommend the population-based delivery of Cogmed within a screening paradigm.

Methods
Design and Setting

We have previously reported our trial protocol (Supplement 1).34 The Memory Maestros study35 is a population-based randomized controlled clinical trial comparing a computerized working memory intervention (Cogmed) with usual classroom teaching in Grade 1 students with low working memory(Grade 1 refers to the second year of formal primary school education in Victoria, Australia). Figure 1 shows the CONSORT diagram, and Figure 2 shows a graphical representation of the trial. Approval was obtained from the Human Research Ethics Committee at the Royal Children’s Hospital in Melbourne, Australia, the Victorian Department of Education and Training, and the Catholic Education Office. All parents provided written informed consent.

Grade 1 students from 44 primary schools in metropolitan Melbourne (with a population of 4.1 million in 2012), Australia, participated. Schools were approached according to a random sequence that was generated to recruit a sample representative of each of the 3 Victorian school sectors (government, Catholic, and independent) and from a range of sociodemographic backgrounds.

Eligibility and Recruitment

Recruitment was carried out over all 4 terms of the 2012 school year (from February to December). Screening and intervention occurred in succession within each school. Teachers sent home recruitment packets containing a baseline questionnaire, information statement, and consent form. We simultaneously obtained consent to screen the working memory of the child and, in the event of it being low, consent for the child to enroll in the trial and follow-up.

Research assistants administered a 10-minute computerized working memory screening during school hours to each child whose parents had consented, within 2 weeks of completing the baseline questionnaire. Children were classified as having low working memory if their scores on either the Backward Digit Recall or Mister X subtest from the Automated Working Memory Assessment were below the 15th percentile, or if both scores were below the 25th percentile.36 The cut points were based on internally generated percentile ranks because our sample was much larger than the Automated Working Memory Assessment normative sample. Eligibility cut points were revised midyear, to reflect the developmental progression in working memory over time.35

Children with low working memory were eligible to be randomized. We excluded children with disabilities (eg, cerebral palsy, vision/hearing impairments, or pervasive developmental disorders) that were likely to prevent participation in the intervention program and children and families with insufficient English language abilities that were likely to prevent participation in the consent, assessment, or intervention procedures of the trial.

Randomization and Blinding

An independent statistician randomized children into usual classroom teaching (control arm) or the working memory training program (intervention arm) using a computerized random number sequence. Assignment was at a ratio of 1:1 intervention to control, stratified by school; thus, each school included a balanced number of children participating in the intervention and control arms. Researchers assessing outcomes were blinded to group allocation.

Intervention

Children were taken out of class in groups (up to 4 students) by a research assistant to participate in the Cogmed RM program, comprising 20 to 25 sessions delivered over 5 to 7 weeks. Each child had their own computer and noise-cancelling headphones to limit distractions. Sessions took 45 minutes, on average (range, 35-60 minutes). Children completed 8 of 12 tasks. A new task was introduced every 5 to 6 days. This adaptive program matches difficulty level to the child’s performance on a trial-by-trial basis. Tasks involve the temporary storage and manipulation of verbal and/or visuospatial information in a computer game format, such as recalling a sequence of numbers that light up in a certain order. Motivational features include verbal feedback, the displaying of “high scores,” and the accumulation of “points” when tasks are successfully completed.

Research assistants (“training aides”) were trained through role-playing and field observations before trial commencement. Weekly meetings with the research assistants and project manager (a trained Cogmed “coach”) were held to discuss any issues that may arise (eg, child motivation), and a standard approach was agreed on, as recommended in the Cogmed training manual.37

Outcome Measures

Table 1 summarizes the trial measures. The primary outcomes were the word reading, spelling, and math computation subscales of the Wide Range Achievement Test 4 at 12 months (also measured at 24 months), each with a normative mean (SD) of 100 (15).38 Secondary outcomes measured at 12 and 24 months included working memory performance (also measured at 6 months), behavior, and health-related quality of life.

Intervention Costs

Costs in 2014 Australian dollars (A$) included training of Cogmed coach and training aides, screening, and intervention delivery. Research staff members prospectively recorded time and the materials used. Staff time was valued at staff wage rates (including 20% on-costs), child time was valued at A$0, car travel was valued at standard medium car unit costs, computer hardware and software were valued at the unit costs experienced in the study, and intervention software (provided at zero cost) was valued at A$0.

Statistical Methods

We used intention-to-treat analyses to compare outcomes at 6, 12, and 24 months between the intervention and control arms. Mean outcomes were compared using linear regression in unadjusted analyses and analyses adjusted for factors specified a priori that may have affected outcomes and may not have been fully balanced by randomization: child’s sex, IQ, and primary caregiver’s education. Clustering of children within schools was accounted for using robust regression techniques in which the variance estimates are adjusted to account for the similarity between children within schools.39

Ancillary sensitivity analyses were also conducted. First, for the working memory outcomes that were repeatedly measured at 3 or more time points, we ran random-effects regression analyses to reexamine treatment effects by time within a longitudinal regression model. This was precluded for our principal analysis because, in keeping with the “screen plus intervene” model being tested, most of our key outcomes were not measured at baseline. Second, we reran all analyses using a multiply imputed data set to estimate possible effects of missing data within the intention-to-treat analyses.40 The multiple imputation model included all variables used in the complete data analysis, with a series of 50 data sets imputed using chained equations. All analyses were undertaken using Stata version 14 (StataCorp).

Sample Size

To detect a clinically important difference of 0.3 SD in the primary outcome measures at a significance level of .05 with 80% power, we required 175 children in each of the 2 trial arms (350 in total) at our primary end point. Allowing for 20% attrition, we found that our target sample size at recruitment was 438 children.

Results

Of the 67 schools approached, 44 (65.7%) agreed to participate. Figure 2 shows that, of the 2747 parents approached, 1761 (64.1%) consented, and 1723 of 1761 children (97.8%) completed the screening assessment (ie, the parent to child ratio was 1:1). Table 2 shows that boys and girls were similarly represented. About two-thirds of parents had completed a tertiary education, somewhat higher than the 43% expected from Australian census data.41

Of the 1723 children screened, 452 met the inclusion criteria and were randomized, 226 in each study arm. At 6, 12, and 24 months postrandomization, 90.5%, 89.5%, and 87.8% of children remained in the study. Demographic characteristics between children who did and children who did not participate in outcome assessments were similar at all time points.

Intervention Delivery

Of the 226 children randomly assigned to the intervention arm, 204 (90.3%) completed at least 20 training sessions. The children who competed the training sessions and the children who did not were comparable on baseline screening assessment and characteristics.

Outcomes

Table 3 shows the complete-case outcomes comparisons. At 6 months, children in the intervention arm had higher visuospatial short-term memory (mean difference, 5.47 [95% CI, 2.87-8.07]; P < .001) and verbal working memory scores (mean difference, 2.91 [95% CI, 0.02-5.79]; P = .04) than children in the control arm. Relationships were similar after adjustment. Only the visuospatial short-term memory benefits remained at 12 months, and none were apparent by 24 months.

Despite this transient short-term memory benefit, there was little evidence for improved academic outcomes (Table 3). Children in the intervention arm had poorer word reading (mean difference, −1.81 [95% CI, −3.78 to 0.15]; P = .07) and math computation scores (mean difference, −2.64 [95% CI, −5.48 to 0.20]; P = .07) at 12 months than children in the control arm. The evidence for lower scores in math computation for the intervention group was stronger at 24 months (mean difference, −3.03 [95% CI, −5.39 to −0.67]; P = .01), although the effect size was small (0.2). We note the inflated potential for chance findings due to the number of outcomes and time points. All other academic outcomes were similar at both 12 and 24 months, although a consistent theme was for slightly lower scores in the intervention group. Despite low working memory at baseline, mean word reading and spelling scores of children in both groups were slightly above the mean US normative population value of 100, while the mean math computation score was around a third of a standard deviation below. The rate of completion of the required 20 sessions was so high (>90%) that we did not run dose-response analyses. The parent, child, and teacher ratings spanning attention problems, social-emotional difficulties, and quality of life were similar in the intervention and control groups at 12 and 24 months, with mean scores typical for that age.

Ancillary Analyses

The eTable in Supplement 2 shows the treatment effects reestimated using random-effects regression models and also the principal analysis repeated using the multiply imputed data set, which is also presented graphically in the eFigure in Supplement 2. These reanalyses did not substantively change the outcome values or any conclusions.

Intervention Costs and Cost-Effectiveness

Costs of training (A$12 919), materials (A$15 939), screening (A$54 205), and intervention delivery (A$150 956) summed to A$234 020: A$1035 per child randomly assigned to the intervention arm. If offered to all Victorian Grade 1 children in the lowest quartile for working memory, this would equate to over A$18 million per annum ($1035 × 25% × 75 000 + annual school intake). This does not include the costs of the Cogmed program, which was made available to the trial at no charge but currently retails for around A$1500 per child.

Discussion
Principal Findings

This randomized controlled clinical trial examined the effectiveness of an adaptive working memory intervention in improving population-based academic outcomes for children with low working memory. Despite an advantage to some working memory measures at 6 and 12 months in line with other studies of Cogmed,22,31 there were no evident benefits to academic outcomes at 12 or 24 months. This lack of effect is also seen in the parent and teacher ratings of attention, social-emotional difficulties, and quality of life. This lack of benefit must be considered in light of the costs in terms of price (over A$1000 per child, plus cost of the program itself), the loss of around 15 to 20 hours of usual classroom teaching for each child, and opportunity (other remediation that could have been offered instead).

Strengths and Weaknesses of the Study

Our trial was randomized and controlled, used an intention-to-treat analysis, and is the largest trial of working memory training to date, to our knowledge. Allocation and outcomes assessment were blinded. The study groups were recruited from a large population-based cohort, allowing generalizability. The retention rate was high, maximizing power and minimizing bias. The completion rate for the intervention arm was high, allowing us to evaluate long-term efficacy with confidence; this is supported by the similar conclusions from ancillary analyses using multiply imputed outcomes data sets. We exceeded our required sample size for the directly assessed primary outcome measure but not for the parent- or teacher-reported outcomes. However, complete-case analyses closely resembled those using the multiply imputed outcomes data set, and neither showed any trends toward effectiveness for these secondary outcomes.

A potential limitation of our study is that the screening protocol used only 2 subtests of the Automated Working Memory Assessment. This mimicked what we felt would be possible if this model was to be rolled out on a large scale: screening all children quickly, with rapid progression to intervention. However, this comes at a likely cost in terms of measurement precision. Although our working memory screening tests have been widely used in previous research in this field,22,24 like many standardized cognitive measures, they suffer from task impurity and tap skills additional to working memory (eg, Mister X subtest requires visuoperceptual ability).

The generalizability of these results may not extend to children who do not speak English or to children whose parents have a lower educational level than those in our study. The children in our study were at the lowest end of the recommended age range for the Cogmed RM program. Lack of a nonadaptive control group could have been seen as a limitation had there been an intervention effect, but it is not relevant for a noneffective intervention.

Interpretation in Light of Other Study Findings

Our 6-month working memory effect sizes are smaller than those in previous trials in this field. This is not an uncommon finding when scaling up to a population level with a rigorous design42; previous Cogmed studies varied in participant characteristics (including participants with attention-deficit/hyperactivity disorder, learning difficulties, special education needs, and low working memory) and/or used weaker designs (including nonrandom allocation and lack of a control group). Most of these studies test the intervention on older children. Our findings sit between those of the initial, small school-based studies reporting that Cogmed training can enhance visuospatial22,24,31,43-45 and verbal short-term memory44 and visuospatial22,24,31,43 and verbal working memory,22,24,31,43-45 and an increasing number of studies indicating little benefit for verbal short-term memory.22,31,43,44,46

Our 12-month working memory and academic outcomes are comparable to those of the largest randomized trial, to date, but go further by virtue of our larger sample size, higher retention, and subsequent longer follow-up. In the trial by Dunning et al,31 more than 800 children 7 to 9 years of age from 9 UK schools were screened for low working memory; 34 participated in Cogmed, 30 in a nonadaptive version, and 30 received teaching as usual. At 12 months, moderate-to-large gains in verbal working memory were sustained in the Cogmed group (n = 15) compared with the nonadaptive group (n = 19), but there was little evidence of improved academic performance. Smaller, methodologically less rigorous studies have reported some academic benefits. Egeland et al47 reported a slight increase in reading scores at 8 months in a small randomized controlled trial (N = 57) of children with attention-deficit/hyperactivity disorder, and Holmes et al22,43 and Dahlin22,43 reported an increase in mathematics scores at 6 and 7 months, respectively, in small (N = 42 and N = 57) nonrandomized trials.

A controversial issue33,39 is whether targeting one specific cognitive skill (eg, working memory) can improve another (eg, academic function). Klingberg48 argues that working memory training can result in changes in activity (“plasticity”) in neural networks that may then result in a transfer of effects to nontrained tasks. However, a primary school classroom offers a complex interplay between the temperaments of students and teachers, cognitive abilities, and engagement, which may not correspond with the highly structured and predictable computer-based training environment. Future interventions may benefit from extending programs such as Cogmed to include real-world activities and incorporate explicit strategy training paired with education intervention.

Implications for Clinicians and Policy Makers

With no evidence of benefit in our primary outcomes, the intervention is not cost-effective. Our intervention delivery costs were inflated by high travel costs, as city-center–based training aides traveled to schools. In practice, individual schools may deliver the intervention using local staff. However, any reduction in travel costs would be balanced against the cost of the intervention program, which was waived for this trial. Given the consistently, slightly lower academic scores in the intervention group, particularly for word reading and math computation, our results raise questions regarding the potential for harm by taking children out of class on a regular basis for several weeks to provide an intervention such as this. On balance, we cannot recommend this intervention as a population-level selective prevention strategy.

Conclusions

It is feasible to implement population-based working memory screening for children 6 to 7 years of age and deliver a working memory training program (Cogmed). Although this benefitted some elements of memory at 6 months, given the high cost and the lack of benefit to academic outcomes or any other outcomes 12 or 24 months after randomization, we cannot recommend its population-based delivery as a selective prevention program. Longer-term follow-up of the trial’s cohort will clarify any lasting effects, whether harmful or beneficial. In the meantime, we recommend that equally rigorous trials test the other indications currently targeted by adaptive computerized training programs.

Back to top
Article Information

Accepted for Publication: December 4, 2015.

Corresponding Author: Gehan Roberts, MPH, PhD, Centre for Community Child Health, Royal Children’s Hospital, Flemington Road, Parkville VIC 3052, Australia (gehan.roberts@rch.edu.au).

Published Online: March 7, 2016. doi:10.1001/jamapediatrics.2015.4568.

Author Contributions: Drs Roberts and Quach had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Roberts, Anderson, Gathercole, Gold, Mensah, Rickards, Ainley, Wake.

Acquisition, analysis, or interpretation of data: Roberts, Quach, Spencer-Smith, Anderson, Gathercole, Gold, Sia, Mensah, Wake.

Drafting of the manuscript: Roberts, Quach, Spencer-Smith, Gathercole, Wake.

Critical revision of the manuscript for important intellectual content: Roberts, Spencer-Smith, Anderson, Gathercole, Gold, Sia, Mensah, Rickards, Ainley, Wake.

Statistical analysis: Quach, Anderson, Mensah.

Obtained funding: Roberts, Gathercole, Gold, Mensah, Wake.

Administrative, technical, or material support: Roberts, Quach, Gold, Sia, Wake.

Study supervision: Roberts, Spencer-Smith, Anderson, Gold, Rickards, Wake.

Conflict of Interest Disclosures: None reported.

Funding/Support: The trial is funded by the National Health Medical Research Council in Australia, as follows: Project Grant 1005317; Early Career Fellowship 607384 (Dr Roberts); Senior Research Fellowship 1046518 (Dr Wake); Capacity Building Grant 425855 and Early Career Fellowship 1035100 (Dr Gold); Senior Research Fellowship 1081288 (Dr Anderson); and Capacity Building Grant 436914 and Early Career Fellowship 1037449 (Dr Mensah). Dr Quach is funded by an Australian Research Council Discovery Early Career Award DE140100751. The project also received support through the Centre for Research Excellence in Child Language, which is funded by the National Health Medical Research Council in Australia and based at the Murdoch Childrens Research Institute, which is supported by the Victorian Government’s Operational Infrastructure Program.

Role of the Funder/Sponsor: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank Bibi Gerner, DEdPsych, at Population Health, Murdoch Childrens Research Institute, for assisting Dr Quach, who was a postdoctoral research fellow and the project manager. She received no compensation outside of her regular manager salary.

References
1.
Prior  M, Sanson  A, Smart  D, Oberklaid  F.  Reading disability in an Australian community sample.  Aust J Psychol. 1995;47(1):32-37. doi:10.1080/00049539508258766.Google ScholarCrossref
2.
4226.0—Aspects of literacy: profiles and perceptions, Australia, 1996. Australian Bureau of Statistics website. http://www.abs.gov.au/ausstats/abs@.nsf/mf/4226.0. Released April 30, 1997. Accessed January 29, 2016.
3.
Westwood  P.  What Teachers Need to Know About Learning Difficulties. Camberwell, Victoria, Australia: ACER Press; 2008.
4.
Gorman  JC.  Emotional Disorders and Learning Disabilities in the Elementary Classroom: Interactions and Interventions. Thousand Oaks, CA: Corwin Press; 2001.
5.
Dyck  MJ, Hay  D, Anderson  M, Smith  LM, Piek  J, Hallmayer  J.  Is the discrepancy criterion for defining developmental disorders valid?  J Child Psychol Psychiatry. 2004;45(5):979-995.PubMedGoogle ScholarCrossref
6.
Whitehurst  GJ, Lonigan  CJ.  Child development and emergent literacy.  Child Dev. 1998;69(3):848-872.PubMedGoogle ScholarCrossref
7.
Shaywitz  SE, Shaywitz  BA, Fletcher  JM, Escobar  MD.  Prevalence of reading disability in boys and girls: results of the Connecticut Longitudinal Study.  JAMA. 1990;264(8):998-1002.PubMedGoogle ScholarCrossref
8.
Lyon  GR, Fletcher  JM, Shaywitz  SE,  et al. Rethinking learning disabilities. In: Finn  CE  Jr, Rotherham  AJ, Hokanson  CR  Jr, eds.  Rethinking Special Education for a New Century. Progressive Policy Institute; 2001:259-287.
9.
Committee on Integrating the Science of Early Childhood Development; Board on Children, Youth, and Families; National Research Council.  From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press; 2000.
10.
Doyle  O, Harmon  CP, Heckman  JJ, Tremblay  RE.  Investing in early human development: timing and economic efficiency.  Econ Hum Biol. 2009;7(1):1-6.PubMedGoogle ScholarCrossref
11.
Gathercole  SE, Alloway  TP.  Practitioner review: short-term and working memory impairments in neurodevelopmental disorders: diagnosis and remedial support.  J Child Psychol Psychiatry. 2006;47(1):4-15.PubMedGoogle ScholarCrossref
12.
Gathercole  SE, Alloway  TP.  Working Memory and Learning: A Practical Guide for Teachers. London, England: Sage; 2008.
13.
Miyake  A, Friedman  NP, Emerson  MJ, Witzki  AH, Howerter  A, Wager  TD.  The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: a latent variable analysis.  Cogn Psychol. 2000;41(1):49-100.PubMedGoogle ScholarCrossref
14.
St Clair-Thompson  HL, Gathercole  SE.  Executive functions and achievements in school: shifting, updating, inhibition, and working memory.  Q J Exp Psychol (Hove). 2006;59(4):745-759.PubMedGoogle ScholarCrossref
15.
Gathercole  SE, Brown  L, Pickering  SJ.  Working memory assessments at school entry as longitudinal predictors of National Curriculum attainment levels.  Educ Child Psychol. 2003;20(3):109-122.Google Scholar
16.
Gathercole  SE, Alloway  TP, Willis  C, Adams  AM.  Working memory in children with reading disabilities.  J Exp Child Psychol. 2006;93(3):265-281.PubMedGoogle ScholarCrossref
17.
Geary  DC, Hoard  MK, Bailey  DH.  Fact retrieval deficits in low achieving children and children with mathematical learning disability.  J Learn Disabil. 2012;45(4):291-307.PubMedGoogle ScholarCrossref
18.
Beck  SJ, Hanson  CA, Puffenberger  SS, Benninger  KL, Benninger  WB.  A controlled trial of working memory training for children and adolescents with ADHD.  J Clin Child Adolesc Psychol. 2010;39(6):825-836.PubMedGoogle ScholarCrossref
19.
Green  CT, Long  DL, Green  D,  et al.  Will working memory training generalize to improve off-task behavior in children with attention-deficit/hyperactivity disorder?  Neurotherapeutics. 2012;9(3):639-648.PubMedGoogle ScholarCrossref
20.
Grunewaldt  KH, Løhaugen  GC, Austeng  D, Brubakk  AM, Skranes  J.  Working memory training improves cognitive function in VLBW preschoolers.  Pediatrics. 2013;131(3):e747-e754.PubMedGoogle ScholarCrossref
21.
Hardy  KK, Willard  VW, Allen  TM, Bonner  MJ.  Working memory training in survivors of pediatric cancer: a randomized pilot study.  Psychooncology. 2013;22(8):1856-1865.PubMedGoogle ScholarCrossref
22.
Holmes  J, Gathercole  SE, Dunning  DL.  Adaptive training leads to sustained enhancement of poor working memory in children.  Dev Sci. 2009;12(4):F9-F15.PubMedGoogle ScholarCrossref
23.
Klingberg  T, Fernell  E, Olesen  PJ,  et al.  Computerized training of working memory in children with ADHD—a randomized, controlled trial.  J Am Acad Child Adolesc Psychiatry. 2005;44(2):177-186.PubMedGoogle ScholarCrossref
24.
Holmes  J, Gathercole  SE.  Taking working memory training from the laboratory into schools.  Educ Psychol (Lond). 2014;34(4):440-450. PubMedGoogle ScholarCrossref
25.
Cortese  S, Ferrin  M, Brandeis  D,  et al; European ADHD Guidelines Group (EAGG).  Cognitive training for attention-deficit/hyperactivity disorder: meta-analysis of clinical and neuropsychological outcomes from randomized controlled trials.  J Am Acad Child Adolesc Psychiatry. 2015;54(3):164-174.PubMedGoogle ScholarCrossref
26.
Peijnenborgh  JCAW, Hurks  PM, Aldenkamp  AP, Vles  JSH, Hendriksen  JGM.  Efficacy of working memory training in children and adolescents with learning disabilities: a review study and meta-analysis [published online April 17, 2015].  Neuropsychol Rehabil. doi:10.1080/09602011.2015.1026356.PubMedGoogle Scholar
27.
Spencer-Smith  M, Klingberg  T.  Benefits of a working memory training program for inattention in daily life: a systematic review and meta-analysis.  PLoS One. 2015;10(3):e0119522.PubMedGoogle ScholarCrossref
28.
Westerberg  H, Klingberg  T.  Changes in cortical activity after training of working memory—a single-subject analysis.  Physiol Behav. 2007;92(1-2):186-192.PubMedGoogle ScholarCrossref
29.
Astle  DE, Barnes  JJ, Baker  K, Colclough  GL, Woolrich  MW.  Cognitive training enhances intrinsic brain connectivity in childhood.  J Neurosci. 2015;35(16):6277-6283.PubMedGoogle ScholarCrossref
30.
Melby-Lervåg  M, Hulme  C.  Is working memory training effective? a meta-analytic review.  Dev Psychol. 2013;49(2):270-291.PubMedGoogle ScholarCrossref
31.
Dunning  DL, Holmes  J, Gathercole  SE.  Does working memory training lead to generalized improvements in children with low working memory? a randomized controlled trial.  Dev Sci. 2013;16(6):915-925.PubMedGoogle Scholar
32.
Schwaighofer  M, Fischer  F, Bühner  M.  Does working memory training transfer? a meta-analysis including training conditions as moderators.  Educ Psychol. 2015;50(2):138-166. doi:10.1080/00461520.2015.1036274.Google ScholarCrossref
33.
Shipstead  Z, Redick  TS, Engle  RW.  Is working memory training effective?  Psychol Bull. 2012;138(4):628-654.PubMedGoogle ScholarCrossref
34.
Roberts  G, Quach  J, Gold  L,  et al.  Can improving working memory prevent academic difficulties? a school based randomised controlled trial.  BMC Pediatr. 2011;11:57.PubMedGoogle ScholarCrossref
35.
Roberts  G, Quach  J, Mensah  F,  et al.  Schooling duration rather than chronological age predicts working memory between 6 and 7 years: Memory Maestros Study.  J Dev Behav Pediatr. 2015;36(2):68-74.PubMedGoogle ScholarCrossref
36.
Alloway  T, Gathercole  S, Pickering  S.  Automated Working Memory Assessment. London, England: The Psychological Corporation; 2007.
37.
Cogmed Coaching Manual: Cogmed Working Memory Training [computer program]. Version 1.0.7. Sydney, Australia: Pearson Australia; 2009.
38.
Wilkinson  GS, Robertson  GJ.  Wide Range Achievement Test 4 Professional Manual. Lutz, FL: Psychological Assessment Resources; 2006.
39.
von Bastian  CC, Oberauer  K.  Effects and mechanisms of working memory training: a review.  Psychol Res. 2014;78(6):803-820.PubMedGoogle ScholarCrossref
40.
Sterne  JA, White  IR, Carlin  JB,  et al.  Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls.  BMJ. 2009;338:b2393.PubMedGoogle ScholarCrossref
41.
6227.0.55.003—Education and work, Australia—additional data cubes, May 2013. Australian Bureau of Statistics website. http://www.abs.gov.au/ausstats/abs@.nsf/mf/6227.0.55.003. Released November 29, 2013. Accessed January 29, 2016.
42.
Wake  M, Tobin  S, Girolametto  L,  et al.  Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let’s Learn Language cluster randomised controlled trial.  BMJ. 2011;343:d4741.PubMedGoogle ScholarCrossref
43.
Dahlin  KIE.  Working memory training and the effect on mathematical achievement in children with attention deficits and special needs.  J Educ Learning. 2013;2(1):118. doi:10.5539/jel.v2n1p118.Google ScholarCrossref
44.
Holmes  J, Gathercole  SE, Place  M, Dunning  DL, Hilton  KA, Elliott  JG.  Working memory deficits can be overcome: impacts of training and medication on working memory in children with ADHD.  Appl Cogn Psychol. 2010;24(6):827-836. doi:10.1002/acp.1589.Google ScholarCrossref
45.
Mezzacappa  E, Buckner  JC.  Working memory training for children with attention problems or hyperactivity: a school-based pilot study.  School Ment Health. 2010;2(4):202-208. doi:10.1007/s12310-010-9030-9.Google ScholarCrossref
46.
Gray  SA, Chaban  P, Martinussen  R,  et al.  Effects of a computerized working memory training program on working memory, attention, and academics in adolescents with severe LD and comorbid ADHD: a randomized controlled trial.  J Child Psychol Psychiatry. 2012;53(12):1277-1284.PubMedGoogle ScholarCrossref
47.
Egeland  J, Aarlien  AK, Saunes  BK.  Few effects of far transfer of working memory training in ADHD: a randomized controlled trial.  PLoS One. 2013;8(10):e75660.PubMedGoogle ScholarCrossref
48.
Klingberg  T.  Training and plasticity of working memory.  Trends Cogn Sci. 2010;14(7):317-324.PubMedGoogle ScholarCrossref
×