Figure 1. Study search flowchart.
Figure 2. Number of pediatric stroke studies using validated outcome measures (A) and outcome measures used most over time (B). WAIS indicates Wechsler Adult Intelligence Scale; WISC, Wechsler Intelligence Scale for Children; and WPPSI, Wechsler Preschool and Primary Scale of Intelligence.
Engelmann KA, Jordan LC. Outcome Measures Used in Pediatric Stroke StudiesA Systematic Review. Arch Neurol. 2012;69(1):23-27. doi:10.1001/archneurol.2011.1015
Author Affiliations: Department of Neurology, Division of Pediatric Neurology, Johns Hopkins University School of Medicine, Baltimore, Maryland. Dr Jordan is now with the Department of Neurology, Division of Pediatric Neurology and Stroke, Vanderbilt University Medical Center, Nashville, Tennessee.
Because no gold-standard outcome measure or measures exist to allow comparison of pediatric stroke study outcomes in clinical trials, we designed a systematic review of the literature to survey the current use of pediatric stroke outcome measures. Studies that used at least 1 standardized measure to assess the outcome of children with ischemic or hemorrhagic stroke, from full-term newborn to age 18 years, were included. Although 34 studies were included, an additional 36 studies could not be included because ad hoc, author-generated outcome measures were used. Excluding those measures in neuropsychological batteries, 38 unique outcome measures were used. The Wechsler Intelligence Scales, Pediatric Stroke Outcome Measure, and Bayley Scales of Infant Development were among the most used, but 79% of outcome measures were used by no more than 2 studies. Although many measures used have been validated for use in children with other medical conditions or for adults with stroke, only 1 measure has been specifically validated for use in pediatric ischemic stroke. To maximize comparability of future clinical trial results, agreement regarding a preferred pediatric stroke outcome scale or battery of measures is paramount; these measures should be reliable, responsive to change, and specifically validated for use in children with stroke.
Although pediatric stroke occurs in about 2 or 3 per 100 000 children, treatment is still largely based on low levels of evidence.1,2 Three sets of pediatric stroke guidelines exist, but there are no clinical trials to inform treatment outside of sickle cell disease.3- 5 More clinical trials aiming to improve pediatric stroke treatment are on the horizon, yet no gold-standard outcome measure is available to assess and compare the resulting outcomes.
Several institutions have recently highlighted the importance of validated, reliable outcome measures for patient-oriented research. The National Institutes of Health have begun investing in initiatives such as the Patient-Reported Outcomes Measurement Information System (PROMIS), which aims to develop tools to reliably and validly measure patient-reported outcomes in adults.6,7 Similarly, the goal of the common data element project at the National Institute of Neurological Disorders and Stroke is to standardize the collection of investigational data to facilitate comparison of results across studies and more effectively aggregate information into significant metadata sets.8
The aim of this systematic review is to assess the standardized outcome measures currently used in pediatric stroke studies, which will serve as a foundation for understanding the appropriate measures for clinical trials in this population.
Eligible studies included children from birth to age 18 years with ischemic stroke, hemorrhagic stroke, or both, had more than 5 subjects, and evaluated children for neurological or functional outcome status with a recognized outcome measure. Studies were excluded if the subjects were solely preterm infants, had purely intraventricular hemorrhage, or experienced stroke exclusively due to trauma. Studies with mixed study populations (eg, term and preterm infants with stroke) were included only if data could be separated. Studies using ad hoc or unrecognized outcome measures were excluded, as were review, non-English, nonhuman, and abstract-only studies.
Electronic searches of CINAHL, EMBASE, PubMed, and Web of Science were performed in August 2010 using a combination of all relevant PubMed medical subject heading terms and keywords relating to children, stroke, and outcome measures. Identified studies were imported into a reference manager, and duplicates of identical studies were removed.
Two raters agreed on and independently applied inclusion and exclusion criteria to the studies in title, abstract, and full-text reviews. The reference lists of all included articles and of appropriate review articles were examined to identify additional relevant studies.
Study quality was assessed insofar as each study was screened for clear reporting of methods and data. Information on validity, reliability, and general and psychometric data on standardized outcome measures was retrieved from both the outcome measures' sources and original studies.
Frequency of outcome measure use as well as information on validity and reliability were compiled. This study was designated exempt by the institutional review board.
The initial search returned 2996 unique studies, of which 30 were suitable for inclusion. Reasons for exclusion are detailed in Figure 1. Hand-searching found an additional 4 studies, resulting in 34 included studies. Of note, 36 studies were excluded because ad hoc, descriptive outcome measures were used rather than standardized measures. For example, many studies defined outcome solely by reporting neurological sequelae (eg, hemiparesis, epilepsy, cognitive impairment, motor deficits), while others used subjective stratifications such as mild, moderate, or severe deficits.
A detailed description of each included study with aim, sample size and characteristics, and outcome measures is shown in supplemental Table 1 (http://kc.vanderbilt.edu/kennedy_pdfs/JordanLori_supp.pdf). Of the 34 studies, 19 were focused on ischemic stroke only, 5 were focused on hemorrhagic stroke only, and 10 included both types of stroke. Infants were exclusively the subjects of 8 studies, 8 studies included children older than 1 year only, and 18 included both age groups. A median of 2 outcome measures were used per study (range, 1-7 outcome measures). More than 1 outcome measure was used in 29 studies (85%).
The most commonly applied outcome measure was the age-appropriate form of the Wechsler Intelligence Scales, used in 34% of studies. The second most prevalent outcome measure was the Pediatric Stroke Outcome Measure (PSOM), used in 7 studies (21%); more prevalence details are provided in the Table. Notably, 24 of 38 outcome measures were used in 1 included study each (63%).
Descriptions and psychometric properties of outcome measures used more than once are detailed in the Table, with more detailed information and additional outcome measures provided in supplemental Table 2. A standardized neurological examination, the PSOM, has been validated for infants and children with ischemic stroke. Of the 12 most used outcome measures, 9 (75%) have been validated in children. Interrater reliability data were variable both for specific outcome measures (eg, the Glasgow Outcome Scale, ranging from 0.31-0.79 depending on the study) and across all outcome measures. Most tools are pediatric measures of cognitive ability (California Verbal Learning Test–Children's Version, Griffiths Scales of Mental Development, Stanford-Binet Intelligence Scale, Wechsler Intelligence Scales), development (Bayley Scales of Infant Development, Denver Developmental Screening Tests), or overall health (Child Health Questionnaire, Glasgow Outcome Scale, modified Rankin Scale, 36-Item Short Form Health Survey, Vineland Adaptive Behavior Scales).
All included studies using standardized outcome measures were conducted within the previous 2 decades. Figure 2A shows graphically that pediatric stroke studies using recognized outcome measures are increasingly prevalent in the literature. The temporal application of the most used outcome measures, those used 3 or more times, is shown in Figure 2B, which demonstrates the increased use of a variety of outcome measures over time. Notably, use of the validated PSOM has increased.
Currently, there are wide variations in the application of pediatric stroke outcome measures. At this time, the PSOM, a standardized neurological examination, is the best validated outcome measure with direct validation in children aged 0 to 18 years with ischemic stroke. Yet, in pediatric stroke there are many potential domains to assess, including but not limited to adaptive functioning, cognition, emotional health, behavior, and quality of life. Preferably, overall health, cognitive development, and physical development would be assessed both objectively and subjectively, from the patient's and/or caregiver's perspective.
Agreement among researchers, clinicians, and perhaps patients and their families regarding key outcome domains to measure is necessary prior to undertaking further validation and reliability studies for pediatric stroke outcome tools. We can learn much from other fields in this regard. For example, researchers in neuromuscular disease have pushed for outcome measures that are not only valid and reliable but also responsive to improvement or loss of function so as to capture clinically relevant change over time.33 Advanced statistical methods have also been performed to better use data from clinical trials in neurorehabilitation and multiple sclerosis.34
To chart the way forward in pediatric stroke, collaboration among pediatric stroke professionals, clinical trialists, and experts in statistics and clinimetrics is needed. There are many competing issues; for the purposes of clinical trials, investigators would prefer a single composite measure of global outcome such as the modified Rankin Scale used in adult stroke that does not require scoring by a physician or psychologist.35 A battery of measures is more costly and complex but in theory would better capture function and disability after stroke in children. Finally, patient-reported outcomes have become increasingly important. The PROMIS uses modern psychometric methods, including item response theory, to construct question banks that may be used to create computerized adaptive tests to measure outcomes more efficiently and precisely. Computerized adaptive tests use an algorithm whereby only the most informative items targeting an individual's functioning levels are selected, thus reducing the burden of traditional fixed-length questionnaires that may force patients to answer irrelevant items. Item response theory–based scales have interval-level (linear) scaling for better interpretation of variation, calibration of items across a broad range to overcome floor and ceiling effects, and increased precision to allow more sensitivity to change.36 In traditional ordinal scale–based outcome measures such as the modified Rankin Scale, a 1-point change from 5 (severe disability) to 4 (moderately severe disability) is not the same distance as a 1-point change from slight to no disability, making change over time more difficult to interpret.33,34
While the PROMIS has significant advantages, it is important to remember that in young children, patient-reported outcomes are measured via parental responses or proxy. In studies of the most widely used pediatric quality-of-life measure, the Pediatric Quality of Life Inventory (supplemental Table 2), only moderate correlation was found between self- and proxy-report in older children; parents consistently underestimated their child's health-related quality of life, perhaps due to anxiety. Better correlation was found in children with chronic health conditions (correlations ranging from 0.5-0.61) and for physical rather than psychological and social proxy-reports.37,38 Given these concerns, patient-reported outcomes should not be the only outcomes in children. Particularly in young children who are difficult to assess, developmental measures may still be needed.
The creation of outcome assessment guidelines will facilitate appropriate outcome measure selection for future studies as well as communication and comparison of treatment results. If a gold-standard pediatric stroke outcome assessment is not established, the comparability of pediatric stroke trial results will be undermined, potentially delaying the effective treatment of pediatric patients with stroke for years to come.
Correspondence: Lori C. Jordan, MD, PhD, Department of Neurology, Division of Pediatric Neurology and Stroke, Vanderbilt University Medical Center, 2200 Children's Way, DOT 11242, Nashville, TN 37232 (firstname.lastname@example.org).
Accepted for Publication: May 18, 2011.
Author Contributions: Mr Engelmann and Dr Jordan had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Engelmann and Jordan. Acquisition of data: Engelmann and Jordan. Analysis and interpretation of data: Engelmann and Jordan. Drafting of the manuscript: Engelmann and Jordan. Critical revision of the manuscript for important intellectual content: Engelmann and Jordan. Statistical analysis: Engelmann and Jordan. Administrative, technical, and material support: Engelmann. Study supervision: Jordan.
Financial Disclosure: Dr Jordan has served as a consultant for Berlin Heart.
Funding/Support: Dr Jordan is supported by grant K23NS062110 from the National Institute of Neurological Disorders and Stroke.
Role of the Sponsor: The funding agency had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.
Additional Contributions: Victoria H. Goode, MLIS, assisted with the design and conduct of the study.