Figure 1. Patient flowchart.
Figure 2. Proportion of patients who had died within 30 days of admission determined by PLAN (preadmission comorbidities, level of consciousness, age, and neurologic deficit) score categories in the derivation and validation cohorts.
Figure 3. Proportion of patients who had died within 1 year of admission by PLAN (preadmission comorbidities, level of consciousness, age, and neurologic deficit) score categories in the derivation and validation cohorts.
Figure 4. Proportion of patients who had died (in the hospital) or were severely dependent (modified Rankin score, 5-6) at discharge by PLAN (preadmission comorbidities, level of consciousness, age, and neurologic deficit) score categories in the derivation and validation cohorts.
Figure 5. Proportion of patients with modified Rankin (m-R) categories at discharge by PLAN (preadmission comorbidities, level of consciousness, age, and neurologic deficit) score categories in the entire cohort. Modified Rankin score: 0 indicates no symptoms; 1, no significant disability despite symptoms (able to carry out all usual duties and activities); 2, slight disability (unable to carry out all previous activities but able to look after own affairs without assistance); 3, moderate disability (requiring some help but able to walk without assistance); 4, moderately severe disability (unable to walk without assistance and unable to attend to bodily needs without assistance); 5, severe disability (bedridden, incontinent, and requiring constant nursing care and attention); and 6, dead. Data were available for 9763 patients (99.2%).
O’Donnell MJ, Fang J, D’Uva C, Saposnik G, Gould L, McGrath E, Kapral MK, Investigators of the Registry of the Canadian Stroke Network FT. The PLAN ScoreA Bedside Prediction Rule for Death and Severe Disability Following Acute Ischemic Stroke. Arch Intern Med. 2012;172(20):1548-1556. doi:10.1001/2013.jamainternmed.30
Author Affiliations: Health Research Board Clinical Research Facility, Geata an Eolais, National University of Ireland, Galway, Ireland (Drs O’Donnell and McGrath); Institute for Clinical Evaluative Sciences (ICES) (Drs Fang, Saposnik, and Kapral) and Departments of Neurology (Dr Saposnik) and Medicine (Dr Kapral), University of Toronto, Toronto, Ontario, Canada; Division of Geriatrics, Joseph Brant Hospital, Burlington, Ontario, Canada (Ms D’Uva); and Division of Neurology, Hamilton Health Sciences, Hamilton, Ontario, Canada (Ms Gould).
Background We sought to develop and validate a simple clinical prediction rule for death and severe disability after acute ischemic stroke that can be used by general clinicians at the time of hospital admission.
Methods We analyzed data from a registry of 9847 patients (4943 in the derivation cohort and 4904 in the validation cohort) hospitalized with acute ischemic stroke and included in the Registry of the Canadian Stroke Network (July 1, 2003, to March 31, 2008; 11 regional stroke centers in Ontario, Canada). Outcome measures were 30-day and 1-year mortality and a modified Rankin score of 5 to 6 at discharge.
Results Overall 30-day mortality was 11.5% (derivation cohort) and 13.5% (validation cohort). In the final multivariate model, we included 9 clinical variables that could be categorized as preadmission comorbidities (5 points for preadmission dependence [1.5], cancer [1.5], congestive heart failure [1.0], and atrial fibrillation [1.0]), level of consciousness (5 points for reduced level of consciousness), age (10 points, 1 point/decade), and neurologic focal deficit (5 points for significant/total weakness of the leg , weakness of the arm , and aphasia or neglect ). Maximum score is 25. In the validation cohort, the PLAN score (derived from preadmission comorbidities, level of consciousness, age, and neurologic deficit) predicted 30-day mortality (C statistic, 0.87), death or severe dependence at discharge (0.88), and 1-year mortality (0.84). The PLAN score also predicted favorable outcome (modified Rankin score, 0-2) at discharge (C statistic, 0.80).
Conclusions The PLAN clinical prediction rule identifies patients who will have a poor outcome after hospitalization for acute ischemic stroke. The score comprises clinical data available at the time of admission and may be determined by nonspecialist clinicians. Additional studies to independently validate the PLAN rule in different populations and settings are required.
Acute ischemic stroke results in death or severe dependence in approximately one-third of patients hospitalized with acute stroke by 1 month.1 At the time of admission, clinicians are frequently requested to estimate the likelihood of death and severe disability in the intermediate- and long-term poststroke period. Such information may be used to inform clinical decision making, such as resuscitation status, deciding on a palliative approach to care, or projecting the need for rehabilitative services and long-term care. In general, clinicians assume a nonstandardized approach to estimating prognosis, based on clinical experience and knowledge from the literature of clinical factors that are associated with an increased or reduced risk of death and severe disability.
Several prognostic rules have been developed for acute ischemic stroke2- 20; however, none has achieved widespread use in clinical practice. Clinical prediction rules are more likely to be accepted in clinical practice if they are memorable, are easy to use by nonspecialist clinicians, and possess adequate discrimination to predict clinically important outcomes. In stroke medicine, notable examples of prediction rules that are used widely for other clinical purposes include CHADS2 (congestive heart failure, hypertension, age >75 years, diabetes mellitus, and previous stroke or transient ischemic [TIA] attack)21 and ABCD2 (age, blood pressure, clinical features, duration of TIA, and presence of diabetes mellitus),22 which predict the risk for ischemic stroke in patients with atrial fibrillation and recent TIA, respectively. In fact, as exemplified by CHADS2 and ABCD2,23- 25 the simplicity of a prediction rule appears to be a more important determinant of its clinical use than high predictive precision.
In the Registry of the Canadian Stroke Network (RCSN), we derived and validated a simple clinical prediction rule for acute ischemic stroke that may be used by nonspecialist clinicians at the time of admission.
The RCSN collects detailed clinical data on consecutive patients with acute stroke and TIA seen in emergency departments or admitted to 11 regional stroke centers in Ontario, Canada.26 Patients with TIA, with intracerebral or subarachnoid hemorrhage, or younger than 18 years were excluded from the present study (Figure 1). In addition, patients receiving thrombolysis therapy were excluded from the analyses, since thrombolysis alters the natural history of disability after stroke and is not available in many centers (but is included in a post hoc sensitivity analysis in the present study).
The RCSN is “prescribed” under Ontario's Personal Health Information Protection Act, and patient data are collected without patient consent for the purpose of facilitating the provision of stroke care in the province of Ontario. Therefore, the cohort represents consecutive hospital admissions of patients with acute ischemic stroke. The RCSN is housed at the Institute for Clinical Evaluative Sciences in Ontario, where it is linked to population-based administrative databases using encrypted unique patient identifiers.
Approval for use of the RCSN was obtained from the Research Ethics Board at each participating center, as well as from the Research Ethics Board of Sunnybrook Health Sciences Centre. The study was reviewed and approved by the RCSN Publications Review Committee.
In consecutive patients with ischemic stroke, medical record data abstraction was performed during and after the hospital admission by experienced neurology research nurses (including L.G.) using custom software and laptop computers. Chart reabstraction studies26 have shown excellent agreement within the RCSN database, with κ scores greater than 0.8 for key variables (age, sex, stroke type, and comorbid conditions).
For the present study, we included all routinely captured baseline variables that have been reported to be associated (or proposed to be associated) with a poor outcome (death or severe dependence). Variables were divided into 3 categories. The first of these categories included preadmission factors, including age, sex, preadmission dependence, hypertension, diabetes mellitus, hyperlipidemia, congestive heart failure, myocardial infarction/angina, atrial fibrillation, chronic liver disease, dementia, and cancer. A history of hypertension, cancer, diabetes mellitus, and smoking was based on documentation in the patient's admission notes and did not include a new diagnosis made during incident hospitalization. The second category comprised initial clinical features, including weakness of the face, arms, and legs; aphasia; dysphagia; neglect; visual field deficit; and side of the symptoms. Although the RCSN records stroke severity based on the Canadian Neurological Scale (CNS),27 we did not use this composite measure in the present study because stroke severity scales are not completed in most stroke patients in routine clinical practice.19 The final category of variables included results of investigations (at the time of admission), including blood pressure (systolic and diastolic), temperature, hemoglobin level, white blood cell count, and creatinine. We did not include stroke subtype, based on either the Oxfordshire Community Stroke Project or Trial of Org 10172 in Acute Stroke Treatment (TOAST) subtype, because both require specialist expertise for valid determination and TOAST requires the results of investigations that usually are not available at the time of admission of many patients. Clinical assessments were completed by a local stroke specialist (usually a neurologist). Neuroimaging was performed in 98% of the patients.
Disability at discharge was captured using the modified Rankin score, which is recorded for all patients in the RCSN database. The m-Rankin score at discharge was available for 9763 (99.2%) of patients included in the present study. Death at 30 days and 1 year after hospital admission was identified by linkages with the Registered Persons Database, which captures all deaths within Canada.
Our approach to developing the clinical prediction rule is consistent with the approach of previous studies.28 After exclusion of patients without ischemic stroke and those who received thrombolysis therapy, the cohort was divided into 2 groups based on the date of admission (before or after December 1, 2005), which served as the derivation and validation cohorts (Table 1).
Our overall objective was to develop and evaluate a simple, user-friendly, score-based prediction rule that uses the same scoring system for each of the outcome measures, rather than developing 3 separate prediction rules for each outcome. Since the 3 outcomes are related, we anticipated some consistency in the estimates for most variables. Accordingly, selection of the candidate variables and their scores were based on (1) independent strength of association between each variable and each of the 3 outcomes on multivariate analysis, (2) prevalence of the variable, and (3) subcategory of the variable (ie, preadmission, clinical presentation, and result of investigation), enabling a simple categorized scoring system.
On univariate analyses, we selected candidate variables for inclusion in the multivariate analyses on the basis of their ability to predict (P ≤ .20) 30-day mortality, death, or severe dependence at the time of discharge (modified Rankin score of 5-6), and 1-year mortality. Limb weakness was derived from the CNS,27 which classified weakness into proximal and distal and into mild, significant, and total weakness for patients without comprehension deficit. On exploratory analyses, mild weakness was not significantly associated with any outcome. Therefore, weakness was dichotomized into significant or total weakness or mild or no weakness in either proximal or distal portion of the limb (or both). For patients with a comprehension deficit (18% of the entire cohort), the CNS documents whether power of the limb was equal or unequal to that of the corresponding limb, which was used as the dichotomy in these patients. For continuous variables, we used cubic spline plots to determine the relationship (strength and shape) between predictor variables and each outcome,29 and cutoff points were selected on the basis of interpretation of the graphical representation. We also explored combining some variables that would not be expected to correlate with each other (eg, aphasia and neglect, because these are most often opposing hemispheric symptoms/signs). Discrimination of the model was assessed by the area under the receiver operating characteristic curve, and calibration was assessed by the Hosmer and Lemeshow χ2 statistic and by comparing predicted and observed mortality or severe dependence in the sample using the Pearson correlation coefficient.30 We used a regression coefficient–based scoring method, developed from estimates generated by the multivariate model. Integer scores were assigned by dividing the risk-factor coefficient by the age coefficient (decades) and rounding to the nearest unit. Finally, based on the score for each variable within each outcome, we selected a consensus score that best represented the association with each of the outcomes studied, which was based on the mean integer score for all 3 outcomes. The overall risk score was calculated by adding each component score. Ultimately, modest sacrifices in precision were made to enable a clinical prediction rule that was simple, memorable, logical, and applicable to all outcome measures. For example, our final scoring system was modified to permit a total score of 25 points, with three 5-point categories and one 10-point category. We initially developed a prediction rule that used only clinical variables and then assessed the effect of adding variables for results of investigations on both the C statistic and the proportion of participants who were categorized as very low and very high risk (the C statistic may be insensitive to important changes in classification categories). We validated the scoring system in the validation cohort for all 3 outcome measures. In a post hoc analysis, we explored whether the clinical prediction score identified patients with a good outcome (ie, modified Rankin score, 0-2). Analyses were conducted using commercial software (SAS, version 9.2; SAS Institute Inc).
In total, 9847 patients—4943 in the derivation cohort (July 1, 2003, to November 30, 2005) and 4904 in the validation cohort (December 1, 2005, to March 31, 2008)—hospitalized with acute ischemic stroke were included. Table 1 reports selected characteristics of the derivation and validation groups. The mean age was 73 years, 48% were female, and the median CNS score was 9 (quartiles 1-3, 6.5-10.5). Overall 30-day mortality was 11.5% in the derivation cohort and 13.5% in the validation cohort.
Results of univariate analyses for all potential risk factors are presented in Table 2. For aphasia and neglect, the proportion of patients with both symptoms and signs was low and the magnitude of association on multivariate analyses was similar; therefore, these factors were combined into a single variable. For temperature, we observed a bimodal relationship, with an increased risk of mortality for lower (<36°C) and higher (>38°C) temperatures. Table 3 provides the results of multivariate analyses.
In the final multivariate model for clinical variables (Table 3), we included 9 variables that could be categorized as preadmission comorbidities (5 points for preadmission dependence [1.5], cancer [1.5], congestive heart failure [1.0], and atrial fibrillation [1.0]), level of consciousness (5 points for reduced level of consciousness), age (1 point per decade, maximum of 10 points), and neurologic deficit (5 points for severe proximal weakness of the leg , weakness of the arm , or aphasia or neglect ), to give a maximum score of 25. The PLAN score (based on preadmission comorbidities, level of consciousness, age, and neurologic deficit) (Table 4) predicted all 3 clinical outcomes in the derivation and validation cohorts (C statistic, 0.82-0.89). In the validation cohort, the PLAN score predicted 30-day mortality (0.87; 95% CI, 0.85-0.88), death or severe dependence at discharge (0.88; 0.87-0.89), and 1-year mortality (0.84; 0.82-0.85) (Figures 2, 3, and 4 and Table 3). In the entire cohort, 49.2% of patients had a PLAN score less than 10; 2.3% of these patients had a stroke associated with in-hospital death or severe disability at discharge (modified Rankin score, 5-6), and 13.2% of patients had a score higher than 15; 64.4% of these patients had a stroke associated with in-hospital death or severe disability by discharge (Table 5).
There was good calibration for all models (Hosmer and Lemeshow χ2; P > .20 for all models). In addition, there was high correlation between observed and expected 30-day mortality, 1-year mortality, and modified Rankin score of 5 to 6 by discharge (Pearson correlation coefficient, 0.99 for all models).
The C statistic for a favorable outcome (modified Rankin score, 0-2) was 0.77 (95% CI, 0.76-0.79) in the derivation cohort and 0.80 (0.78-0.81) in the validation cohort (Figure 5).
In the final multivariate model of the clinical and investigation variables, we included an additional 4 variables. These included glucose level, 180 mg/dL or more (1 point) (to convert to millimoles per liter, multiply by 0.0555); hemoglobin level, 11.0 g/dL or less (1.5 points) (to convert to grams per liter, multiply by 10); white blood cell count, 10 000/μL or more (1.5 points) (to convert to ×109 cells per liter, multiply by 0.001); and temperature lower than 36°C or higher than 38°C (1 point) (maximum score, 30).
The PLAN–Investigational Testing (PLAN-IT) score predicted all 3 clinical outcomes in the derivation and validation cohorts, but the incremental gains in the C statistic estimates were negligible (Table 3) and the proportion of patients who were classified as very high and low risk was similar.
We completed subgroup analyses by sex, age, ethnicity (white and nonwhite), side of symptoms, Oxfordshire Community Stroke Project subtype, and inpatients who received thrombolysis therapy (excluded from the primary analysis). We observed an apparent reduction in the C statistic for 2 patient groups: those with lacunar stroke and those receiving thrombolysis therapy (Table 6).
We reported a new clinical prediction rule (PLAN) with a simple scoring system that can be used by general clinicians at the time of admission of patients with acute ischemic stroke and appears to have adequate discrimination for use in clinical practice. Although the PLAN rule requires external validation in other populations, it represents a promising tool for use in hospital-based medicine.
Several prognostic clinical prediction rules have been developed in this patient population, but none is used widely in routine clinical practice.2- 20 Some prediction rules are not used clinically because of insufficient precision or limited generalizability because the population did not represent consecutive patients admitted with acute ischemic stroke.3- 5,9,12,15,17 Other rules, which report excellent precision,2,8,13,16,19,20 are difficult to use because of complex scoring systems that are a challenge to recall or calculate without web-based memory aids, they require a specialist's interpretation of investigations (eg, computed tomography and magnetic resonance imaging of the brain6,10), or they include a subscale that may require formal training (eg, National Institutes of Health Stroke Scale [NIHSS]31 or CNS27). Two recent clinical prediction rules19,20 were developed and validated in a large cohort of consecutive patients with acute ischemic stroke, and both reported excellent C statistics (0.85) for predicting mortality. However, similar to some other previous studies,6,9,10,17 both require completion of a stroke severity scale, such as the NIHSS. The NIHSS31 requires standardized training, is unfamiliar to many nonstroke specialist physicians, and is completed in fewer than half the patients admitted to the Get With the Guidelines–Stroke Program hospitals.19 Our instrument does not require completion of a subscale, uses a few key clinical variables that can be completed by general clinicians at the time of admission, and appears to have adequate discrimination for use in clinical practice.32 Of the other prediction models, the one developed by Counsell et al8 is most similar to the PLAN score and shares variable constructs. However, different definitions of key variables precluded a direct comparison between instruments, and a scoring system for their instrument was not reported.
A clinical prediction rule, such as the PLAN rule, provides a standardized approach to estimating prognosis in groups of patients. For individual patients, its use should complement, and not replace, clinical assessment and judgment. Similar to other scales, some of the dichotomized variables included in the score do not represent the full spectrum of disease seen in clinical practice, most notably for cancer and congestive heart failure. In addition, our rule does not include all conditions that may influence prognosis, such as advanced liver or renal disease. Similarly, we did not include the results of neuroimaging, which many clinicians use to inform prognosis. Although Reid et al18 failed to show that inclusion of baseline neuroimaging (eg, computed tomography) measures improved prognostic prediction, others6,33 have reported the prognostic importance of findings on neuroimaging. As such, the PLAN score provides a semiquantitative approach to a relatively imprecise clinical task.
Our study has several limitations. First, our population included patients admitted to regional stroke centers in Canada, and the findings may not be applicable to patients in other settings. In particular, outcomes after stroke may be influenced by greater availability of organized stroke services, stroke units, and potentially larger uptake of evidence-based therapies to prevent recurrent major vascular events and medical complications. Also, some measured (eg, atrial fibrillation) and unmeasured (eg, intracranial stenosis) etiologic factors may vary by region.34 Second, our study is a cross-sectional measurement of predictor variables and does not capture the dynamic change of neurologic deficit, which is an important determinant of outcome.35 Therefore, change in neurologic deficit will have important prognostic implications not captured in the PLAN score and emphasizes the importance of using the admission PLAN score as an adjunct to ongoing clinical assessment. Finally, our score is confined to patients with acute ischemic stroke and does not include patients with intracerebral hemorrhage, for whom other scales have been developed.36,37 Moreover, within the population of patients with acute ischemic stroke, we observed mitigated predictive precision for patients receiving thrombolysis therapy and those with lacunar stroke. Several factors may explain the reduced ability of the PLAN score to predict outcome in patients with lacunar stroke, including a lower proportion of those with a reduced level of consciousness and cortical symptoms; a lower prevalence of comorbid conditions, such as atrial fibrillation and congestive heart failure; and a lower overall mortality. Similarly, patients receiving thrombolysis therapy are likely to represent a narrower spectrum of stroke severity, and alternative approaches12 to prediction of outcome may be required in this population. Accordingly, the PLAN score needs to be interpreted with caution in these patient groups.
A key strength of the PLAN score is its ability to predict the composite of severe dependence and mortality; many previous studies have used only mortality.11,13,16,19,20 We believed it to be important to include severe disability, since some patients report very severe disability to be an outcome that is comparable to death.38 In addition, we sought to develop a single score for each outcome rather than separate ones for mortality and severe dependence, since using different scoring methods would likely reduce their application in clinical practice. Although based on post hoc analyses, our analysis also suggests that the PLAN score may be used to predict favorable outcome (ie, modified Rankin score, 0-2), which would extend its clinical applicability (Figure 5). We did not find that the addition of the results of investigations improved our ability to predict any outcome. A score that relies on clinical variables alone is of greater clinical utility in practice.
In conclusion, the PLAN clinical prediction rule identifies patients who will have a poor outcome after hospitalization for acute ischemic stroke. The rule uses clinical data available at the time of admission and may be completed by nonspecialist clinicians. Additional studies to independently validate the PLAN rule in different populations and settings are required.
Correspondence: Martin J. O’Donnell, MB, PhD, Health Research Board Clinical Research Facility, Geata an Eolais, National University of Ireland, Galway, Ireland (email@example.com).
Accepted for Publication: June 16, 2012.
Published Online: October 15, 2012. doi:10.1001/2013.jamainternmed.30
Investigators of the RCSN: The Investigators of the Canadian Stroke Registry are as follows: Frank Silver, MD, FRCPC (Coordinating Centre at the ICES); Moira Kapral, MD, MSc, FRCPC (ICES); Jack Tu, MD, MSc, PhD, FRCPC (ICES); R. Hall, BASc, MSc, PhD (ICES); J. Richards, RN (ICES); Annette Robertson, RN, RDCS (ICES); Neville Bayer, MD (St Michael's Hospital, Toronto, site principal investigator); Sandra Black, MD (Sunnybrook & Women's College Health Sciences Centre, Toronto, site principal investigator); Donald Brunet, MD (Kingston General Hospital, Kingston, Ontario site principal investigator); Vladimir Hachinski, MD (London Health Sciences Centre, London, Ontario, site principal investigator); David Howse, MD (Thunder Bay Regional Health Sciences Centre, Thunder Bay, Ontario, site principal investigator); John Maher, MD (Royal Victoria Hospital, Barrie, Ontario, site principal investigator); Wieslaw Oczkowski, MD (Hamilton Health Sciences, site principal investigator); Stephen Phillips, MD (Queen Elizabeth II Health Sciences Centre, Halifax, Nova Scotia, site principal investigator); Dan Selchen, MD (Trillium Health Centre, Mississauga, Ontario, site principal investigator); Mukul Sharma, BSc, MD, FRCPC, MSc (The Ottawa Hospital, Ottawa, Ontario, site principal investigator); Frank Silver, MD (University Health Network/Toronto Western Hospital, Toronto, site principal investigator); and Peter Zalan, MD (Hôpital Régional de Sudbury Regional Hospital, Sudbury, Ontario, site principal investigator).
Author Contributions: The authors had full access to the data in the study and take responsibility for the integrity of the data and accuracy of the data analysis. Study concept and design: O’Donnell, D’Uva, Saposnik, Gould, and Kapral. Acquisition of data: O’Donnell, D’Uva, Gould, and Kapral. Analysis and interpretation of data: O’Donnell, Fang, Saposnik, Gould, McGrath, and Kapral. Drafting of the manuscript: D’Uva and Saposnik. Critical revision of the manuscript for important intellectual content: O’Donnell, Fang, Saposnik, Gould, McGrath, and Kapral. Statistical analysis: O’Donnell, Fang, and D’Uva. Obtained funding: Kapral. Administrative, technical, and material support: O’Donnell and Gould. Study supervision: O’Donnell and Saposnik.
Conflict of Interest Disclosures: None reported.
Funding/Support: The RCSN is funded by grants from the Canadian Stroke Network and the Ontario Ministry of Health and Long-term Care and based at the ICES. The ICES is supported by the Ontario Ministry of Health and Long-term Care. Dr Kapral is supported by the Canadian Stroke Network and holds a New Investigator Award from the Canadian Institutes for Health Research.
Disclaimer: The results and conclusions are those of the authors and should not be attributed to any sponsoring organization.