Application of the 2017 Revised McDonald Criteria for Multiple Sclerosis to Patients With a Typical Clinically Isolated Syndrome

Key Points

Question  What is the diagnostic accuracy of the 2017 criteria vs the 2010 criteria to predict clinically definite MS (CDMS) in patients with a typical clinically isolated syndrome (CIS)?

Findings  This study included 229 patients with a clinically isolated syndrome. The sensitivity of the revised McDonald 2017 criteria was higher than for the 2010 criteria, and specificity was lower for the 2017 criteria.

Meaning  The 2017 revision of the McDonald MS criteria leads to a higher number of MS diagnoses in patients with a less active disease course.

Importance  In 2017, the International Panel on Diagnosis of Multiple Sclerosis revised the McDonald 2010 criteria for the diagnosis of multiple sclerosis (MS). The new criteria are easier to apply and could lead to more and earlier diagnoses. It is important to validate these criteria globally for their accuracy in clinical practice.

Objective  To evaluate the diagnostic accuracy of the 2017 criteria vs the 2010 criteria in prediction of clinically definite MS in patients with a typical clinically isolated syndrome (CIS).

Design, Setting and Patients  A total of 251 patients at Erasmus MC, Rotterdam, the Netherlands, in collaboration with several regional hospitals, fulfilled the inclusion criteria. Thirteen patients received another diagnosis early in the diagnostic process and therefore were excluded from the analyses. Nine patients with CIS declined to participate in the study. This left 229 patients who were included between March 2006 and August 2016 in this prospective CIS cohort. Patients underwent a baseline magnetic resonance imaging scan within 3 months after onset of symptoms and, if clinically required, a lumbar puncture was performed. Data were analyzed between December 2017 and January 2018.

Main Outcomes and Measures  Sensitivity, specificity, accuracy, and positive and negative predictive value were calculated after 1, 3, and 5 years for the 2017 vs the 2010 criteria.

Results  Among the 229 patients with CIS, 167 were women (73%), and the mean (SD) age was 33.5 (8.2) years. One hundred thirteen patients (49%) were diagnosed as having CDMS during a mean (SD) follow-up time of 65.3 (30.9) months. Sensitivity for the 2017 criteria was higher than for the 2010 criteria (68%; 95% CI, 57%-77% vs 36%; 95% CI, 27%-47%; P < .001), but specificity was lower (61%; 95% CI, 50%-71% vs 85%; 95% CI, 76%-92%; P < .001). Using the 2017 criteria, more MS diagnoses could be made at baseline (n = 97 [54%]; 95% CI, 47%-61% vs n = 46 [26%]; 95% CI, 20%-32%; P < .001). In the group with at least 5 years of follow-up, 33% of patients who were diagnosed as having MS using the 2017 criteria did not experience a second attack during follow-up vs 23% when using the 2010 criteria.

Conclusions and Relevance  The 2017 revised McDonald criteria are associated with greater sensitivity but less specificity for a second attack than the previous 2010 criteria. The tradeoff is that it leads to a higher number of MS diagnoses in patients with a less active disease course.

Multiple sclerosis (MS) is a chronic inflammatory demyelinating disease of the central nervous system (CNS), with substantial heterogeneity in severity and prognosis.¹ In 85% of patients, MS starts with a clinically isolated syndrome (CIS), a first clinical episode of CNS demyelination.² A CIS can remain a single event but can also be followed by the relapsing disease MS. Multiple sclerosis is diagnosed based on clinical or magnetic resonance imaging (MRI) evidence of dissemination in space (DIS) and time (DIT). The diagnostic criteria for MS evolved over the years to diagnose MS earlier and more easily.^3-6

Up to now, MS can be diagnosed when a typical CIS is followed by a new clinical event or when there are new lesions on T2-weighted images on a follow-up MRI scan. Since the McDonald 2010 criteria, MS can also be diagnosed based on a single baseline MRI scan showing asymptomatic contrast-enhancing lesions.⁶

Disease-modifying therapies (DMTs) can be administered early in the disease course, even at time of CIS. Disease-modifying therapies can delay a second attack after CIS^7-9 and have potential to prevent future disability.^10,11 However, DMTs have adverse effects. To select patients for early treatment, it is important to predict accurately who will develop a relapsing disease course and who will not.¹² Accurate diagnostic criteria are therefore essential.

In 2017, new diagnostic criteria were proposed for MS in patients with a typical CIS.¹³ These criteria are easier to apply than the McDonald 2010 criteria (Box). The most important addition is that the new criteria allow MS diagnosis when the MRI scan meets criteria for DIS and unique oligoclonal bands (OCB) are present in CSF, even in absence of DIT on the MRI scan. The other major difference is that not only asymptomatic but also symptomatic lesions can be used to demonstrate DIS and DIT on MRI. Furthermore, cortical lesions can be used to demonstrate dissemination in space.¹³

Box Section Ref ID

These revisions will presumably lead to a higher number of MS diagnoses at time of CIS. However, the revised criteria may be accompanied by a higher rate of false-positive (FP) diagnosis. Therefore, it is of great importance to apply these criteria to available cohorts of patients with CIS and validate their accuracy in clinical practice. We aimed to evaluate the diagnostic accuracy of these novel criteria.

Patients with a suspected first episode of MS were included in our prospective cohort of patients with CIS (Predicting the Outcome of a Demyelinating Event [PROUD] study), a multicenter study in Erasmus MC, Rotterdam, the Netherlands, a tertiary referral center for patients with MS (MS center ErasMS) in collaboration with several regional hospitals. Patients were between ages 18 and 50 years and were included between March 2006 and August 2016, within 3 months after onset of clinical symptoms, and with at least 1 year of follow-up. Patients were assessed at baseline and reassessed annually. No patients with a history of neurological symptoms suggestive for CNS demyelination were included. Patients underwent baseline MRI and routine laboratory tests to rule out other possible diagnoses. When an alternative diagnosis was made, the patient was excluded from analyses.

The study protocol was approved by the Ethics Committee of Erasmus MC, Rotterdam, the Netherlands, and of the other participating centers. All patients provided written informed consent.

A relapse was defined as new symptoms of neurological dysfunction or subacute worsening of existing symptoms after 30 days of improvement or stable disease and no evidence of an alternative diagnosis.¹⁴ Symptoms had to exist for longer than 24 hours and not be preceded by fever. Exacerbations were confirmed by neurological examination. Clinically definite MS (CDMS) was defined as clinical DIS and DIT as described by Poser et al.³

Magnetic resonance imaging scans were performed on a 1.5-T magnet with a standard head coil (Philips or General Electric). Images were obtained in the axial plane, and the following pulse sequences were used: T1-weighted conventional spin-echo, spin-echo proton-density weighted, T2-weighted spin-echo, and/or fluid-attenuated inversion recovery sequence, with 2- to 5-mm images.

All baseline MRI scans were scored for DIS using the McDonald 2010 criteria and the 2017 revisions to the McDonald criteria (n = 229). To evaluate the 2010 and 2017 criteria for DIT and DIS plus DIT, we selected patients who had a baseline MRI scan that included T1 images after gadolinium administration or scans that did not show any T2 hyperintense lesions (n = 180). We performed subanalyses for patients of whom we had data on OCB (n = 124), patients with baseline spinal cord MRI available (n = 79), and patients who were not treated with DMT before CDMS diagnosis (n = 135).

For statistical analyses we used SPSS software, version 24.0 (SPSS Inc) and GraphPad Prism5 (GraphPad Software). Patients who fulfilled the diagnostic criteria at time of the first attack and were diagnosed as having CDMS during follow-up were considered as true positives (TP). False-positives were defined as fulfilling the diagnostic criteria for MS at baseline but not diagnosed as having CDMS during follow-up. Patients who did not fulfill the diagnostic criteria and who were not diagnosed as having CDMS during follow-up were considered true negatives (TN), and patients who did not fulfill the diagnostic criteria at baseline but were diagnosed as having CDMS were considered as false-negatives (FN).

The following ratios were calculated:

Sensitivity: [TP / (TP+FN)] × 100

Specificity: [TN / (TN+FP)] × 100

Positive Predictive Value (PPV): [TP / (TP+FP)] × 100

Negative Predictive Value (NPV): [TN / (TN+FN)] × 100

Accuracy: [(TP + TN)/(TP + TN + FP + FN)] × 100

Ratios were calculated with a 95% confidence interval for DIS, DIT (with and without OCBs), and MS diagnoses on baseline MRI scans using the 2010 and 2017 McDonald criteria after 1, 3, and 5 years of follow-up. We compared sensitivity and specificity between the 2010 and 2017 criteria using the McNemar test.

For group comparison of continuous parametric variables, we used 2-tailed t test, and for nonparametric data, we used Mann-Whitney U test. For categorical data, we applied χ². Time to diagnosis using the 2010 and 2017 criteria and time to CDMS were analyzed using Kaplan-Meier curves and compared using log-rank test. Hazard ratios for time to CDMS were calculated using Cox proportional hazard regression analysis. Patients without a second attack during follow-up were considered as censored observations. P values less than .05 were considered significant, and all P values were 2-sided.

In total, 251 patients fulfilled the inclusion criteria, and 13 patients were diagnosed as having other diagnoses than MS, all at less than 3 months of follow-up (neuromyelitis optica, spinal cord tumor, chronic relapsing inflammatory optic neuropathy, meningioma, vascular, sarcoidosis, vitamin B₁₂ deficiency, or psychogenic). Nine patients declined to participate in the study. After these exclusions, 229 patients with CIS were eligible for analysis. All patients had at least 1-year follow-up time.

One hundred thirteen patients (49%) were diagnosed as having CDMS during a mean (SD) follow-up of 65.3 (30.9) months. Median time to CDMS diagnosis was 23.4 months (interquartile range, 10.2-45.3 months). Fifty-five patients (24%) were treated with DMT before CDMS diagnosis. The median time between onset of symptoms and baseline MRI scan was 3.7 weeks (interquartile range, 1.6-6.3 weeks). One hundred eighty patients (79%) had MRI scans with postgadolinium images available or scans that did not show any abnormalities. Baseline spinal cord MRI was performed in 107 patients (47%), of whom 78 (73%) had spinal cord symptoms. In 148 patients (65%), OCBs were assessed. Table 1 shows the baseline characteristics of the included patients.

The 2010 criteria for DIS were fulfilled for 124 of 229 patients (54%). Of them, 74 (60%) experienced a second relapse (CDMS) during follow-up. One hundred forty-nine patients (65%) fulfilled the 2017 criteria for DIS, and 89 (60%) were diagnosed as having CDMS.

To evaluate the 2010 and 2017 criteria for DIT, we selected patients who had a baseline MRI scan including postgadolinium T1 images (n = 180). The 2010 criteria for DIT were fulfilled for 46 of 180 patients (26%). Thirty-three of these patients (72%) were diagnosed as having CDMS. One hundred twenty-six of 180 patients (70%) fulfilled DIT according to the 2017 criteria. Of them, 76 (60%) experienced a second attack. Table 2 shows sensitivity, specificity, and accuracy with 95% confidence intervals for DIS and DIT according to the 2010 and 2017 criteria.

Using the McDonald 2010 criteria, 46 of 180 patients with CIS (26%; 95% CI, 20%-32%) were diagnosed as having MS at baseline. When using the 2017 criteria, 51 more patients were diagnosed as having MS at baseline for a total of 97 patients (54%; 95% CI, 47%-61%; P < .001). Thirty-three of 46 patients with MS (72%) using the 2010 criteria and 62 of 97 patients with MS (64%) using the 2017 criteria had a second attack during follow-up, resulting in a sensitivity of 36%; 95% CI, 27%-47% (2010) vs 68%; 95% CI, 57%-77% (2017) (P < .001) and a specificity of 85%; 95% CI, 76%-92% (2010) vs 61%; 95% CI, 50%-71% (2017) (P < .001). Table 2 shows sensitivity, specificity, and accuracy for the 2010 and 2017 criteria. For patients with at least 5-year follow-up (n = 88), 14 of 22 patients (64%) diagnosed as having MS using the 2010 criteria had a second attack before year 5. For the 2017 criteria, the total number of patients was 26 of 48 (54%).

This implies that 22 of 48 patients (46%), with at least 5-year follow-up and diagnosed as having MS at baseline using the 2017 criteria, were not yet diagnosed as having CDMS after 5 years. For the 2010 criteria, this was 8 of 22 patients (36%). For the total follow-up time in the group with at least 5-year follow-up, these numbers were 16 of 48 (33%) (2017) and 5 of 22 (23%) (2010). Sensitivity, specificity, PPV, NPV, and accuracy at 1, 2, and 5 years and for the total follow-up time for DIS and DIT for the 2010 and 2017 criteria are shown in the eTable in the Supplement.

The Figure shows the survival curves for CDMS according to the Poser et al criteria,³ the McDonald 2010 criteria, and the revised 2017 criteria. Multiple sclerosis diagnosis is made earlier using the 2017 compared with the 2010 criteria (χ² = 40.49; P < .001). Table 2 shows the hazard ratios for DIS, DIT, and DIS and DIT for the 2010 and 2017 criteria.

Using the new 2017 criteria, twice as many patients could be diagnosed as having MS at baseline compared with the 2010 criteria (97 vs 46 patients). In 32 of these extra MS diagnoses (63%), the diagnosis could be made based on DIT fulfillment with OCB in CSF. Of these 32 patients, 47% were not diagnosed as having CDMS during follow-up. The other major difference between the criteria is that not only asymptomatic but also symptomatic lesions could be used to demonstrate DIS and DIT on MRI. This led to the other 19 additional MS diagnosis at baseline (37%). Of these patients, 31% had no second attack.

For all subanalyses, we selected patients who had a baseline MRI scan that included T1 images after gadolinium administration (n = 180). Patients with known OCB status were selected for the first subanalysis (n = 124[ 69%]). Sensitivity for the 2010 vs the 2017 criteria was 32% (95% CI, 21%-45%) and 70% (95% CI, 57%-80%), respectively (P < .001). The specificity was 75% (95% CI, 55%-89%) vs 53% (95% CI, 40%-67%) (P < .001). In 79 of 180 patients (44%), a baseline spinal cord MRI was performed. Here, sensitivity for the 2010 criteria was 29% (95% CI, 16%-45%) and for the 2017 criteria was 72% (95% CI, 58%-86%) (P < .001). The specificity was 84% (95% CI, 67%-93%) vs 49% (95% CI, 32%-65%) (P < .001).

Because DMT could have postponed a second attack, we performed another subanalysis excluding the patients treated with DMT before CDMS diagnosis, leaving 135 of 180 patients (75%) eligible for analyses. Here, sensitivity for the 2010 criteria was 27% (95% CI, 17%-40%) and for the 2017 criteria was 58% (95% CI, 45%-70%) (P < .001). Specificity was 90% (95% CI, 81%-96%) vs 73% (95% CI, 61%-82%) (P < .001).

We evaluated the accuracy of the 2017 revisions of the McDonald criteria vs the McDonald 2010 criteria to predict CDMS at the moment of a first demyelinating attack. The criteria for DIS and DIT used in the 2010 and 2017 criteria were applied to a prospective cohort of 229 patients with CIS during a mean follow-up of 5.4 years.

We observed higher sensitivity for the 2017 criteria than for the 2010 criteria (68% vs 36%). However, as expected, specificity for the 2017 criteria was lower (61% vs 85%). The accuracy did not differ significantly (accuracy: 61% [2010] and 64% [2017]). Accuracy of the McDonald 2010 criteria was similar to earlier studies validating the 2010 criteria in CIS cohorts.^15-18

High sensitivity of diagnostic criteria is important to allow earlier initiation of DMT, which has been shown beneficial for disease outcome.^7,9,10 On the other hand, incorrect diagnoses and unnecessary treatment should be avoided. In our data, specificity of the 2017 criteria was significantly lower than for the 2010 criteria. Earlier data showed that the previous McDonald criteria lead to a higher number of MS diagnoses in patients who will not have a second attack.¹⁹ When new, less strict criteria are introduced, the Will Rogers phenomenon could be observed.²⁰ This refers to the statistical observation that when a boundary on a scale is moved to the left, the outcome for both patient groups improves. More patients with CIS will be moved to the MS group; therefore, the MS group will probably have a more favorable outcome with a lower attack frequency. This makes retrospective comparison with older studies in MS research challenging because the overall prognosis for MS patients is getting better.

With these new MS criteria, MS diagnosis includes a single CIS attack without subsequent clinical disease activity and therefore comes even closer to the group with radiologically isolated syndrome (RIS). Patients with RIS cannot be diagnosed as having MS using the current criteria. Because DMTs can delay a second attack in patients with CIS and have a potential to prevent future disability, the discussion comes closer to whether patients with RIS who are at high risk for future attacks and already have cognitive problems²¹ should also be diagnosed as having MS and be treated with DMT.

We found that after 5 years of follow-up, the number of patients fulfilling the criteria but with no second attack was higher for the 2017 than for the 2010 criteria (46% vs 36%). However, in the period after these 5 years of follow-up, 8 more patients were diagnosed as having CDMS. Of them, 6 fulfilled the 2017 criteria at baseline and 3 fulfilled the 2010 criteria. This shows that a longer follow-up will probably lead to a higher PPV of the criteria.

Another potential explanation for the high number of patients who fulfilled the criteria but did not experience a second event is treatment with DMT before CDMS diagnosis. This could have postponed or prevented a second attack and therefore lowered the specificity of the criteria. We did not exclude patients who used DMT before CDMS diagnosis; this could have introduced selection bias. Instead of exclusion, we performed a separate analysis with the group that did not receive DMT before second attack. In this subanalysis, the specificity increased for both criteria. But after at least 5 years of follow-up, even in this untreated group, 4 of 15 patients (27%) who fulfilled the 2010 criteria and 11 of 33 patients (33%) who fulfilled the 2017 criteria did not experience a second attack.

Sixty-three percent of the extra MS diagnoses using the new diagnostic 2017 criteria were made based on OCB in CSF; 47% of these patients did not experience a second attack. The other 37% of new MS diagnoses could be made based on demonstrating DIS or DIT with asymptomatic enhancing lesions on MRI; 31% of these did not experience a second attack. Inclusion of OCB in the 2017 criteria especially seems to contribute to the lower specificity.

There are some limitations to this study. Although there was a sufficient follow-up to draw conclusions (mean follow-up time, 5.4 years), the range is rather wide (SD, 2.6 years). The possibility remains that after further follow-up, additional patients will have a second attack. However, the total number of patients in this study was large enough to allow subanalyses for different minimum follow-up times, and this demonstrated maintenance of accuracy together with a limited decrease of sensitivity (eTable in the Supplement).

Second, not all patients underwent a baseline spinal cord MRI scan and lumbar puncture. Although it is known to increase sensitivity,^22,23 it is not common clinical practice to include spinal cord MRI in the routine diagnostic workup when there are no spinal cord symptoms. Also, a lumbar puncture was not always clinically required and we did not have ethical permission for obtaining CSF for research purposes alone. Therefore, the decision to perform a lumbar puncture or a spinal cord MRI was not random. To avoid selection bias, we did not exclude the patients who did not have CSF or spinal cord MRI data available. In this way, our findings are more applicable to the general clinical practice. However, we did perform subanalyses, and specificity and sensitivity did not differ from the total group.

Third, although applied in most studies on MS criteria for patients with CIS,^24-26 the use of sensitivity, specificity, NPV, and PPV is somewhat problematic. They imply reference to a criterion standard. Obviously, in this patient group we do not have neuropathologic information. Earlier studies used pathologically confirmed CDMS cases to demonstrate that the highest rate of correct diagnoses could be made when using the Poser CDMS criteria.^27,28 Therefore, we accepted the use of CDMS as a proxy to pathological evidence to confirm MS diagnosis.

Lastly, in the 2017 criteria, cortical lesions are added as an extra parameter to show DIS. Advanced MRI techniques are required to visualize cortical lesions. These techniques are hardly available in routine clinical practice¹³; therefore, at present we cannot study the contribution of this parameter. A 2016 study,²³ in which the Magnetic Resonance Imaging in MS 2016 MRI criteria were validated in a large retrospective CIS cohort, showed that inclusion of cortical lesions did not affect DIS criteria performance.²⁵

In conclusion, the 2017 revisions to the McDonald criteria are more sensitive than the previous 2010 criteria. Therefore, the new diagnostic criteria will probably increase the proportion of MS diagnoses. However, the specificity is significantly lower when applied to our cohort of patients with CIS, leading to a higher number of MS diagnoses with a less active disease course, at least in the first years after onset.

Corresponding Author: Rogier Q. Hintzen, MD, PhD, Department of Neurology, MS Center ErasMS, Erasmus MC, PO Box 2040, 3000 CA Rotterdam, the Netherlands (r.hintzen@erasmusmc.nl).

Accepted for Publication: June 1, 2018.

Published Online: August 6, 2018. doi:10.1001/jamaneurol.2018.2160

Author Contributions: Dr Hintzen had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: van der Vuurst de Vries, Hintzen.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: van der Vuurst de Vries.

Critical revision of the manuscript for important intellectual content: Mescheriakova, Wong, Runia, Jafari, Samijn, de Beukelaar, Wokke, Siepman, Hintzen.

Statistical analysis: van der Vuurst de Vries, Jafari.

Administrative, technical, or material support: Jafari.

Supervision: Jafari, Hintzen.

Conflict of Interest Disclosures: Dr Jafari received honoraria for serving on advisory boards for Teva, Merck, Roche, and Sanofi Genzyme. Dr Samijn received honoraria for serving on advisory boards for Merck, Sanofi Genzyme, and Roche and received travel grants from Merck. He participated in trials with BiogenIdec, Merck, Roche, and Sanofi Genzyme outside the submitted work. Dr Hintzen received honoraria for serving on advisory boards for BiogenIdec, Roche, and Sanofi. He participated in trials with BiogenIdec, Merck, Roche, Sanofi Genzyme, and Novartis, outside the submitted work. No other disclosures are reported.

Funding/Support: The study was supported by the Dutch Multiple Sclerosis Research Foundation.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the patients and the participating centers of the Predicting the Outcome of a Demyelinating Event study. No compensation was received from a funding source.