Diagnostic Performance of Cortical Lesions and the Central Vein Sign in Multiple Sclerosis

Key Points Question Can multiple sclerosis (MS) be differentiated from a wide range of non-MS conditions showing brain white matter lesions using solely imaging biomarkers for cortical lesions (CLs) and central vein sign (CVS)? Findings In this cross-sectional study including 1051 participants, the presence of CLs had high specificity and low sensitivity, while application of the 40% CVS rule resulted in high specificity and moderate sensitivity for MS diagnosis. CVS and CLs outperformed the contribution of infratentorial, periventricular, and juxtacortical lesions in supporting the diagnosis of MS. Meaning The findings indicate that CVS and CLs may be valuable tools to increase the accuracy of MS diagnosis.

For CVS assessment, lesions smaller than 3 mm in the shortest diameter were automatically removed from the WML masks.The resulting masks were manually refined to remove confluent lesions, registered to the space of the susceptibility-based image, and used as reference for CVS rating.Whenever T2*-weighted and FLAIR images were available, the combined FLAIR* contrast was generated. 8nce CL and CVS assessments were performed on native images, in which the location and morphology of WMLs may bias the rater toward a specific diagnosis, additional analyses were performed to quantify the reproducibility of the assessments after blinding the raters to the general appearance of the scan.For CLs, a subset of 200 randomly-selected 3D-T1/MP2RAGE images (distributed among various MRI contrasts and participating centers to replicate the overall cohort's distribution) were re-assessed for the presence of CLs after nullifying the white matter signal.Specifically, an automatic segmentation of the white matter was obtained with SAMSEG; 9 the segmentation was then used to remove the white matter signal from the native images (after a one-voxel erosion, to preserve the border between cortical gray matter and white matter in the final images; eFigure 9, panel A).The agreement in CL count obtained on native images and on white matter-nullified images, as measured with the intraclass correlation coefficient (ICC), 10 was 0.96 (95% confidence interval [CI]: 0.94-0.97).For CVS, one randomly-selected WML for each participant was re-assessed for the presence of CVS after being cropped from the native image (eFigure 9, panel B).The agreement in the CVS rating on native and cropped images, as measured with the Cohen's kappa coefficient, was 0.847 (95%-CI: 0.811-0.884).

CL subtypes
On native 3D-T1/MP2RAGE images, CLs were manually categorized -by consensus -in intracortical (i.e.confined to the cortex), and leukocortical (i.e.concomitantly involving the cortex and white matter).Leukocortical lesions were further differentiated in those with prevalent gray matter/white matter involvement.The presence of curvilinear/"wormshaped" CLs 2 was also recorded.The prevalence of CL subtypes in the different diagnosis groups is reported in eTable 4.

Inter-rater agreement in CL count
The inter-rater agreement in CL count was estimated with the ICC, using a two-way mixed-effects model.The interrater agreement was estimated separately for each MRI contrast used for CL assessment.

Comparison between sequences for CVS assessment
We compared the diagnostic performance of the CVS on different susceptibility-based sequences by using the DeLong method. 11Additionally, we compared the proportion of CVS-positive lesions on different susceptibility-based sequences in patients with MS/CIS by using logistic regression models, adjusted for age, sex, and disease duration.The diagnostic performance of the CVS was not significantly different in the subgroup of subjects with T2*-weighted images and in the subgroup of subject with SWI images (AUC=0.893[95%-CI: 0.850-0.935]and AUC=0.872[95%-CI: 0.839-0.905],respectively; p=.45).Similarly, there was no difference in the diagnostic performance in the subgroup of subjects with optimized, submillimetric 3D-EPI images and in the subgroup of subjects with SWI images (AUC=0.877[95%-CI: 0.817-0.937]and AUC=0.872[95%-CI: 0.839-0.905],respectively; p=.88).Based on Youden's index, the best threshold for discrimination between MS/CIS and non-MS was 26% on SWI images, and 34% on 3D-EPI images.The proportion of CVS-positive lesions in patients with MS/CIS was not significantly different between T2*-weighted and SWI images (odds ratio [OR]: 1.024; p=.60), while it was significantly higher on optimized, submillimetric 3D-EPI images compared to SWI images (OR: 1.182; p<.001).

Random forest model
The random forest model was fitted in the group of patients with availability of both CL and CVS data (n=932).The variables included in the random forest model were: 1) the proportion of CVS-positive lesions, 2) CL count, 3) presence/absence of periventricular WMLs, 4) presence/absence of juxtacortical WMLs, and 5) presence/absence of infratentorial WMLs.In a sensitivity analysis, CVS and CL were used as dichotomous variables: specifically, the fulfillment of the "40%-CVS rule" and the presence/absence of CL were the variables entered in the model, together with the presence/absence of periventricular, juxtacortical, and infratentorial WMLs.The AUC was 0.931 (95%-CI: 0.912; 0.951) in training, and 0.905 (95%-CI: 0.869-0.940) in test subsets.The mean decrease in accuracy (MDA) was 63.0 for CVS, 39.2 for CLs, 16.3 for infratentorial WMLs, 12.8 for periventricular WMLs, and 10.6 for juxtacortical WMLs.

Analyses within the subgroup of participants with availability of oligoclonal bands information
Information on CSF-specific oligoclonal bands (OCBs) status was available for 505 participants: 371 with MS, 48 with CIS, 14 with AQP4-positive NMOSD, 16 with seronegative-NMOSD, 13 with MOGAD, 8 with migraine, 25 with inflammatory vasculopathies, and 10 with cerebrovascular disease.Within this cohort, the presence of CSF-specific oligoclonal bands was associated with a sensitivity of 87.1%, a specificity of 80.2%, and an accuracy of 85.9% for the discrimination between MS/CIS and non-MS.We assessed the relative contribution of i) OCBs status, ii) CL count, and iii) the proportion of CVS-positive lesions in supporting the distinction between MS/CIS and non-MS conditions using a multivariable logistic regression model.Specifically, the diagnosis (MS/CIS vs non-MS) was used as the dependent variable, while OCBs status, CL count, and the proportion of CVS-positive lesions were included as independent variables.In this model, all three biomarkers demonstrated significant and independent associations with the diagnosis, as outlined below:

Between-sex differences in the proportion of CVS-positive lesions
Post-hoc analyses were conducted to further characterize the observed difference in CVS prevalence between sexes.The analyses encompassed: 1. the assessment of the association between the proportion of CVS-positive lesions and sex in participants with MS, adjusting for potential clinical, demographic, and MRI confounding factors.The analysis was conducted with a multivariable logistic regression model including age, disease duration, EDSS, disease course, and white matter lesion load as covariates; 2. the assessment of the association between the proportion of CVS-positive lesions and sex in participants with non-MS conditions.The analysis was conducted with a logistic regression model using sex as exploratory variable; 3. a comparison of the diagnostic performance of the CVS in females and males.The performance of the proportion of CVS-positive lesions in discriminating between MS/CIS and non-MS conditions was explored with ROC curves separately in males and females.The AUC of the ROC curves obtained in the two groups was compared with the DeLong method. 11e results confirmed a negative association between the proportion of CVS-positive lesions and female sex among participants with MS also when adjusting for relevant

Simplified criteria for CVS assessment
Due to the need to evaluate all lesions that meet the NAIMS criteria in order to estimate the proportion of CVS-positive lesions, this process can be time-consuming, potentially hindering its practical use in clinical settings.Consequently, various simplified criteria have been proposed in the literature.In our study, we have investigated the diagnostic performance of four simplified algorithms for CVS assessment in a subset of the study cohort: 1. "Select-3" [12][13][14] : for each patient, 3 lesions of the subcortical or deep white matter were randomly selected on FLAIR/T2 images.Scans with <3 candidate lesions were excluded.The selected lesions were then assessed for the presence of the CVS on the susceptibility-based contrast.2. "Pick-6" [13][14] : for each patient, 6 white matter lesions were randomly selected on FLAIR/T2 images.Scans with <6 candidate lesions were excluded.The selected lesions were then assessed for the presence of the CVS on the susceptibility-based contrast.3. "Select-n*" 12,[14][15][16] : the presence of CVS-positive lesions was directly determined on the FLAIR* contrast.
Participants were classified as having MS if at least n lesions displayed the CVS over the entire FLAIR* image, with n ranging from 1 ("Select-1*") to 6 ("Select-6*").4. The performance of the "Select-6*" approach was also assessed with an algorithm ("Select-6* 2 ") classifying participants as having MS if they met one of the two following criteria: 1. at least 6 lesions exhibiting the CVS; 2. CVS-positive lesions outnumbering CVS-negative lesions, in case fewer than 6 CVS-positive lesions were present. 14,15,17e performance of the "Select-3" and "Pick-6" algorithms was assessed in a subgroup consisting of half of the study cohort (ensuring that the selected group had a distribution across centers/MRI protocols and diagnoses representative of the entire cohort).The performance of the "Select-n*" algorithms was assessed in all participants with the availability of a 3D-EPI contrast (n=310).
For all algorithms, we assessed the diagnostic performance for various thresholds, with ROC curves.We then compared the diagnostic performance of each algorithm to that of the proportion of CVS-positive lesions within the same subset of participants using the DeLong method. 11sults of the analyses are reported in eTables 7-5 and eFigures 10-12.

Average duration for CL and CVS assessments
© 2023 Cagol A et al.JAMA Neurology.
The time required to conduct CL and CVS assessments was quantified in a subgroup of participants.Specifically:  For the CL assessment, we randomly selected 20 3D-T1 or MP2RAGE scans, ensuring a balanced distribution across MRI protocols and centers.Two raters independently evaluated the selected scans and measured the time needed to complete the assessment for each scan.
 For the CVS assessment, we randomly selected 20 scans.To account for the significant impact of white matter lesion load on CVS assessment time, we ensured that 5 scans were selected from each of the four quartiles of the distribution of the white matter lesion load within the study population.The scans were selected from participants who had T2* contrast availability, allowing us to compare the time required for assessing the proportion on CVSpositive lesions with that for CVS assessment based on the Select-6* 2 algorithm.As for the CL assessment, two raters independently evaluated the selected scans and measured the time needed to complete the assessment for each scan.
The median time required to complete the CL assessment was 4.3 minutes (range: 3.4-6.4).The median time required to complete the CVS assessment based on the proportion of CVS-positive lesions was 3.3 minutes (range: 1.0-8.5).The median time required to complete the CVS assessment based on the Select-6* 2 algorithm was 1.1 minutes (range: 0.9-1.6), which was significantly shorter compared to the time required for the assessment based on the proportion of CVSpositive lesions (p<.001).
eTable 1. Overview of MRI protocols

eFigure 1 .
Diagnostic performance of CLs, CVS, and their combination in the entire cohort and in the subgroup of patients with < 2 years of disease duration Entire cohort: n=934; including only MS/CIS patients with < 2 years of disease duration: n=657.Abbreviations: CLs, cortical lesions; CVS, central vein sign; AUC, area under the curve.

eFigure 2 .eFigure 3 .
Comparison of CVS diagnostic performance in the entire cohort vs in subjects with ≥3 lesions suitable for assessment Entire cohort: n=934; subjects with ≥ 3 lesions suitable for CVS analysis: n=773; subjects with < 3 lesions suitable for CVS analysis: n=161.Abbreviations: CVS, central vein sign; MS, multiple sclerosis; CIS, clinically isolated syndrome; AUC, area under the curve.Comparison of CVS diagnostic performance: using a proportion-based threshold vs using the absolute number of CVS-positive lesions eTable 3: Sensitivity, specificity and accuracy for various absolute numbers of CVS-positive lesions

eFigure 9 .eTable 5 .
Examples of images used to blind raters to the general appearance of the scanPanel A: example of 3D-T1 image in which the signal of the white matter was removed.Panel B: example of cropped white matter lesions on FLAIR* images.Diagnostic performance of the "Select-3" algorithm

Comparison of the diagnostic performance of the "Select-3" algorithm and the proportion of CVS-positive lesions eTable 6. Diagnostic performance of the "Pick-6" algorithm
Abbreviations: AUC, area under the curve; CI, confidence interval.