[Skip to Content]
[Skip to Content Landing]
Views 1,214
Citations 0
Research Letter
April 20, 2020

Comparison of Placebos and Propensity Score Adjustment in Multiple Sclerosis Nonrandomized Studies

Author Affiliations
  • 1Section of Biostatistics, Department of Health Sciences, University of Genova, Genova, Italy
  • 2Biogen International GmbH, Baar, Switzerland
  • 3Biogen Inc, Cambridge, Massachusetts
  • 4IRCCS Ospedale Policlinico San Martino, Genova, Italy
JAMA Neurol. 2020;77(7):902-903. doi:10.1001/jamaneurol.2020.0678

In the last decade, methods based on propensity scores (PSs) have been frequently used in multiple sclerosis (MS) studies comparing disease-modifying treatments in nonrandomized observational settings.1 Propensity score adjustment was applied even in situations when all the necessary conditions for its applicability were not satisfied. The PS adjustment can reduce the intrinsic selection bias of nonrandomized studies only if all the confounders are measurable and are at least minimally overlapped between the treatment groups. Sometimes the calendar period or the geographical region of study conduction can be completely nonoverlapping between the compared groups. In such cases, in the causal inference jargon, the positivity assumption, requiring that the probability to receive any of the treatments in all the PS strata is higher than 0, is violated. We will show, with a practical example, the extent of failure of PS adjustment when the positivity assumption is violated.

Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Propensity Score Adjustment in Multiple Sclerosis Nonrandomized Studies: a Neurologist's Perspective
    Luca Prosperini, MD, PhD | S. Camillo-Forlanini Hospital, MS Center and Neurology Unit, Rome, Italy
    I have read with great interest the research letter by Signori et al. and I fully agree that propensity score-based comparisons of non-randomized observational data should be interpreted with great caution. The Authors have elegantly demonstrated the bias generated by the violation of the positivity assumption in the context of PS-based adjustment that have been frequently used in multiple sclerosis (MS) studies.
    However, I guess, the question is… why there is an increasing number of real world studies comparing different treatment strategies in MS?
    The answer probably is that “…in the era of precision medicine, the aim in the
    clinic is no longer to (simply) prescribe an available treatment to a given patient, naive or nonresponder, but to prescribe the best possible treatment for that given patient, with a very specific set of clinical and paraclinical characteristics” (Tur C et al. Neurology 2019; 93:1-17).
    Multicenter, randomized, controlled clinical trials are undoubtedly the gold standard to obtain evidence-based data on the relative efficacy and safety profile of MS treatments. Unfortunately, head-to-head clinical trials are scarce and indirect comparisons of placebo-controlled trials are also biased by multiple factors, including (but not limited to) difference in study design, targeted population, baseline patients’ characteristics and behaviour of placebo groups.
    Consequently, important clinical questions would remain unanswered without real world studies.
    To give you some examples.
    (1) Is escalation to monoclonal antibodies better than switching to different immunomodulant agents? In April 2012, the SURPASS study (ClinicalTrials.gov Identifier: NCT01058005) was stopped prematurely by the sponsor due to slow patient enrolment. In the following years, observational PS-based studies suggested that escalation is better than switching in patients with unsatisfactory response to platform therapies (Prosperini et al. Mult Scler 2012; 18:64-71; Brown et al. JAMA 2019; 321:175-187).
    (2) Is Natalizumab superior to Fingolimod on MS-related brain tissue damage? In May 2016, the REVEAL study (ClinicalTrials.gov Identifier: NCT02342704) was stopped prematurely by the sponsor for business decision. Later, observational PS-based studies suggested the superiority of Natalizumab over Fingolimod on conventional magnetic resonance imaging markers (Barbin et al. Neurology 2016; 86:771-8; Baroncini D et al. Mult Scler 2016; 22:1315-26).
    (3) Which is the best oral drugs? In October 2019, the PRAG-MS study (ClinicalTrials.gov Identifier: NCT03345940), supported by PCORI, was terminated due to slow enrolment rate. More recently, observational PS-based studies aimed to compare the currently available oral drugs (Kalincik et al. J Neurol Neurosurg Psychiatry 2019; 90: 458-468; Ontaneda D et al. Mult Scler Relat Disord 2019; 27:101-111).
    My impression is that pharmaceutical companies have no interest in carrying out head-to-head trials which instead would provide extremely relevant clinical answers. In the absence of these data, we will continue to rely on real world studies, which although only hypotheses-generating and less robust, nevertheless provide relevant insights in the complex therapeutic management of MS.