Randomized Trial Replication Using Observational Data for Comparative Effectiveness of Secukinumab and Ustekinumab in Psoriasis: A Study From the British Association of Dermatologists Biologics and Immunomodulators Register | Dermatology | JAMA Dermatology | JAMA Network
[Skip to Navigation]
Figure.  Forest Plot of the Risk Ratio (RR) Estimates for Participants Achieving Psoriasis Area and Severity Index of 2 or Lower at 12 Months
Forest Plot of the Risk Ratio (RR) Estimates for Participants Achieving Psoriasis Area and Severity Index of 2 or Lower at 12 Months
Table 1.  Baseline Characteristics of the Ustekinumab and Secukinumab Cohorts
Baseline Characteristics of the Ustekinumab and Secukinumab Cohorts
Table 2.  Outcome of Absolute PASI of 2 or Lower at 12 Months for Secukinumab and Ustekinumaba
Outcome of Absolute PASI of 2 or Lower at 12 Months for Secukinumab and Ustekinumaba
1.
Sbidian  E, Chaimani  A, Afach  S,  et al.  Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis.   Cochrane Database Syst Rev. 2020;1:CD011535. doi:10.1002/14651858.CD011535.pub3 PubMedGoogle Scholar
2.
Mason  KJ, Barker  JNWN, Smith  CH,  et al; BADBIR Study Group.  Comparison of drug discontinuation, effectiveness, and safety between clinical trial eligible and ineligible patients in BADBIR.   JAMA Dermatol. 2018;154(5):581-588. doi:10.1001/jamadermatol.2018.0183 PubMedGoogle Scholar
3.
Yiu  ZZN, Mason  KJ, Barker  JNWN,  et al; BADBIR Study Group.  A standardization approach to compare treatment safety and effectiveness outcomes between clinical trials and real-world populations in psoriasis.   Br J Dermatol. 2019;181(6):1265-1271. doi:10.1111/bjd.17849 PubMedGoogle Scholar
4.
Hernán  MA, Robins  JM.  Using big data to emulate a target trial when a randomized trial is not available.   Am J Epidemiol. 2016;183(8):758-764. doi:10.1093/aje/kwv254 PubMedGoogle Scholar
5.
Franklin  JM, Schneeweiss  S.  When and how can real world data analyses substitute for randomized controlled trials?   Clin Pharmacol Ther. 2017;102(6):924-933. doi:10.1002/cpt.857 PubMedGoogle Scholar
6.
Franklin  JM, Pawar  A, Martin  D,  et al.  Nonrandomized real-world evidence to support regulatory decision making: process for a randomized trial replication project.   Clin Pharmacol Ther. 2020;107(4):817-826. doi:10.1002/cpt.1633PubMedGoogle Scholar
7.
Thaçi  D, Blauvelt  A, Reich  K,  et al.  Secukinumab is superior to ustekinumab in clearing skin of subjects with moderate to severe plaque psoriasis: CLEAR, a randomized controlled trial.   J Am Acad Dermatol. 2015;73(3):400-409. doi:10.1016/j.jaad.2015.05.013 PubMedGoogle Scholar
8.
Blauvelt  A, Reich  K, Tsai  TF,  et al.  Secukinumab is superior to ustekinumab in clearing skin of subjects with moderate-to-severe plaque psoriasis up to 1 year: results from the CLEAR study.   J Am Acad Dermatol. 2017;76(1):60-69.e9. doi:10.1016/j.jaad.2016.08.008PubMedGoogle Scholar
9.
Iskandar  IY, Ashcroft  DM, Warren  RB,  et al.  Demographics and disease characteristics of patients with psoriasis enrolled in the British Association of Dermatologists Biologic Interventions Register.   Br J Dermatol. 2015;173(2):510-518. doi:10.1111/bjd.13908 PubMedGoogle Scholar
10.
Burden  AD, Warren  RB, Kleyn  CE,  et al; BADBIR Study Group.  The British Association of Dermatologists’ Biologic Interventions Register (BADBIR): design, methodology and objectives.   Br J Dermatol. 2012;166(3):545-554. doi:10.1111/j.1365-2133.2012.10835.x PubMedGoogle Scholar
11.
Mavridis  D, Salanti  G, Furukawa  TA, Cipriani  A, Chaimani  A, White  IR.  Allowing for uncertainty due to missing and LOCF imputed outcomes in meta-analysis.   Stat Med. 2019;38(5):720-737. doi:10.1002/sim.8009 PubMedGoogle Scholar
12.
Yiu  ZZN, Mason  KJ, Hampton  PJ,  et al; BADBIR Study Group.  Drug survival of adalimumab, ustekinumab and secukinumab in patients with psoriasis: a prospective cohort study from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR).   Br J Dermatol. 2020;183(2):294-302. doi:10.1111/bjd.18981 PubMedGoogle Scholar
13.
Dávila-Seijo  P, García-Doval  I.  Drug survival analysis is not a good method for assessing the safety or effectiveness of systemic therapies in psoriasis.   Actas Dermosifiliogr. 2017;108(1):3-5. doi:10.1016/j.adengl.2016.11.001 PubMedGoogle Scholar
14.
Dahabreh  IJ, Robins  JM, Hernán  MA.  Benchmarking observational methods by comparing randomized trials and their emulations.   Epidemiology. 2020;31(5):614-619. doi:10.1097/EDE.0000000000001231 PubMedGoogle Scholar
15.
Jo  B, Stuart  EA.  On the use of propensity scores in principal causal effect estimation.   Stat Med. 2009;28(23):2857-2875. doi:10.1002/sim.3669 PubMedGoogle Scholar
16.
Schwartz  S, Gatto  NM, Campbell  UB.  Extending the sufficient component cause model to describe the Stable Unit Treatment Value Assumption (SUTVA).   Epidemiol Perspect Innov. 2012;9:3. doi:10.1186/1742-5573-9-3 PubMedGoogle Scholar
17.
Schneeweiss  S, Patrick  AR, Stürmer  T,  et al.  Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results.   Med Care. 2007;45(10)(suppl 2):S131-S142. doi:10.1097/MLR.0b013e318070c08e PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    December 2, 2020

    Randomized Trial Replication Using Observational Data for Comparative Effectiveness of Secukinumab and Ustekinumab in Psoriasis: A Study From the British Association of Dermatologists Biologics and Immunomodulators Register

    Author Affiliations
    • 1Centre for Dermatology Research, Salford Royal NHS Foundation Trust, The University of Manchester, Manchester Academic Health Science Centre, National Institute for Health Research (NIHR) Manchester Biomedical Research Centre, Manchester, United Kingdom
    • 2Institute of Translational and Clinical Medicine, Newcastle University Medical School, Newcastle upon Tyne, United Kingdom
    • 3Department of Dermatology, Royal Victoria Infirmary and NIHR Newcastle Biomedical Research Centre, Newcastle Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom
    • 4St John’s Institute of Dermatology, Guy’s and St Thomas’ NHS Foundation Trust, London, United Kingdom
    • 5Versus Arthritis Centre for Epidemiology, The University of Manchester, Manchester, United Kingdom
    JAMA Dermatol. 2021;157(1):66-73. doi:10.1001/jamadermatol.2020.4202
    Key Points

    Question  What is the effectiveness of secukinumab compared with ustekinumab for the treatment of psoriasis in an everyday clinical setting?

    Findings  In this comparative effectiveness research study of 1231 patients receiving either secukinumab or ustekinumab for psoriasis, both drugs had lower treatment effectiveness in a real-world clinical setting than in a trial setting. Secukinumab had superior effectiveness compared with ustekinumab, and the estimate of this relative effect using observational data met regulatory and estimate agreement with trial data.

    Meaning  Results of this study found a gap between the efficacy of biologic therapies in an idealized trial setting and the effectiveness of biologic therapies in the real-world clinical setting in the treatment of psoriasis; however, a target trial emulation approach can provide robust estimates of relative effectiveness that can be used for clinical and regulatory decision-making.

    Abstract

    Importance  Treatments for psoriasis may be less effective in everyday practice than in clinical trials. Emulating a target trial using data from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) can provide treatment effect estimates that are robust and can inform both clinicians and regulatory bodies.

    Objectives  To assess the comparative effectiveness of ustekinumab and secukinumab in patients with psoriasis, and to test whether the relative effectiveness estimate of the CLEAR trial, a randomized clinical trial that compared secukinumab with ustekinumab for psoriasis, can be replicated.

    Design, Setting, and Participants  This comparative effectiveness research study used a target trial emulation approach and was performed between November 2007 and August 2019. Data were obtained from BADBIR, a multicenter longitudinal pharmacovigilance register of patients with moderate to severe psoriasis in the United Kingdom and Republic of Ireland. Participants had chronic plaque psoriasis, were 18 years or older, and had at least 1 record of a Psoriasis Area and Severity Index (PASI) of 12 or higher before their initiation to secukinumab or ustekinumab. Propensity score (PS) 1:1 matched analysis and inverse probability treatment weighted analysis were performed.

    Main Outcomes and Measures  The primary outcomes were the risk ratio (RR) and the risk difference (RD) for achieving PASI of 2 or lower after 12 months of therapy for secukinumab compared with ustekinumab. Methods to account for missing outcome data were complete case analysis, nonresponder imputation, last observation carried forward, inverse probability of censoring weighting, and multiple imputation. Regulatory and estimate agreement metrics were used to benchmark the effect estimates in this study against those in the CLEAR trial.

    Results  A total of 1231 patients were included in the analysis, with 917 receiving ustekinumab and 314 receiving secukinumab. Secukinumab was superior to ustekinumab in all analyses, except under the nonresponder imputation method, in the proportion of participants achieving a PASI of 2 or lower (PS-weighted complete case analysis: RR, 1.28 [95% CI, 1.06-1.55]; RD, 11.9% [1.6-22.1]). All analyses, except for nonresponder imputation, reached regulatory agreement in both PS-matching and PS-weighted analyses.

    Conclusions and Relevance  This comparative effectiveness study found that secukinumab resulted in more patients achieving a PASI of 2 or lower after 12 months of therapy compared with ustekinumab in patients with psoriasis. Target trial emulation in this study resulted in regulatory and estimate agreement with the CLEAR randomized clinical trial; further such studies may help fill the evidence gap when comparing other systemic therapies for psoriasis.

    Introduction

    Evidence of the comparative effectiveness of systemic treatments for psoriasis is important to inform the treatment choice of patients and clinicians. Randomized clinical trials (RCTs) that compare 2 agents are the criterion standard level of evidence for treatment efficacy. However, current RCTs in psoriasis have several limitations. First, little incentive exists in the pharmaceutical industry to perform head-to-head studies unless drug manufacturers are confident that they will be able to demonstrate the superiority of their product. Second, evidence of the comparative efficacy of nonbiologic systemic therapies is lacking.1 Third, RCTs do not fully reflect the outcomes observed in real-world settings.2,3 Comparative effectiveness studies of systemic therapies for psoriasis that use observational data can provide findings to inform clinical evidence in the absence of RCT results. Such data may also help to quantify the efficacy-effectiveness gap, where efficacy is the performance of treatments under ideal conditions and effectiveness is the performance of treatments in a clinical setting, between trial and real-world outcomes if trial data are available.

    Comparative effectiveness studies can be affected by several sources of bias, including, for example, selection, confounding, and immortal time bias. A method to avoid these biases is to explicitly characterize a target trial (ie, the ideal RCT that would be conducted if feasible) to provide a structured approach to guide the analysis and thereby minimize bias.4-6 This approach includes outlining the key aspects of the target trial, such as the eligibility criteria, treatment strategies, assignment procedure, follow-up period, and causal contrasts of interest, and then applying this design as much as possible to the observational data set.

    We performed a benchmarking study to compare the effectiveness of secukinumab (Cosentyx) with that of ustekinumab (Stelara) using data from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) by emulating a pragmatic RCT. The relevant corresponding RCT was the CLEAR (A 52-week, Multicenter, Randomized, Double-blind Study of Subcutaneous Secukinumab to Demonstrate Efficacy as Assessed by Psoriasis Area and Severity Index at 16 Weeks of Treatment Compared to Ustekinumab and to Assess Long-term Safety, Tolerability and Efficacy in Subjects With Moderate to Severe Plaque Psoriasis) trial,7,8 a head-to-head comparison between secukinumab and ustekinumab. Our objective was to test 2 hypotheses: (1) the real-world effectiveness of both therapies in BADBIR was lower than that reported in the CLEAR trial, and (2) the relative effectiveness between the therapies documented in BADBIR in an emulated pragmatic RCT would be similar to their relative efficacy in an explanatory RCT (CLEAR trial). We used the agreement metrics described in an article by Franklin et al6 to evaluate whether we were successful in replicating the relative effect estimates of the CLEAR trial. These metrics were regulatory agreement, which indicates whether the study replicates the direction and statistical significance of the RCT findings; estimate agreement, which indicates whether the study effect estimate lies within the 95% CI of the effect estimate in the RCT; and the magnitude and direction of the differences between the study and RCT results, as measured by the standardized difference.

    Methods

    This comparative effectiveness research study was performed between November 2007 and August 2019. BADBIR was approved in 2007 by the National Health Service Research Ethics Committee North West England. All participants in the registry gave written informed consent. This approval extends to the present study. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Data Source and Eligibility Criteria

    BADBIR is a large registry of patients with moderate to severe psoriasis in the UK and Republic of Ireland who were receiving either a traditional systemic therapy, a new oral small molecule systemic therapy, or a biologic therapy. Established in September 2007 as a pharmacovigilance register, BADBIR contains detailed information on the disease characteristics, severity, and comorbid conditions of patients collected at baseline entry into the register. Subsequently, data, including disease severity and any adverse events (AEs), are collected every 6 months for the first 3 years and then annually thereafter. The study design and baseline characteristics of patients recruited in BADBIR have been reported in detail.9,10 A data cutoff date of August 2019 was used in the present study. Details of the reference RCT, the CLEAR trial, are shown in the eAppendix in the Supplement.

    Individuals eligible for the present study had chronic plaque psoriasis and were 18 years or older. Past use of any systemic therapy was allowed, with the exception of ustekinumab, secukinumab, or other biologic therapies that targeted the interleukin 17 and 23 pathways. Individuals needed to have at least 1 record of a Psoriasis Area and Severity Index (PASI) of 12 or higher prior to the initiation of the biologic under investigation. We restricted the drug initiation to dates on or after September 2013, when both secukinumab and ustekinumab were available to patients in the UK clinical setting, and before September 2018 to allow individuals time to complete 12 months of treatment. We placed no exclusions on the basis of comorbid conditions to maximize external validity.

    Treatment Strategies and Follow-up

    Participants were receiving either ustekinumab or secukinumab. Any concomitant topical therapies were allowed, as were concomitant systemic treatments for psoriasis, which we adjusted for in the data analysis. Dosing strategies were established by the individual clinicians. Follow-up for each participant started on the date that ustekinumab or secukinumab was initiated. Demographic characteristics of each participant were documented on or before this start date. Follow-up ended at the occurrence of death, loss to follow-up, discontinuation of therapy, or 12-month outcome. The reasons for discontinuation of therapy (eg, ineffectiveness, AEs, or remission) were documented by the clinical research team.

    Outcomes

    The primary outcome of this study was the difference in the proportion of participants who achieved a PASI of 2 or lower after 12 months of therapy between the 2 comparator cohorts. We performed sensitivity analyses using PASI 90 (ie, ≥90% improvement in PASI) as an alternative secondary outcome. Justification for choosing these outcomes are provided in the eAppendix in the Supplement. The AEs and serious AEs were coded using the Medical Dictionary for Regulatory Activities (MedDRA) terms and presented in a descriptive analysis.

    Statistical Analysis

    A propensity score (PS) was fitted using multivariable logistic regression to account for potential confounding between the secukinumab and ustekinumab cohorts. Two PS methods were used: (1) PS nearest-neighbor optimal 1:1 matching with a caliper of 0.05 was performed to identify the secukinumab and ustekinumab cohorts in the first analysis, and (2) inverse probability treatment weighting was used to balance the 2 cohorts in the second analysis. Details of the covariates within the PS and the methods used are described in the eAppendix in the Supplement.

    Only eligible individuals with no missing baseline data for the potential confounders were included in the study. In the case of missing outcome data, the outcome was imputed as a nonresponder after 12 months of treatment if the treatment was discontinued before that time point because of ineffectiveness or if an AE for psoriasis flare was entered within 1 month of the cessation of therapy. The outcome was imputed as a responder after 12 months of treatment if the treatment was discontinued because of disease remission.

    To investigate the implication of missing outcome data for the effect estimate, we used nonresponder imputation (NRI), last observation carried forward (LOCF), multiple imputation (MI), and inverse probability of censoring weighting (IPCW) to account for missing outcome data in the separate analyses for the intention-to-treat estimate (ie, analysis of participants according to the original treatment assignment) and the complete case analysis (CCA) for the per-protocol estimate (ie, analysis of participants according to the completion of the assigned treatment). A summary of these methods is presented in the eAppendix in the Supplement.

    Generalized linear models were fitted with a log link for the relative risk ratio (RR) and an identity link for the risk difference (RD). Robust SEs were estimated, and matched sets were accounted for in the PS-matched analysis. We performed a sensitivity analysis to investigate the implication of weight truncation for the PS-weighted analysis. We used the aforementioned metrics (ie, regulatory agreement, estimate agreement, and standardized difference) to benchmark the effect estimates in this study with those in the CLEAR trial7,8 that compared the PASI 90 achievement between secukinumab and ustekinumab.

    All data analyses were performed with Stata, version 16.1 (StataCorp LLC).

    Results

    A total of 1231 patients were included in the analysis, with 917 receiving ustekinumab and 314 receiving secukinumab. Table 1 shows the baseline characteristics of the 2 cohorts, and the patient flow diagram is presented in eFigure 1 in the Supplement. The PASI at 12 months was recorded for 132 participants (42.0%) in the secukinumab cohort and 417 participants (45.5%) in the ustekinumab cohort (eFigure 1 in the Supplement).

    The numbers of participants in each cohort who experienced AEs and serious AEs are presented in eTable 1 in the Supplement, with few serious AEs reported within 12 months in both cohorts. Fewer than 5 AEs in the secukinumab and ustekinumab cohorts were coded with the MedDRA high-level term of fungal infection disorders. No AEs in the secukinumab cohort and 5 AEs in the ustekinumab cohort were coded with MedDRA high-level term of noninfective colitis.

    The PS matching analysis included 622 participants, with 311 participants in each treatment cohort (eFigure 2 in the Supplement). Weight truncation did not improve the precision of the estimate materially and therefore was not implemented (eTable 2 in the Supplement). Both PS matching and PS weighting methods resulted in 2 balanced cohorts (eTables 3 and 4 in the Supplement).

    The RRs (Figure) and RDs (eFigure 3 in the Supplement) that compared the proportion of participants who achieved a PASI of 2 or lower at 12 months in the secukinumab and ustekinumab cohorts, estimated from the generalized linear models using the missing outcome imputation methods, are presented in Table 2. Table 2 also shows the corresponding outcome measures from the CLEAR trial. The proportion of participants who achieved the outcome was lower in this study, regardless of the methods used for confounding adjustment or outcome imputation, compared with the CLEAR trial (Table 2).

    All methods to account for missing outcome data, except NRI in the PS weighting, reached regulatory agreement for RRs and RDs (Table 2). However, a discrepancy was found regarding which method achieved estimate agreement. All methods achieved estimate agreement in the PS-weighted analysis, except for NRI for RD (Table 2). In contrast, CCA (RR, 1.55; 95% CI, 1.19-2.01), NRI (RR, 1.45; 95% CI, 1.04-2.03), IPCW (RR, 1.50; 95% CI, 1.14- 1.96), and MI (RR, 1.44; 95% CI, 1.07-1.94) all overestimated the RR in the PS-matched analysis, violating estimate agreement. The PS-matched analysis performed better in estimation of RD, reaching estimate agreement for all missing outcome imputation methods (CCA: 19.5% [95% CI, 0.1%-31.6%]; NRI: 7.2% [95% CI, 0.4%-13.9%]; IPCW: 18.3% [95% CI, 5.5%-31.1%]; LOCF: 14.0% [95% CI, 5.9%-22.1%]; MI: 16.9% [95% CI, 3.3%-30.6%]) (Table 2). The method that resulted in the lowest standardized difference for RR and RD between this study and the CLEAR trial was LOCF with PS matching (RR, 0.009; RD, −0.003). Secukinumab was superior to ustekinumab in all analyses, except under the nonresponder imputation method, in the proportion of participants achieving a PASI of 2 or lower (PS-weighted complete case analysis: RR, 1.28 [95% CI, 1.06-1.55]; RD, 11.9% [1.6-22.1]). The PS-weighted sensitivity analysis using PASI 90 as an alternative outcome gave similar results (eTable 5 in the Supplement).

    Discussion

    To our knowledge, this study was the first to use a target trial approach in comparative effectiveness research of systemic therapies for psoriasis. Using this method, we showed that a robust analysis of data from BADBIR can yield comparative treatment effect estimates that replicate the estimates reported in an RCT and therefore can be reliable for clinical decision-making.

    We found that NRI was not an appropriate missing outcome analysis method for the observational data. Using NRI was overly conservative and introduced nondifferential misclassification that biased the effect estimate toward the null and underestimated the effect size for each treatment cohort. The LOCF method produced the most precise results, but its estimate was less accurate when using PS weighting than the corresponding estimate in the CLEAR trial, shifting the effect estimate slightly toward the null. In addition, single imputation methods such as NRI and LOCF lead to an underestimation of SEs and uncertainty and may increase the likelihood of finding a false-positive result.11 We found that CCA, MI, and IPCW generated similar effect estimates and width of CIs using PS weighting.

    Insights for Clinical Practice

    Taking the point estimate from CCA under PS weighting, we observed a 17.5% reduction for secukinumab and 15.1% reduction for ustekinumab between efficacy and effectiveness using the 12-month end point (Table 2). Patients and clinicians should, therefore, be informed that the probability of achieving a PASI of 2 or lower or PASI 90 was lower for both secukinumab and ustekinumab cohorts in the real world than the odds reported in the CLEAR trial, and secukinumab had statistically superior effectiveness compared with ustekinumab. When counseling patients about the likely outcome of biologic therapies on the basis of figures from clinical trials, clinicians should include a caveat that the real-world outcome may be around 15% lower than that found in clinical trials.

    We have shown that the target trial method using observational data can lead to robust treatment effect estimates for psoriasis. As a result, we believe that clinicians can interpret other or future comparative effectiveness studies with a target trial framework with confidence and use this information in shared decision-making as an adjunct to data from RCTs.

    Previous studies have shown that drug survival of secukinumab is similar to that of ustekinumab at year 1 when only the discontinuation because of ineffectiveness is taken into account.12 Thus, the results of this study suggest that drug survival alone cannot be a sensitive proxy for effectiveness.13

    Assumptions

    An important bias to consider is that knowledge of the findings from the corresponding CLEAR trial may have altered the current study. To further test the robustness of this process, the method should be repeated using BADBIR data to conduct a comparative effectiveness study of drugs with corresponding, ongoing RCTs. Similar to other benchmarking studies, this study assumed that trial participation had no direct impact and that no unmeasured confounding was associated with the impact of trial participation.14

    There are 2 assumptions with PS methods in observational data. The first is the strong ignorability of sample selection assumption, which assumes that no unobserved confounding leads to the differences in the distribution of potential outcomes for the 2 treatments.15 This assumption is unlikely to be completely true, but given that the effect estimate in this study closely replicated that in the CLEAR trial, it is unlikely that any unmeasured confounding has important implications. The second assumption is the stable unit treatment value assumption, which assumes that the treatment has the same implication for an individual regardless of how the individual came to be treated and that the implication of the treatment for the individual is independent of the treatment of other individuals.16 This assumption is associated with whether the study results can be generalized beyond the study population, and our ability to replicate the RR from the CLEAR trial suggests that this second assumption may also hold true.

    This calibration study is a first step in demonstrating that real-world evidence can be analyzed in a way that tests other hypotheses of the comparative effectiveness of psoriasis treatments. We encourage researchers to test the robustness of this approach by conducting benchmarking replication studies using other psoriasis registries and a target trial framework. Although this approach cannot and should not take the place of the criterion standard of head-to-head biologics RCT, it has the potential to further such trials and may help fill the comparative effectiveness gap for systemic treatments of psoriasis. Such replication studies may provide more timely but similarly robust information to inform clinical decision-making.

    Strengths and Limitations

    We were able to perform this study because of the detailed baseline participant data captured in BADBIR, including important covariates from demographic data, disease severity, and specific site involvement, comorbid conditions, and concomitant treatments. Having such data allowed us to balance the 2 cohorts to minimize confounding. The target trial approach also necessitated a new user active-comparator study design, a robust design that leads to balanced patient baseline covariates between 2 treatment comparators.17 Unlike analyses that used routinely collected health care or insurance claims data, the present study faced little potential misclassification in the diagnosis of psoriasis and the specific biologic therapy the patient received. This analysis would not be possible if we used routinely collected health care or insurance claims data because their sources generally do not record disease severity along with commencement or subsequent claims of a drug.

    The main limitation pertains to the amount of missing outcome data, with approximately 45% of the cohort having a recorded PASI at 12 months available for analysis (eFigures 1 and 2 in the Supplement). We believe that it is reassuring that the CCA and the MI analyses in the PS-weighted analysis using the entire cohort, which estimated the mean treatment effect in the whole population, gave RRs similar to those in the CLEAR RCT. This finding suggests that this missing mechanism of PASI at 12 months was either missing completely at random or missing at random; both situations can be accounted for using the analyses in this study. The other possibility was that the PASI was missing not at random but was missing similarly in both treatment cohorts and thus not exerting a substantial differential implication for the estimate of relative treatment effect. Because BADBIR does not require recording of data at 16 weeks, we were unable to use the primary end point of PASI 90 at week 16 in the CLEAR trial as the benchmark effect estimate. Although the AEs were reported herein for completeness, the comparative safety of secukinumab and ustekinumab should not be interpreted from this analysis; rather, an analysis that considers the potential confounders specific to the AE should be undertaken in a larger cohort.

    Conclusions

    In this comparative effectiveness research study, treatment with secukinumab resulted in a higher proportion of patients reaching a PASI of 2 or lower after 12 months of therapy compared with the ustekinumab cohort in the BADBIR population, but an efficacy-effectiveness gap was found for both treatments. A target trial emulation approach can be used to perform comparative effectiveness studies in BADBIR with a high degree of agreement with a corresponding RCT for relative effectiveness of therapies for psoriasis. We suggest the use of this target trial framework in future comparative effectiveness studies that use data from BADBIR. Such research also should consider using PS weighting for confounding adjustment and either the CCA, MI, or IPCW method for imputing missing outcome data.

    Back to top
    Article Information

    Accepted for Publication: September 6, 2020.

    Published Online: December 2, 2020. doi:10.1001/jamadermatol.2020.4202

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Yiu ZZN et al. JAMA Dermatology.

    Corresponding Author: Zenas Z. N. Yiu, PhD, Centre for Dermatology Research, Salford Royal NHS Foundation Trust, The University of Manchester, Manchester Science Park, Pencroft Way, 1st Floor Bright Bldg, Manchester, Greater Lancashire M15 6GZ, United Kingdom (zenas.yiu@manchester.ac.uk).

    Author Contributions: Dr Yiu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Yiu, Lunt, Griffiths, Warren.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Yiu, Mason.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Yiu, Lunt.

    Obtained funding: Yiu, Griffiths, Warren.

    Administrative, technical, or material support: Lunt.

    Supervision: Smith, Lunt, Griffiths, Warren.

    Conflict of Interest Disclosures: Dr Mason reported receiving personal fees from Janssen, LEO Pharma, Lilly, and Novartis outside the submitted work. Dr Reynolds reported receiving lecture fees from AbbVie, Almirall, Celgene, Janssen, and UCB Pharma Ltd outside the submitted work; receiving grants from and serving on the medical advisory board of Novartis; serving on the medical advisory board of Almirall; institutional support form AnaptysBio; and being a National Institute for Health Research (NIHR) senior investigator. Dr Smith reported receiving other from AbbVie, Novartis, Pfizer, Sanofi, Boehringer Ingelheim, and Sobi; grants from a Medical Research Council–funded stratified medicine consortium with multiple industry partners; and grants from IMI (Horizon 2020)-funded European consortium with multiple industry partners during the conduct of the study. Dr Griffiths reported receiving grants from AbbVie, Almirall, and Amgen; personal fees from Bristol Myers Squibb; and grants and personal fees from Janssen, LEO Pharma, Lilly, Novartis, Sandoz, and UCB Pharma Ltd during the conduct of the study as well as being an NIHR emeritus senior investigator. Dr Warren reported receiving grants from AbbVie, Almirall, Amgen, Celgene, Janssen, Lilly, LEO Pharma, Novartis, Pfizer, and UCB Pharma Ltd during the conduct of the study, as well as personal fees from AbbVie, Almirall, Amgen, Arena, Avillion, Bristol Myers Squibb, Boehringer Ingelheim, Celgene, Janssen, LEO Pharma, Lilly, Novartis, Pfizer, Sanofi, and UCB Pharma Ltd outside the submitted work. No other disclosures were reported.

    Funding/Support: This research was supported in part by the NIHR Manchester Biomedical Research Centre and the NIHR Newcastle Biomedical Research Centre. Dr Yiu was funded by an NIHR Academic Clinical Lectureship through The University of Manchester. Drs Griffiths and Warren were funded in part by grant MR/L011808/1 from the Medical Research Council. The study group British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) is coordinated by The University of Manchester and funded by the British Association of Dermatologists (BAD). The BAD receives income from AbbVie, Almirall, Amgen, Celgene, Janssen, LEO Pharma, Lilly, Novartis, Samsung Bioepis, Sandoz Hexal AG, and UCB Pharma Ltd for providing pharmacovigilance services. This income finances a separate contract between the BAD and The University of Manchester that coordinate BADBIR. BADBIR acknowledges the support of the NIHR through the clinical research networks and the contribution made by clinical research associates who continue to facilitate recruitment into the registry.

    Role of the Funder/Sponsor: All decisions concerning analysis, interpretation, and publication are made independently of any industry contribution. All relevant information regarding serious adverse events mentioned in this publication has been reported to the appropriate company as per the contractual agreements/standard operating procedures. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    The BADBIR Study Group: Jonathan N. W. N. Barker, MD (King’s College London); Marilyn Benham (British Association of Dermatologists); Fiona Browne, MB, BCh (Crumlin Children’s Hospital); A. David Burden, MD (University of Glasgow); Ian Evans, MSc (The University of Manchester); Christopher E. M. Griffiths, MD (The University of Manchester); Sagair Hussain, PhD (British Association of Dermatologists); Brian Kirby, MD (St Vincent’s University Hospital); C. Elise Kleyn, MD (The University of Manchester); Philip Laws, MBChB (The Leeds Teaching Hospitals NHS Trust); Linda Lawson, MSc (Aberdeen University); Kayleigh J. Mason, PhD (The University of Manchester); Kathleen McElhone, PhD (The University of Manchester); Teena Mackenzie, RN, BSc (Oxford University Hospitals); Tess McPherson, MD (University of Oxford); Ruth Murphy, PhD (Nottingham University Hospitals); Anthony Ormerod, MD (Aberdeen University); Caroline Owen, MD (East Lancashire Hospital NHS Trust); Eleanor Pearson (Norfolk & Norwich University Hospitals NHS Foundation Trust); Nick J. Reynolds, MD (Newcastle University); Josh Richards, PhD (Patient Representative); Catherine H. Smith, MD (King’s College London); and Richard B. Warren, PhD (The University of Manchester).

    Disclaimer: The views expressed herein are those of the authors and do not reflect the official policy or position of the NHS, the NIHR, or the Department of Health.

    Additional Contributions: We thank all of the patient participants in BADBIR, without whom we would not be able to carry out the project. The substantial contribution of the BADBIR team to the administration of the project has been vital, and we especially acknowledge the pharmacovigilance team for their tireless work: Kathleen McElhone, PhD; Shamila Irshad, BSc; Victoria Wilde, BSc; and Saliha Tahir, BSc. We thank Sagair Hussain, PhD, for his support as the clinical project manager of BADBIR. We are grateful to the members of the Data Monitoring Committee (Robert Chalmers, MBChB; Carsten Flohr, PhD, chair; David Prieto-Merino, PhD; and Richard Weller, MD) and the BADBIR Steering Committee (Jonathan Barker, MD; Marilyn Benham, CEO of BAD; David Burden, MD, chair; Christopher Griffiths, MD, chief investigator; Ian Evans, BSc; Sagair Hussain, PhD; Brian Kirby, MD; Linda Lawson; Kathleen McElhone, PhD; Tess Macpherson, PhD; Kayleigh Mason, PhD; Ruth Murphy, PhD; Anthony Ormerod, MD; Caroline Owen, MBChB; Nick Reynolds, MD, chair of Research Committee; Catherine Smith, MD; and Richard Warren, PhD). These individuals received no additional compensation, outside of their usual salary, for their contributions. We acknowledge the enthusiastic collaboration of all of the dermatologists and specialist nurses in the United Kingdom and the Republic of Ireland who provided the data.

    References
    1.
    Sbidian  E, Chaimani  A, Afach  S,  et al.  Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis.   Cochrane Database Syst Rev. 2020;1:CD011535. doi:10.1002/14651858.CD011535.pub3 PubMedGoogle Scholar
    2.
    Mason  KJ, Barker  JNWN, Smith  CH,  et al; BADBIR Study Group.  Comparison of drug discontinuation, effectiveness, and safety between clinical trial eligible and ineligible patients in BADBIR.   JAMA Dermatol. 2018;154(5):581-588. doi:10.1001/jamadermatol.2018.0183 PubMedGoogle Scholar
    3.
    Yiu  ZZN, Mason  KJ, Barker  JNWN,  et al; BADBIR Study Group.  A standardization approach to compare treatment safety and effectiveness outcomes between clinical trials and real-world populations in psoriasis.   Br J Dermatol. 2019;181(6):1265-1271. doi:10.1111/bjd.17849 PubMedGoogle Scholar
    4.
    Hernán  MA, Robins  JM.  Using big data to emulate a target trial when a randomized trial is not available.   Am J Epidemiol. 2016;183(8):758-764. doi:10.1093/aje/kwv254 PubMedGoogle Scholar
    5.
    Franklin  JM, Schneeweiss  S.  When and how can real world data analyses substitute for randomized controlled trials?   Clin Pharmacol Ther. 2017;102(6):924-933. doi:10.1002/cpt.857 PubMedGoogle Scholar
    6.
    Franklin  JM, Pawar  A, Martin  D,  et al.  Nonrandomized real-world evidence to support regulatory decision making: process for a randomized trial replication project.   Clin Pharmacol Ther. 2020;107(4):817-826. doi:10.1002/cpt.1633PubMedGoogle Scholar
    7.
    Thaçi  D, Blauvelt  A, Reich  K,  et al.  Secukinumab is superior to ustekinumab in clearing skin of subjects with moderate to severe plaque psoriasis: CLEAR, a randomized controlled trial.   J Am Acad Dermatol. 2015;73(3):400-409. doi:10.1016/j.jaad.2015.05.013 PubMedGoogle Scholar
    8.
    Blauvelt  A, Reich  K, Tsai  TF,  et al.  Secukinumab is superior to ustekinumab in clearing skin of subjects with moderate-to-severe plaque psoriasis up to 1 year: results from the CLEAR study.   J Am Acad Dermatol. 2017;76(1):60-69.e9. doi:10.1016/j.jaad.2016.08.008PubMedGoogle Scholar
    9.
    Iskandar  IY, Ashcroft  DM, Warren  RB,  et al.  Demographics and disease characteristics of patients with psoriasis enrolled in the British Association of Dermatologists Biologic Interventions Register.   Br J Dermatol. 2015;173(2):510-518. doi:10.1111/bjd.13908 PubMedGoogle Scholar
    10.
    Burden  AD, Warren  RB, Kleyn  CE,  et al; BADBIR Study Group.  The British Association of Dermatologists’ Biologic Interventions Register (BADBIR): design, methodology and objectives.   Br J Dermatol. 2012;166(3):545-554. doi:10.1111/j.1365-2133.2012.10835.x PubMedGoogle Scholar
    11.
    Mavridis  D, Salanti  G, Furukawa  TA, Cipriani  A, Chaimani  A, White  IR.  Allowing for uncertainty due to missing and LOCF imputed outcomes in meta-analysis.   Stat Med. 2019;38(5):720-737. doi:10.1002/sim.8009 PubMedGoogle Scholar
    12.
    Yiu  ZZN, Mason  KJ, Hampton  PJ,  et al; BADBIR Study Group.  Drug survival of adalimumab, ustekinumab and secukinumab in patients with psoriasis: a prospective cohort study from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR).   Br J Dermatol. 2020;183(2):294-302. doi:10.1111/bjd.18981 PubMedGoogle Scholar
    13.
    Dávila-Seijo  P, García-Doval  I.  Drug survival analysis is not a good method for assessing the safety or effectiveness of systemic therapies in psoriasis.   Actas Dermosifiliogr. 2017;108(1):3-5. doi:10.1016/j.adengl.2016.11.001 PubMedGoogle Scholar
    14.
    Dahabreh  IJ, Robins  JM, Hernán  MA.  Benchmarking observational methods by comparing randomized trials and their emulations.   Epidemiology. 2020;31(5):614-619. doi:10.1097/EDE.0000000000001231 PubMedGoogle Scholar
    15.
    Jo  B, Stuart  EA.  On the use of propensity scores in principal causal effect estimation.   Stat Med. 2009;28(23):2857-2875. doi:10.1002/sim.3669 PubMedGoogle Scholar
    16.
    Schwartz  S, Gatto  NM, Campbell  UB.  Extending the sufficient component cause model to describe the Stable Unit Treatment Value Assumption (SUTVA).   Epidemiol Perspect Innov. 2012;9:3. doi:10.1186/1742-5573-9-3 PubMedGoogle Scholar
    17.
    Schneeweiss  S, Patrick  AR, Stürmer  T,  et al.  Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results.   Med Care. 2007;45(10)(suppl 2):S131-S142. doi:10.1097/MLR.0b013e318070c08e PubMedGoogle Scholar
    ×