A literature review was conducted and meetings were held to determine candidate items, followed by conceptualization of items into domains. Delphi surveys and live meetings were conducted to finalize the core domain set. IDEOM indicates International Dermatology Outcome Measures; OMERACT, Outcome Measures in Rheumatology.11
The inner ring (left) includes the 6 core domains that are considered required to measure in every psoriasis clinical trial. The middle ring includes 1 domain (skin manifestations) with 5 subdomains that are not required but may be important depending on the study objectives. The outer ring (right) includes 4 domains that are considered the “research agenda,” or items that may be important but need further study. BSA indicates body surface area.
eTable. Item Distillation and Domain Development Summary
Customize your JAMA Network experience by selecting one or more topics from the list below.
Callis Duffin K, Merola JF, Christensen R, et al. Identifying a Core Domain Set to Assess Psoriasis in Clinical Trials. JAMA Dermatol. 2018;154(10):1137–1144. doi:10.1001/jamadermatol.2018.1165
What should be measured in every psoriasis clinical trial?
In this Delphi survey, patients, health care professionals, and other stakeholders agreed that these 6 domains should be measured in all psoriasis clinical trials: skin manifestations (subdomains of primary skin manifestations, skin lesion location, palmoplantar psoriasis, and scalp psoriasis), psoriasis and psoriatic arthritis symptoms, investigator global assessment, patient global assessment, health-related quality of life, and treatment satisfaction.
These 6 core domains should be considered mandatory measures in all future psoriasis interventional trials, although capture of additional domains may be important based on the research question.
There is no consensus on which domains should be measured or which instruments should be used in clinical trials for psoriasis therapies.
To achieve international consensus among psoriasis stakeholders on a core set of domains that should be measured in all psoriasis clinical trials.
Design, Setting, and Participants
Literature review, pre-Delphi survey exercises, nominal group discussions, and audience voting at 4 stakeholder meetings were used to develop candidate domains for 2 rounds of a Delphi survey. Stakeholders were patients or advocates of patients with psoriasis and health care professionals (HCPs) with expertise in psoriasis, including physicians, scientists, advocacy organization representatives, and regulators. Delphi surveys were conducted electronically from October through December 2015 and between September and October 2016. Stakeholder discussions with audience response voting were conducted at live meetings in the United States, Canada, and Italy from January 2013 to December 2016 to refine and ratify the core set of domains.
Main Outcomes and Measures
Two rounds of an electronic Delphi survey were used to determine consensus. A domain was considered “core” (ie, should be measured in all trials) if a threshold consensus of at least 70% was met in both patient and HCP groups. Domains meeting consensus in only 1 group were considered to be important but were not required to be measured in all trials (“middle ring”). These domains were included for rerating in round 2. Domains that did not meet consensus in either of the groups (“outer ring”) were considered to be of uncertain importance and were placed in the research agenda.
In round 1 of the Delphi survey, 107 HCPs and 14 patients participated. Most HCPs (72 [67%]) were dermatologists between 46 and 64 years old (71 [66%]), white (78 [73%]), and male (75 [70%]) from North America (60 [57%]) and Europe (34 [32%]).There were 10 pharmaceutical industry clinical or health economic scientists, 3 advocacy organization representatives, 2 regulatory agency representatives, and 5 “other.” In the second round, 77 HCPs and 15 patients participated. Of the 20 candidate domains, the following 6 met consensus as core domains: skin manifestations, psoriasis and psoriatic arthritis symptoms, health-related quality of life, investigator global assessment, patient global assessment, and treatment satisfaction. Secondary skin manifestations as well as nail, inverse, genital, and guttate psoriasis were classified as important but not mandatory. Psoriatic arthritis signs, work productivity or participation, economic impact (direct and indirect cost), and cardiovascular disease comprised the research agenda.
Conclusions and Relevance
This iterative Delphi process yielded international consensus among professional and patient stakeholders on 6 domains that should be measured in all clinical trials for psoriasis. Future International Dermatology Outcome Measures group efforts will focus on development of a core outcome measurement set for psoriasis trials.
Psoriasis is a complex inflammatory disease that primarily affects the skin, nails, and joints. Although advances in the understanding of the pathophysiology of psoriasis and randomized clinical trials (RCTs) have yielded efficacious therapies for moderate to severe plaque psoriasis, substantial gaps remain in our ability to consistently measure and compare the contribution of therapies to the many aspects of psoriatic disease, including the perspective of patients. The instruments used as primary end points in psoriasis RCTs, such as the Psoriasis Area and Severity Index, focus on features of plaques located on the trunk and body, but they do not consistently or adequately assess commonly overlapping phenotypes, such as inverse, genital, scalp, or palmoplantar psoriasis. Moreover, to date, existing instruments have not been rigorously validated to assess these phenotypes or their contributions to quality of life.1
The International Dermatology Outcome Measures (IDEOM) group was established in 2013 to address the need for standardized patient-centered clinical outcome measures to assess disease course and response to treatments of dermatologic diseases in clinical trials and clinical practice.2,3 International Dermatology Outcome Measures was modeled on the Outcome Measures in Rheumatology (OMERACT) initiative, which has led the development of core domain sets and core outcome sets (COSs) for many rheumatologic diseases.4
A COS is a consensus-driven minimum set of outcomes of a specific condition that must be measured and reported in a clinical trial.5 Core outcome sets enable comparison and combination of results across trials and can reduce selective outcome reporting bias and increase clinical interpretability.6,7 Many additional outcomes can be measured, but core measures are considered mandatory.
In dermatology, several groups support COS development, including Core Outcome Measures in Effectiveness Trials,8 Cochrane Skin Group-Core Outcome Set Initiative (CSG-COUSIN),9 and Harmonizing Outcome Measures in Eczema (HOME).10 In 2013, it was determined that the efforts of IDEOM would be focused initially on developing a core domain set and ultimately on a COS for psoriasis clinical trials.
The first step in creating a COS is to determine the core domain set, defined as “the minimum set of Domains and Sub-domains necessary to adequately cover all Core Areas, i.e. fully measure all relevant concepts of a specific health condition within a specific setting.”11(p34) In other words, the core domain set includes a minimum and required set of concepts to be measured, and the COS is the minimum list of instruments.
The objective of the present study was to achieve international consensus on a core domain set for psoriasis clinical trials among stakeholders, including physicians, scientists, pharmaceutical industry representatives, advocacy organizations, payers, regulators, and patients.
Our approach consisted broadly of 2 steps: (1) item generation using literature review, live meetings with nominal group discussions, pre-Delphi electronic surveys of stakeholders, and work group conference calls with stakeholders and patients; and (2) two rounds of an electronic Delphi survey as well as face-to-face discussions after each round. The study was approved and documentation of patient consent was waived by the University of Utah, Salt Lake City, institutional review board.
At the first meeting of the IDEOM stakeholders in 2013, attendees were presented with a literature review of existing domains and measures as well as of gaps in outcome measures in psoriasis studies.3 Subsequent nominal group discussions with stakeholders at our first 2 annual meetings and an electronic pre-Delphi survey generated a total of 193 candidate items (defined as items and constructs that can or should be measured in psoriasis trials). The psoriasis work group, which comprised patients and IDEOM board members, then distilled these items into a conceptual model of candidate domains using The OMERACT Handbook, which is the published methods of OMERACT.11 A second pre-Delphi and discussion at the third annual meeting3 were used to further refine the conceptual model prior to launching the final electronic Delphi intended to achieve consensus on a core domain set for psoriasis clinical trials (Figure 1).
The IDEOM psoriasis stakeholders participated in all aspects of this initiative. For the 2 Delphi survey rounds, stakeholders were divided into the following 2 groups: (1) health care professionals (HCPs), identified as physicians, scientists, and clinical and health economic scientists from the pharmaceutical industry, advocacy organizations, and US government agencies; and (2) the patient group, which also included patient research partners from psoriasis organizations, including IDEOM, Group for Research and Assessment of Psoriasis and Psoriatic Arthritis, and the National Psoriasis Foundation. The patient survey was adapted to patient-friendly language and included photographs when necessary to facilitate understanding of the various clinical features of psoriasis.
Two rounds of the electronic Delphi survey were distributed using Research Electronic Data Capture (commonly known as REDCap), a secure Health Insurance Portability and Accountability Act–compliant web-based application widely used for building and managing online surveys and databases. Round 1 was conducted between October and December 2015. For each of the 20 candidate domains, we asked, “Should [domain] be measured in all psoriasis clinical trials?” Respondents were asked to rate items on a scale of 1 (strongly agree) to 5 (strongly disagree). Between rounds, an in-person stakeholder meeting was held to discuss the results. Round 2 was then conducted between September and October 2016. On the basis of the in-person meeting feedback, the question structure was revised for clarity to the following: “[Domain] should be considered: (1) inner core, required to be measured in all psoriasis clinical trials; (2) middle ring, important but not required to be measured in all psoriasis clinical trials, or (3) outer ring, research agenda (may be important, need more evidence).”
The statistical analysis used for the results of round 1 was adapted from the OMERACT “onion model,” represented by a central core of essential domains (inner core), a middle ring of domains of undetermined significance, and an outer ring for those domains requiring further investigation.12 In the IDEOM-adapted analysis, consensus for inclusion in the inner core was met when at least 70% of both the HCP and patient groups responded strongly agree or agree in round 1, with a lower 95% CI limit of 0.50 or greater. A domain was included in the middle ring when only 1 of the groups achieved at least 70% agreement. Inclusion of a domain in the outer ring occurred when neither group met the 70% threshold agreement. In round 2, inner core consensus was defined as at least 70% agreement among the combined weighted proportion of respondents. Patient and HCP samples were weighted equally (ie, combined P = [(patient P + HCP) × P/2}) to compensate for the potential imbalance between the number of HCP and patient respondents. All analyses were conducted using SPSS, version 20 (SPSS Inc), and SAS, version 9.4, Service Pack 4 (SAS Institute Inc).
Details of the 2 electronic pre-Delphi surveys and meeting discussion were previously published and are summarized in the eTable in the Supplement.3,12 Following the pre-Delphi survey, the psoriasis work groups organized the items into 21 candidate domains (eTable column 2 in the Supplement). A second pre-Delphi survey was sent to stakeholders, and feedback was also obtained at the second annual IDEOM meeting in 2014. At the third annual IDEOM meeting in 2015, additional feedback was obtained using anonymous voting with an audience response system. These processes yielded the 20 candidate domains used in the 2 rounds of the Delphi survey.13
All IDEOM stakeholders were invited to participate in both surveys. The demographic and clinical characteristics of the participants are shown in Table 1. In round 1 of the survey, 107 of 303 persons (35%) in the HCP group and 14 of 31 persons (45%) in the patient group participated. Most HCPs were dermatologists, with 72 (67%) in round 1 and 62 of 77 (80%) in round 2. There were 11 pharmaceutical industry clinical or health economic scientists, 3 advocacy organization representatives, 2 regulatory agency representatives, and 5 “other” in round 1. Most HCPs in round 1 were between 46 and 64 years old (71 [66%]), white (78 [73%]), and male (75 [70%]) from North America (60 [57%]) and Europe (34 [32%]). The patient population included 5 patients with psoriasis alone (36%), 9 with both psoriasis and psoriatic arthritis (PsA) (64%), and 1 patient advocate.
The domains for which consensus was achieved in both the HCP and patient groups included primary skin manifestations, location of skin lesions, palmoplantar psoriasis, scalp psoriasis, patient-reported symptoms, health-related quality of life, investigator global assessment, patient global assessment, and treatment satisfaction. The domains for which consensus threshold was met only in 1 group included secondary skin manifestations, nail psoriasis, inverse psoriasis, genital psoriasis, guttate psoriasis, PsA symptoms, and cardiovascular disease. The domains for which consensus threshold was not met in either group included PsA signs, direct cost, indirect cost, and work productivity (eTable column 3 in the Supplement).
Following round 1, the fourth annual meeting of stakeholders was held March 2, 2016.14 During this 2-day conference, 88 stakeholders discussed the survey results (Table 2) and provided anonymous feedback. Polling via an audience response system showed that 86 stakeholders (98%) felt that the results presented were consistent with what they would have anticipated. The method was clear to 67 participants (76%), with only 3 participants (3%) indicating that it was somewhat unclear. The majority (59 stakeholders [67%]) agreed with the analysis plan to equally weight the patient and HCP responses, given the smaller number of patient participants.
The stakeholders agreed that domains meeting consensus threshold in both groups in round 1 did not need to be rerated in the second Delphi round. Although secondary skin manifestations met the combined threshold in round 1, only 54 of 107 HCPs (50%) agreed that it should be a core domain. It was suggested that the HCP vote could have been influenced by the lack of validated measures for secondary skin features because most psoriasis assessment instruments do not evaluate fissuring, erosions, or other secondary features. However, the patients noted that these features were highly symptomatic and thus should be measured. When polled, the participants voted to include this domain in the second round of the Delphi survey.
During the meeting, the participants discussed the conceptual model of what constitutes a domain (a construct or concept that should be measured) vs a contextual factor. Per The OMERACT Handbook,15 a contextual factor is defined as a “[v]ariable that is not an outcome of the study, but needs to be recognized (and measured) to understand the study results. This includes potential confounders and effect modifiers.”11(p34) This discussion largely surrounded disease phenotype and location, such as nail psoriasis, palmoplantar psoriasis, scalp psoriasis, inverse psoriasis, genital psoriasis, guttate psoriasis, and cardiovascular comorbidities, because these items could be measured as primary outcomes of a trial or as a contextual factor. When polled, 31 of the 88 stakeholders (35%) agreed that these domains represented contextual factors, whereas 18 (20%) agreed that they represent outcomes. A similar discussion ensued regarding the cardiovascular comorbidities domain, for which 41 stakeholders (47%) agreed that it represented a contextual factor, whereas 13 stakeholders (15%) agreed that it represented an outcome factor. It was also proposed that the work productivity domain should be renamed “participation” to account for the contribution of disease to activities in and out of the work place. However, because work productivity was unanimously placed in the research agenda, it was not included in the second round.
In August 2016, 77 of 307 HCPs (25%) and 15 of 31 patients (48%) participated in round 2. The HCP group comprised 66 physicians (mostly dermatologists), 6 pharmaceutical industry representatives, 1 advocacy organization representative, and 4 “other.” Most were from North America (45 [58%]) and Europe (21 [27%]), with other HCP participants from South American, Africa, and Asia. Patient demographics in round 2 were similar to those in round 1. Results of the round 2 survey are presented in Table 3. For the secondary skin manifestations domain, there was a combined agreement of 41%, and thus this domain was placed into the middle ring. For the PsA symptoms domain, 14 (93%) of the patient group, 52 (68%) of the HCP group, and 66 (73%) of the combined group agreed it should represent a core domain. Following the discussion at the stakeholders meeting held in October 2016, it was agreed that PsA symptoms should be placed in the inner core and that psoriasis and PsA symptoms should be grouped into a single domain.
At the fifth annual IDEOM Stakeholders meeting in May 2017, an overview of the methods, results, and final core set was presented to the participants (Figure 2). Among the 88 stakeholders present, 36 (41%) were physicians (mostly dermatologists), 18 (21%) were patients, 19 (22%) were from industry, and the remainder of the stakeholders had demographic characteristics similar to those in the Delphi surveys. Among the 68 who participated, 64 (94%) voted that they agreed that IDEOM should adopt the core domain set for psoriasis clinical trials.
Development of core domain sets and COSs was initiated in rheumatology by OMERACT more than 20 years ago, leading to an international consensus on core measurement sets for many rheumatologic diseases in RCTs and longitudinal studies. Recognizing that COSs are lacking in psoriasis clinical trials, in 2013, IDEOM engaged an international group of stakeholders in a multistep consensus effort to develop the first core domain set for psoriasis clinical trials. Although IDEOM has not published a road map, we have used the same basic methodologic framework as advocated by OMERACT and by HOME, whose published road map is considered by CSG-COUSIN to be the preferred method for COS development in dermatology.10
To develop a core domain set, relevant concepts to be measured must first be determined through qualitative efforts using expert opinion and, importantly, patient input and then distilled into a manageable conceptual model of candidate domains. First, we chose to narrow the scope to psoriasis clinical trials and to involve all stakeholders, including patients, in all face-to-face meetings, nominal group discussions, teleconferences, and Delphi surveys.
A core domain refers to a construct or concept that is necessary to fully understand the contributions of an intervention to a disease. Put another way, if a core domain is not measured, a critical or highly relevant aspect of the disease would not be assessed. Currently, many industry sponsored pivotal phase 2 and 3 clinical trials do use measures that cover most of the core domains. All industry-sponsored moderate to severe plaque psoriasis trials use instruments that measure primary manifestations of disease (Psoriasis Area and Severity Index, Body Surface Area, or Physician Global Assessment), and many trials also use measures assessing scalp and palmoplantar severity. However, the key primary end-point measures, such as physician or investigator global assessments, vary significantly between trials and many are not validated prior to their use.16 Similarly, many sponsors choose from a number of validated but nonpsoriasis-specific instruments to measure health-related quality of life, making meta-analyses and other comparisons between trials challenging.
Other domains that have been placed in the core, such as PsA symptoms and location of skin lesions, are not consistently measured in clinical trials because no rigorously developed instruments are available. This does not preclude a domain from being placed in the core; it simply means that the instruments will need to be developed. During the consensus process in the present study, patients indicated that “location of lesions” was a very important yet frequently overlooked domain; that is, using traditional measures such as body surface area or erythema to assess psoriasis in certain special locations, such as the face or genitals, often underestimates the severity and the impact of these lesions.
Domains that have been placed in the middle and outer rings remain critical to the development and selection of measures in psoriasis clinical trials. Although these domains may not be considered essential in a psoriasis RCT, they should be strongly considered depending on the therapeutic target, the study hypotheses and objectives, and trial resources available. However, the real challenge is that validated instruments to measure domains such as secondary skin manifestations or guttate psoriasis do not exist, or the existing measures for the domain are simply not feasible to implement. For example, despite patients with psoriasis voicing the importance of assessing joint disease, dermatologists conducting trials are generally uncomfortable assessing PsA with current PsA trial instruments, such as a tender and swollen joint counts.
Cardiometabolic comorbidities is a challenging construct to conceptualize under our model. Cardiovascular disease is an important comorbidity in patients with psoriatic disease, and increasingly more data support that therapies such as methotrexate sodium and many biologic agents may improve cardiovascular outcomes.17 Cardiovascular disease surrogate end points have been an outcome for several recent RCTs, such as the Vascular Inflammation in Psoriasis Trial.18 Cardiovascular disease can be classified in several ways under our model, as a pathophysiologic manifestation, an adverse event, or a contextual factor; pragmatically, cardiovascular disease is unlikely to be the primary outcome of most clinical trials on psoriasis therapeutics and therefore has been classified in the research agenda.
There are several limitations of our consensus efforts that must be noted. It is possible that the IDEOM meetings and Delphi surveys did not fully represent stakeholders adequately, given that the response rates for both rounds of the Delphi survey were less than 30%. The IDEOM group has attempted to reduce bias by recruiting a diverse group of participants to the meetings and surveys, holding meetings in Europe and North America, and not restricting participation. Despite our efforts, not all potential stakeholder groups, such as payers, were represented in the Delphi survey or annual meetings. Thus, IDEOM has engaged payers in other venues to discuss our progress and their needs.19 We recognize that the number of patient participants was also quite low, and therefore the patient perspective may not have been adequately represented. Moving forward, we expect to increase patient participation substantially, as patient involvement will be critical for the qualitative aspects of developing new instruments and evaluating existing instruments for feasibility in clinical trials. We have been fortunate to have representatives from the US Food and Drug Administration attend and present at our meetings, and we will continue to partner with them to develop a COS that aligns with their regulatory needs.
This Delphi survey study identified 6 core domains that should be measured in all psoriasis clinical trials. The next steps of IDEOM will be to develop work groups that use the core domain set as a guide to evaluate the existing instruments as candidates for core measurement sets. This effort will involve systematic literature reviews and application of the consensus-based standards for the selection of a health measurement instruments checklist20 and OMERACT guidance to assess validity, reliability, and, perhaps most important, feasibility. A parallel effort will also be necessary to determine existing gaps within the domains and to organize efforts to begin development on measures that evaluate domains such as location of lesions or palmoplantar psoriasis. It will also be important for those work groups to use the items and possibly consider additional consensus exercises with patients to consider development of psoriasis disease-specific health-related quality of life. As we select and develop measures, it will be important to consider their relevance to clinical practice so that clinical trial data are meaningful to practicing dermatologists, payers, and, especially, patients.
Accepted for Publication: December 29, 2017.
Corresponding Author: Kristina Callis Duffin, MD, MS, Department of Dermatology, University of Utah, 30 North 1900 East 4A330, Salt Lake City, UT 84132 (email@example.com).
Published Online: June 6, 2018. doi:10.1001/jamadermatol.2018.1165
Author Contributions: Drs Callis Duffin and Christensen had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Callis Duffin, Merola, Christensen, Garg, Gottlieb, Armstrong.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Callis Duffin, Gottlieb.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Callis Duffin, Christensen, Armstrong.
Obtained funding: Gottlieb.
Administrative, technical, or material support: Merola, Latella, Garg, Armstrong.
Study supervision: Christensen, Gottlieb.
Initiated IDEOM and the process of selecting domains for psoriasis: Gottlieb.
Conflict of Interest Disclosures: Dr Callis Duffin reported serving as a paid investigator, advisor, or consultant for AbbVie, Amgen, Boehringer Ingelheim, Bristol-Myers Squibb, Celgene, Janssen, Eli Lilly and Company, Novartis, Pfizer, Regeneron, Sienna Biopharmaceuticals, and Stiefel Laboratories. Dr Merola reported serving as a paid consultant for Biogen Inc, AbbVie, Eli Lilly and Company, Novartis, Pfizer, Janssen, UCB, Sumumed, Science 37, Celgene, Sanofi, Regeneron, and GlaxoSmithKline plc; as a paid speaker for AbbVie; as a paid investigator for Biogen Inc, Pfizer, Sanofi, Regeneron, Incyte Corp, and Novartis; and has licensed outcome measures to AbbVie and Eli Lilly and Company. Dr Christensen is employed by The Parker Institute, Bispebjerg and Frederiksberg Hospital, which is supported by a core grant from the Oak Foundation. Dr Garg reported serving as a paid consultant and investigator for AbbVie and as a paid investigator for Merck. Dr Gottlieb reported serving as a paid consultant or advisor for Janssen, Celgene, Bristol-Myers Squibb, Beiersdorf, AbbVie, UCB, Novartis, Incyte Corp, Eli Lilly and Company, Dr Reddy’s Laboratories, Valeant Pharmaceuticals International Inc, Dermira, Allergan, and Sun Pharmaceutical Industries Ltd. She has received research grants from Janssen and Incyte Corp. Dr Armstrong reported serving as a paid investigator and advisor/consultant for AbbVie, Janssen, Novartis, Celgene, Eli Lilly and Company, Pfizer, Regeneron, Sanofi, Modernizing Medicine, and Science 37.
Funding/Support: This study was funded in part by IDEOM, a 501(c)(3) nonprofit organization that is supported by corporate sponsorships from AbbVie, Amgen, Celgene Corp, Dermira, Eli Lilly and Company, Leo Pharma, Novartis, Sun Pharmaceutical Industries Ltd, UCB, Valeant, and Sienna Biopharmaceuticals and by private donors, including Advancing Innovation in Dermatology. REDCap is supported by grant 8UL1TR000105 (formerly UL1RR025764) from the National Center for Advancing Translational Sciences.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; or in the preparation, review, or approval of the manuscript.
Disclaimer: Dr Armstrong is the clinical evidence synopsis editor and an editorial board member of JAMA Dermatology, but she was not involved in any of the decisions regarding review of the manuscript or its acceptance.
Additional Contributions: Bob Wong, MS, PhD, College of Nursing, University of Utah, contributed statistical and technical support, and Ms Amanda Pacia, association manager of IDEOM, provided administrative assistance. We thank the patients and stakeholders of IDEOM who participated in the meetings and the Delphi surveys.