A, First screen of the ISS application. Sliding a bar at the bottom allows raters to calculate scores for children younger than 8 years. B, Raters can enter scores for erythema and scale for each of the 10 body sites.
Ratings and 95% CIs for scale, erythema, and total scores were significantly correlated (intraclass correlation coefficient [ICC] >0.9, ICC >0.85, ICC >0.9, respectively).
Ratings and 95% CIs for scale, erythema and total scores were significantly correlated (intraclass correlation coefficient [ICC] >0.9, ICC >0.80, ICC >0.9, respectively).
eFigure 1. Anatomical definitions
eTable. Score calculations for (a) individuals <8 years old and (b) individuals ≥8 years old
eFigure 2. Instructions
eFigure 3. Standard photographs
eFigure 4. Descriptors
eFigure 5. First cohort: inter-rater reliabilities for photographic testing
eFigure 6. Second cohort: inter-rater reliabilities for photographic testing
Customize your JAMA Network experience by selecting one or more topics from the list below.
Sun Q, Asch S, Bayart C, et al. Development and Initial Validation of a Novel System to Assess Ichthyosis Severity. JAMA Dermatol. 2022;158(4):359–365. doi:10.1001/jamadermatol.2021.5917
Is it possible to develop a reliable system for clinicians to assess global ichthyosis severity?
In this qualitative study, a comprehensive scoring system was created to assess global ichthyosis severity in adults and children. A free, user-friendly mobile application was also created to facilitate ease of use in clinical settings.
The study results suggest that the Ichthyosis Scoring System has significant applications for clinical practice and research, can facilitate more accurate assessment of the complete burden of ichthyosis to improve medical decision-making and monitor for changes after treatment, and addresses one of the unmet needs in the field of rare genodermatoses by providing a reliable system for clinicians to use to assess global ichthyosis severity in clinical trials.
A comprehensive, user-friendly system to assess global ichthyosis disease burden is imperative to improving the care of patients with ichthyosis, identifying appropriate participants for clinical trials, and quantifying treatment outcomes. To our knowledge, there is currently no validated scale to objectively and systematically measure ichthyosis severity across the entire body.
To create and evaluate a comprehensive and user-friendly instrument to measure total body ichthyosis severity in adults and children.
Design, Setting, Participants
In this qualitative study, ichthyosis experts participated in the content development of the Ichthyosis Scoring System (ISS). The body was divided into 10 regions, and Likert scales (0-4) were created to quantify scale and erythema, with extensive descriptors and photographic standards. An 83-image teaching set was created from photographs of participants with ichthyosis. Two cohorts of dermatologists (11 total) independently scored all test photographs twice to evaluate interrater and intrarater reliabilities. Participants were enrolled worldwide from referral centers and patient advocacy groups. Participants of all ages, races, and ethnicities were included in the creation of ISS, and dermatologists with varying experience and areas of expertise participated as raters to evaluate the ISS. The study was conducted from 2019 to 2021, and the data were analyzed in 2021.
Main Outcomes and Measures
Intraclass correlation coefficients determined overall reliabilities.
Across both cohorts of 11 dermatologists in total, the intraclass correlation coefficients for total, scale and erythema scores were greater than 0.90 (95% CI, 0.77-0.97), greater than 0.91 (95% CI, 0.79-0.98), and greater than 0.88 (95% CI, 0.72-0.97), respectively. Most body sites exhibited moderate to good interrater reliabilities for scale and erythema. Intrarater reliabilities were good to excellent.
Conclusions and Relevance
The results of this qualitative study demonstrate reproducibility and suggest that the ISS is a reliable system to measure global ichthyosis severity in adults and children.
Ichthyoses are a clinically and etiologically heterogeneous group of disorders defined by localized and/or generalized scaling with varying degrees of erythema. Affected individuals experience substantial disease burden, including mobility-limiting palmoplantar keratoderma, pruritus, heat intolerance, and recurrent skin infections, among others. Qualitative grading of ichthyosis is crucial to helping dermatologists select appropriate treatments in clinical practice, and the multiple randomized clinical trials underway require reliable assessments of ichthyosis severity to identify appropriate participants to receive therapy and quantify treatment outcomes.
To our knowledge, there is no scale to measure ichthyosis severity across the entire body. Currently, only 2 visual ichthyosis scales have undergone rigorous evaluation.1,2 The Congenital Ichthyoses Severity Index lacks detailed written descriptors for each severity level and does not represent the full spectrum of common ichthyosis phenotypes.2 The more recently developed scale, the Visual Index for Ichthyosis Severity (VIIS), is limited to 4 body sites that do not account for regional variation in ichthyosis severity.1 Other severity indices include the Bodemer ichthyosis score and the Ichthyosis Area Severity Index (IASI).3,4 Many of these instruments do not focus on body sites of functional and psychosocial significance to patients themselves.
There is a demand for a more comprehensive and user-friendly system to measure ichthyosis severity in adults and children. To address this need, we created the Ichthyosis Scoring System (ISS). Modeled after the VIIS and IASI, the ISS includes extensive scale descriptors to more accurately reflect the phenotypic variability of the ichthyoses using terminology tailored to the US Food and Drug Administration recommendations that the scale include clear, nonoverlapping, and noncomparative categories. In addition to generating a global score, the ISS also captures severity at sites that have substantial psychosocial and functional relevance to patients regarding their perceived severity of disease, such as the face, palms, and soles. Additionally, unlike prior scoring systems, the ISS recognizes that children, who bear different proportions of body regions, should be scored differently than adults. In this article, we report the development and initial validation of the ISS.
Two ichthyosis experts (A.S.P. and K.A.C.) led the central efforts to define ISS content. Members of the Foundation for Ichthyosis and Related Skin Types Medical and Scientific Advisory Board (S.A., C.B., S.J.B., L.B., A.L.B., J.J.D., P.F., T.F., A.L., B.N., I.P., J.T., and M.L.W.) provided further expert input. This study was approved by the Yale University human investigation committee and was consistent with the Declaration of Helsinki guidelines. Written informed consent and assent were obtained from all participants 7 years and older and from parents for those younger than 18 years. Participants were genetically confirmed for ichthyosis subtype.
The need for a holistic measure of disease severity and severity tracking at individual sites led us to establish the ISS to capture information at 10 body regions: scalp, face, neck, torso (chest, back, abdomen, and groin), elbows, palms, upper extremities (excluding palms and elbows), knees, soles, and lower extremities (excluding knees and soles) (eFigure 1 in the Supplement). Each body site was assessed separately for 2 characteristics most commonly shared among congenital ichthyoses: scale and erythema. Scale and erythema were measured by 5-point Likert scales (0-4) based on increasing clinical severity, with half scores allowed (eg, 0, 0.5, 1, 1.5).
Next, from the National Registry of Ichthyosis and Related Disorders data set, we selected photographic standards for scale and erythema that represented all severity levels. An analysis of more than 1400 individuals worldwide who were enrolled in the registry revealed that more than 90% fell into 3 major clinical phenotypes: scale-predominant (eg, lamellar ichthyosis), erythrodermic (eg, congenital ichthyosiform erythroderma) and erythrokeratoderma (eg, epidermolytic ichthyosis). We aimed to create photographic standards that represented these major phenotypes. All photographs had been taken by professional photographers at the Foundation for Ichthyosis and Related Skin Types family conferences to ensure uniform focus, magnification, positioning, lighting, and background.
For the scale characteristic, photographic standards were selected for 5 body sites (elbows, knees, palms, soles, and torso) to address regional variation in scale morphology. The torso photographic standards can be applied to all other body regions not named previously. We selected 56 images to serve as photographic standards for raters to refer to as a guide when scoring ichthyosis severity. A separate set of 83 images not used in creation of photographic standards served as test photographs that raters scored during the evaluation stage. All photographs were approved by 2 ichthyosis experts (A.S.P. and K.A.C.) and were only selected if the image was of high quality (eg, in focus, well lit, and without obstructions of the area). These images represented the cardinal clinical phenotypes, scale-predominant (19 [23%]), erythrodermic (29 [35%]) and erythrokeratoderma (35 [42%]), while balancing age (45.8% were 18 years and older; 54.2% were younger than 18 years) and sex (67.5% male, 32.5% female) (Table). The photographs represented the full spectrum of mild, moderate, and severe ichthyosis. For the erythema characteristic, a single set of standards that represented all severity levels was also created. While the ISS included more skin images from individuals of other ethnicities and races, including Asian, Black, and Hispanic populations (35 [42.2%]), compared with prior severity scoring systems, we were unable to comprehensively account for all races and ethnicities.
After creating photographic standards, we then created written standard descriptions representing each severity level for scale and erythema. Two experts (A.S.P. and K.A.C.) met to define these descriptors, ensuring that the morphological descriptors adequately captured the range of erythema and scale features among the ichthyoses and adhered to US Food and Drug Administration recommendations. Considering that scale on the palms and soles is quite different from that on other body regions, we created 2 separate sets of written scale descriptors: standards for body regions other than palms and soles and standards for palms and soles only.
Body surface area involvement is also critical for quantifying baseline clinical disease burden and subsequent treatment efficacy. Considering that each body region contributes uniquely to the total surface area, the rule of nines was used to assign each body site a constant value. These values differed for individuals younger than 8 years and those 8 years and older. The final score was calculated by first multiplying the sum of scale (0-4) and erythema (0-4) by the multiplier for each body region and then summing the scores for all 10 regions (eTable in the Supplement). For the scale and erythema subscores, we allowed half scores within the 0 to 4 range. The final overall score was rounded to 1 decimal place to capture serial improvement. When there was differential severity within a body site, raters were instructed to average the different subscores over that site and round the final overall score to 1 decimal place. Raters were also instructed to select the worst intensity of erythema within a body region because erythema can often be obscured by scale. The final score ranged from 0.0 to 8.0 (eTable in the Supplement).
Initial content validation was achieved through expert consensus on the characteristic features of ichthyosis severity, morphologic descriptors that defined the various severity levels for scale and erythema, and interpretation of scores.
Intraclass correlation coefficients (ICCs) were used to determine the overall reliability of the instrument. There are 2 types of ICCs: ICCs for absolute agreement and ICCs for consistency. The former is more rigorous and measures the absolute agreement among raters; the latter assesses whether scores are correlated in an additive manner.5 In this article, we reported ICCs for agreement given that it is a more rigorous standard.
In consultation with the biostatistical team, we determined that with an α of .05 and a power of 0.89, a test set of 83 photographs would permit detection of an overall ICC of at least 0.7. The ISS evaluation was performed through scoring of test photographs, and 11 pediatric and adult dermatologists with varying levels of experience and areas of expertise participated as raters. The first cohort of 6 dermatologists, who did not receive prior formal training in ISS scoring, were each provided with instructions, a score sheet, and a printed booklet of high-resolution photographs of standards, test photographs, and accompanying descriptors (eFigures 2-4 in the Supplement). They independently scored scale and erythema across 83 photographs. Photographs were resent to the same dermatologists 4 weeks later to determine intrarater reliability. The 4-week delay before retest was chosen to reduce rater recall of previous scoring.
A second cohort of 5 dermatologists were formally trained in how to properly use the ISS before scoring test photographs. Each watched a training video and engaged in a scoring practice session with answers and feedback provided at the end. Each rater then received the same materials that the first cohort of raters received. Four weeks later, the package was resent to the same raters to determine intrarater reliability.
Interrater reliability was calculated for both rounds of scoring, and ICCs were estimated using the 2-way random model. Intrarater and total score ICCs were evaluated using the 1-way random and 2-way random models, respectively. Any ICC values of less than 0.5 indicated poor, 0.5 to 0.75 moderate, 0.75 to 0.9 good, and more than 0.9 excellent reliability.5 Analyses were conducted in Stata, version 16.0 (StataCorp).
We created a mobile application on iOS (Apple) and Android (Google) to facilitate ease of use of the instrument. Information can be found at https://apps.apple.com/al/app/iss-calculator/id1531086115 and https://play.google.com/store/apps/details?id=com.app.isscalculator&hl=en_US&gl=US, respectively (Figure 1).
Of the 83 test photographs, 18 (21.7%) were of Asian individuals (phototype II-IV), 9 (10.8%) of Black individuals (phototype V-VI), 8 (9.6%) of Hispanic individuals (phototype IV-V), and 48 (57.8%) of White individuals (phototype I-II). Thirty-eight participants (45.8%) were 18 years and older and 45 (54.2%) were younger than 18 years. The Table displays more detailed characteristics of the test photographs.
The ISS demonstrated excellent overall interrater reliability. In the first cohort, ICC agreement for the total score (sum of the scale and erythema scores) was 0.903 (95% CI, 0.770-0.974). The ICC agreement for the ISS scale (ISS-S) and ISS erythema (ISS-E) scores across all raters were 0.911 (95% CI, 0.789-0.976) and 0.882 (95% CI, 0.723-0.968), respectively. Descriptive statistics are summarized in Figure 2.
When analyzed according to body region, all interrater reliabilities for scale were in the moderate to good category, with ICCs ranging from 0.503 (95% CI, 0.237-0.812) to 0.898 (95% CI, 0.775-0.963), except for elbows in round 1 and lower extremities (excluding knees and soles) in round 2. Interrater reliabilities for erythema were moderate to excellent regardless of site, with ICCs ranging from 0.587 (95% CI, 0.322-0.854) to 0.902 (95% CI, 0.643-0.997). Descriptive statistics for ICC agreement values during both rounds of testing are shown in eFigure 5 in the Supplement.
In the second cohort, the ICC agreement for total ISS score was 0.904 (95% CI, 0.764-0.974). The ICC agreement for ISS-S and ISS-E scores across all raters was 0.902 (95% CI, 0.761-0.973) and 0.880 (95% CI, 0.729-0.967), respectively. Descriptive statistics are summarized in Figure 2. Interrater reliabilities for scale among all body sites were moderate to excellent, with ICCs ranging from 0.55 to -0.941, except for elbows and lower extremities in round 2. Interrater reliabilities for erythema were moderate to excellent, with ICCs ranging from 0.53 to 0.907, except for knees, palms, and soles in round 2. Descriptive statistics for ICC agreement values during both rounds of testing are included in eFigure 6 in the Supplement.
The first cohort exhibited excellent intrarater reliability, with an ICC of 0.954 (95% CI, 0.918-0.974), 0.956 (95% CI, 0.925-0.974), and 0.913 (95% CI, 0.855-0.949) for the ISS, ISS-S, and ISS-E scores, respectively (Figure 3). When analyzed according to body region, all intrarater reliabilities were moderate to excellent, with ICCs ranging from 0.642 to 0.952 for scale and 0.742 to 0.915 for erythema.
In the second cohort, intrarater reliabilities were good to excellent, with ICCs of 0.909 (95% CI, 0.840-0.949), 0.927 (95% CI, 0.871-0.959), and 0.822 (95% CI, 0.700-0.898) for the ISS, ISS-S, and ISS-E scores, respectively (Figure 3). Intrarater reliabilities were moderate to good for scale, with ICCs ranging from 0.625 to 0.886. For erythema, with the exception of soles, intrarater reliabilities for the other 9 body sites were moderate to good, with ICCs ranging from 0.502 to 0.888.
In this qualitative study, we have created a comprehensive scoring system for dermatologists with content validity and reliability. The ISS assesses global ichthyosis severity in adults and children during clinical practice and clinical trials. There have been attempts to grade scale thickness and erythema with various instruments, such as optical coherence tomography and reflectance spectroscopy, respectively.6-9 However, we aimed to design an instrument that could be used regardless of clinical resources in settings ranging from medical missions to academic centers. Medical treatment of patients with ichthyosis requires a systematic method to evaluate global severity, but current grading systems provide limited use regarding whole-body disease burden.
The ISS capitalized on the strengths of the VIIS and IASI. It included detailed descriptors adapted from the VIIS while leveraging the comprehensive assessment of IASI. In developing the ISS, it became evident that subtypes, such as the erythrokeratoderma phenotypes associated with KRT1/KRT10 variants, can exhibit scale that is mostly limited to the palms and soles, sites not tested in other severity indices. Furthermore, to our knowledge, the face, a highly visible site that is often erythematous in Netherton syndrome and congenital ichthyosiform erythroderma, has also never been tested in other severity indices. Thus, the ISS is able to inform clinicians about the potential functional and psychosocial effect of ichthyosis on patients and individually assesses severity across all 10 body sites. Expert consensus was reached on the interpretation of the final scores as follows: 0.0 as normal, a score of more than 0.0 and 2.0 or less as mild, a score of more than 2.0 and 5.0 or less as moderate, and a score of more than 5.0 and 8.0 Or less as severe.
Individual regions were assessed for interrater and intrarater correlations, ensuring that even individual scores could be used with accurate assessment. Thickening and inflammation of palms and soles can be debilitating to patients, and the palms and soles display unique scale morphology, including yellow thickening, desquamative scale and fissuring. As a result, these regions deserved their own set of scale descriptors and were included among the 10 sites. In the first cohort, soles exhibited the highest ICCs for scale during both rounds of testing. Palms had the second and third highest ICCs for scale in round 1 and 2, respectively. Among the second cohort of raters, soles also exhibited the highest ICC during the first round of testing. Palms had the third highest ICC for scale in round 1. The high interrater agreement likely reflects the prominence of distinct scale across these sites and adequate descriptors.
Among both groups of raters, elbows and lower extremities were among the most challenging sites to assess, with lower ICCs for these 2 body regions (eFigures 5 and 6 in the Supplement). Scale on the elbows may have been difficult to assess because natural elbow wrinkles may obscure small adherent scale, especially in elderly individuals with increased skin laxity. The lower extremities saw poor to moderate interrater agreement. A probable explanation is that the lower extremities represent one of the largest surface areas, often with different scale severity at the thigh vs knee, vs popliteal area vs lower leg vs dorsal feet; older patients also often have superimposed xerosis or stasis, which may challenge accurate scoring. In the future, training sessions should focus on these body sites.
In the first group of raters, erythema ICCs for both rounds were all within the moderate to excellent range (eFigure 5 in the Supplement). In the second cohort, erythema ICCs for both rounds were mostly within the moderate to excellent range (eFigure 6 in the Supplement). Erythema on the soles exhibited particularly low agreement among the raters in round 2, possibly because thick scale on the soles can obscure evaluation of underlying erythema. Despite a few sites with poor erythema ICCs in this second group, the overall combined erythema ICC was very good, further supporting the scoring system’s ability to assess total body ichthyosis severity.
Overall, the ISS demonstrated excellent agreement among physicians. This is notable given the heterogeneity of participants analyzed. The interrater reliabilities for total score and aggregate scale scores were near perfect (ICC, >0.9). Interrater reliabilities for aggregate erythema scores were also high (ICC, >0.85). Individual evaluators were consistent in their assessments of overall severity even after 1 month, a period reflective of the timeframe for routine clinic follow-up. This consistency over time is essential for assessing ichthyosis improvement and therapeutic efficacy.
Although a few regions showed poor ICCs, the substantial interrater reliabilities for most body sites and the near-perfect agreement for total score suggest that the ISS is a reliable tool for scoring ichthyosis severity. Substantial interrater reliabilities can be achieved without extensive training, which is an advantage of this scoring system and reflection of its ease of use. Furthermore, our study results overall are reproducible, further supporting the validity and reliability of this scoring system.
The ISS has several relevant applications for dermatologists. In clinical practice, the scale has the potential to improve assessment of a patient’s global ichthyosis burden and monitor for disease progression posttreatment, therefore improving medical decision-making. In research and clinical trials, the ISS allows for better stratification of patients into treatment arms and can more accurately capture improvement of ichthyosis with new therapeutics. It is even possible to track severity and progression of disease at individual sites of interest. The estimated time needed to complete an assessment ranges from 2 to 7 minutes depending on the number of body sites evaluated, and final scores can be easily calculated through a free mobile phone application.
As with all tools that score erythema, there is a need to capture more photographs of ichthyosis in individuals from racial and ethnic minority groups and further refine the erythema score and training to assure applicability for all skin pigmentation types. Another limitation is the lack of testing in live settings, which was curtailed by the SARS-CoV-2 pandemic. In our experience in generating the VIIS, in-person evaluation demonstrated better ICC scores than validation using images alone. In-person evaluations would also allow for the assessment of the groin and buttocks, sites for which test photographs were not available. Future assessments, ideally in the clinic setting, will facilitate further testing for construct and convergent validity, as well as for responsiveness to clinical change in the face of intervention. Finally, the ISS and its mobile application could be simplified to allow patients and families to track the severity of their ichthyosis between clinic visits, including responses to treatment.
The COVID-19 pandemic has fundamentally transformed the landscape of dermatologic care. As teledermatology continues to expand, clinicians will increasingly need to rely on photographs to determine disease severity. Photography has also become a standard tool in clinical trials, often serving as the criterion standard reference when multiple sites enroll patients. Therefore, there is tremendous potential for the ISS to help improve medical decision-making for patients with ichthyosis when only photography is available.
Measurement matters. Assessment of disease severity informs therapeutic plan and prognosis, but because ichthyosis is a rare disorder, many patients fail to receive proper evaluation or treatment during their lifetime. The results of this qualitative study support the ISS as a comprehensive and efficient tool for assessing global ichthyosis severity and specific sites of interest. A mobile application permits rapid score calculation in clinical settings. The robust overall interrater and intrarater reliabilities, coupled with the scale’s ease of use and content validity, suggest that the ISS may be used with accuracy in clinical as well as research settings and potentially adopted globally as the criterion standard ichthyosis severity index.
Accepted for Publication: December 16, 2021.
Published Online: February 16, 2022. doi:10.1001/jamadermatol.2021.5917
Corresponding Authors: Qisi Sun, MD, Yale University School of Medicine, 333 Cedar St, New Haven, CT 06510 (email@example.com); Keith A. Choate, MD, PhD, Yale University School of Medicine, 333 Cedar St, New Haven, CT (firstname.lastname@example.org).
Author Contributions: Drs Sun and Choate had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Sun, Paller, Choate.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Sun, Choate.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Sun, Gan, Deng.
Obtained funding: Sun, Choate.
Administrative, technical, or material support: Sun, Asch, Nelson, Teng, Choate.
Supervision: Sun, Bayliss, Deng, Paller, Choate.
Conflict of Interest Disclosures: Dr Bruckner reported grants from Mayne Pharma outside the submitted work. Dr Funk reported research support and personal fees from Palvella Therapeutics and research support from AbbVie outside the submitted work. Dr Choate reported research support from Regnereon, Mayne, and Anaptsys and personal fees from Lilly, AbbVie, and Janssen outside the submitted work. No other disclosures were reported.
Disclaimer: Dr Bruckner is a Deputy Editor of JAMA Dermatology, but she was not involved in any of the decisions regarding review of the manuscript or its acceptance.