Approach used to assess the potential contribution of intrapatient comparative analysis to the selection of nevi for biopsy, assuming that dermatologists would only consider nevi identified as both morphologically suspicious nevi (MSN) and with the ugly duckling nevi (UDN) for resection.
A, Agreement of clinical vs dermoscopic images among 9 dermatologists using intrapatient comparative analysis (IPCA) with the ugly duckling sign. The dermatologists are represented by the numerals preceded by “E.” The dotted line represents an equal number of lesions with the ugly duckling nevi (UDN) identified in the clinical and dermoscopic images. B, Agreement of clinical vs dermoscopic images among 9 dermatologists using lesion-focused analysis (LFA) to determine morphologically suspicious nevi (MSN). The dermatologists are represented by the numerals preceded by “E.” The dotted line represents an equal number of MSN identified in the clinical and dermoscopic images.
A, Comparison of the percentage of nevi considered for biopsy when the decision is based on lesion-focused analysis (LFA) (nevi considered as morphologically suspicious nevi [MSN]), intrapatient comparative analysis (IPCA) (nevi considered to show the ugly duckling nevi [UDN]), and the combination of the 2 methods at the clinical level (2089 lesions). The dermatologists are represented by the numerals preceded by “E.” B, Comparison of the percentage of nevi considered for biopsy when the decision is based on LFA, IPCA, and the combination of the 2 methods at the dermoscopic level (766 lesions). The dermatologists are represented by the numerals preceded by “E.”
Customize your JAMA Network experience by selecting one or more topics from the list below.
Gaudy-Marqueste C, Wazaefi Y, Bruneu Y, et al. Ugly Duckling Sign as a Major Factor of Efficiency in Melanoma Detection. JAMA Dermatol. 2017;153(4):279–284. doi:10.1001/jamadermatol.2016.5500
What is the effect of intrapatient comparative analysis using the ugly duckling sign on diagnosis of melanoma?
In this experimental study, images of all nevi of 80 patients, including 7 patients with a melanoma, were presented to 9 dermatologists during 2 separate experiments. All melanoma were labeled ugly duckling nevi and access to intrapatient comparative analysis reduced the potential biopsy of nevi by a factor of 6.9.
Intrapatient comparative analysis improves the effectiveness of the diagnosis of melanoma.
Understanding the contribution of the ugly duckling sign (a nevus that is obviously different from the others in a given individual) in intrapatient comparative analysis (IPCA) of nevi may help improve the detection of melanoma.
To assess the agreement of dermatologists on identification of the ugly duckling sign and estimate the contribution of IPCA to the diagnosis of melanoma.
Design, Setting, and Participants
The same 2089 digital images of the nevi of a sample of 80 patients (mean age, 42 years [range, 19-80 years]; 33 men and 47 women), as well as 766 dermoscopic images from a subset of 30 patients (mean age, 40 years [range, 21-75 years]; 12 men and 18 women), were randomly presented to the same 9 dermatologists for blinded assessment from September 22, 2011, to April 1, 2013. The first experiment was designed to mimic an IPCA situation, with images of all nevi of each patient shown to the dermatologists, who were asked to identify ugly duckling nevi (UDN). The second experiment was designed to mimic a lesion-focused analysis to identify morphologically suspicious nevi. Data analysis was conducted from November 1, 2012, to June 1, 2013.
Main Outcomes and Measures
Number of nevi labeled UDN and morphologically suspicious nevi, specificity of lesion-focused analysis and IPCA, and number of nevi identified for biopsy.
Of the 2089 clinical images of nevi from 80 patients (median number of nevi per patient, 26 [range, 8-81]) and 766 dermoscopic images (median number of nevi per patient, 19 [range, 8-81]), all melanomas were labeled UDN and as morphologically suspicious nevi by the 9 dermatologists. The median number of UDN detected per patient was 0.8 among the clinical images of nevi (mean, 1.0; range, 0.48-2.03) and 1.26 among the dermoscopic images (mean, 1.4; range, 1.00-2.06). The propensity to consider more or fewer nevi as having ugly duckling signs was independent of the presentation (clinical or dermoscopic). The agreement among the dermatologists regarding UDN was lower with dermoscopic images (mean pairwise agreement, 0.53 for clinical images and 0.50 for dermoscopic images). The specificity of IPCA was 0.96 for clinical images and 0.95 for dermoscopic images vs 0.88 and 0.85, respectively, for lesion-focused analysis. When both IPCA and lesion-focused analyses were used, the number of nevi considered for biopsy was reduced by a factor of 6.9 compared with lesion-focused analysis alone.
Conclusions and Relevance
Intrapatient comparative analysis is of major importance to the effectiveness of the diagnosis of melanoma. Introducing IPCA using the ugly duckling sign in computer-assisted diagnosis systems would be expected to improve performance.
The ugly duckling describes a nevus that is obviously different from the others in a given individual.1 This term is used in intrapatient comparative analysis (IPCA) of nevi and is widely recognized as a major indicator for suspicion of melanoma both visually and dermoscopically.2,3 We previously demonstrated that the ability of a dermatologist to intuitively perceive a nevus that looks different from others relies on the limited diversity of nevi, which constitutes an individual reference system in each patient.4 A major benefit for training physicians and facilitating self-detection in the at-risk population may come from understanding the robustness, reliability, and diagnostic effect of the IPCA process using the ugly duckling sign in detection of melanoma. Intrapatient comparative analysis has probably always been unconsciously used by dermatologists to support or modulate the interpretation they make from the morphologic analysis of a given lesion. Two cognitive processes are probably at work: a lesion-focused analysis (LFA) to determine if pigmented lesions are morphologically suspicious nevi (MSN) and identifying a nevus with the ugly duckling sign.5 Since LFA and IPCA processes are intrinsically and unconsciously mixed in the physicians’ decision, their respective contribution is not measurable in real-world practice. To estimate the contribution of IPCA to the diagnosis of melanoma, an experimental study with an artificial separation of LFA and IPCA was designed.
The same digital images of the nevi of a sample of patients were randomly presented from September 22, 2011, to April 1, 2013, to the same observers in 2 different experiments designed to reproduce a whole-body skin examination (IPCA) and a morphologic analysis situation (LFA).
The observers were 9 senior dermatologists (C.G.-M., Y.B., R.T., L.T., G.P., J.M., M.-F.A., S.M., and J.-J.G.), all experts in nevi and melanoma and included 5 international leaders from different countries, 3 leaders in the field of dermoscopy, 1 experienced office-based dermoscopy specialist, and 3 senior dermatologists from our research group (some of the 9 dermatologists are in more than 1 category).
Digital images of the nevi of volunteers were collected between November 1, 2009, and March 31, 2011, from the Hôpital La Timone, Marseille, France. Before the study, an independent expert selected a set of clinical images from 80 patients. Images were chosen to represent the variability of nevus phenotypes, not only in terms of the number, size, and morphologic features but also in terms of age and skin type of the patient.4 In a second step, a subset of 766 dermoscopic images from 30 patients was identified from the nevi in the clinical sample. The dermoscopic sample was smaller than the clinical sample because the analysis of dermoscopic images was time consuming. According to French law, the approval of a human participant committee was not mandatory in this setting of an anonymous study with no intervention; hence, institutional review board approval from Aix-Marseille University was not necessary.
The clinical and dermoscopic sets of images were presented separately to the 9 dermatologists during 2 different experiments using a specifically designed software program to generate a straightforward and realistic presentation of nevi.4 There was a 1-year gap between experiments 1 and 2 to reduce memory recall. The experiments were conducted first with the clinical images, then with the dermoscopic images.
In the first experiment, to assess IPCA, digital images of all the nevi of a given patient were shown on a dual-monitor display in the organizing center to each dermatologist, who was asked to identify ugly duckling nevi (UDN), if any. A UDN was defined as a nevus obviously different from the others in a given individual. The same assessment was repeated for each patient; all patients were presented randomly. In the second experiment, to assess LFA for the detection of MSN, all the images in the working set were remotely (by means of a web application) and randomly presented to each dermatologist, who was asked to identify the lesions considered to be MSN (ie, lesions for which he or she would recommend biopsy).
Considering that all melanocytic lesions with the ugly duckling sign are potential melanomas, specificity of IPCA using the ugly duckling sign for the detection of melanoma was calculated as the proportion of nevi that were not labeled as UDN by the dermatologists using the following equation:
Specificity for Melanoma = D/(D + B),
where D is the number of nevi not labeled as UDN and B is the number of nevi that are labeled as UDN.
To estimate the effect of IPCA on the therapeutic decision, we considered that all nevi identified as MSN would be biopsied when IPCA was not available (eg, when a partial skin examination is performed, or images of nevi are transmitted electronically for expert evaluation), whereas when IPCA was available (eg, a whole-body skin examination is performed), only nevi labeled both MSN and UDN would be biopsied. Thus, the ratio of the number of nevi labeled both MSN and UDN to the number of nevi labeled MSN alone represents the proportion of biopsies not performed when considering both the ugly duckling sign and morphologic analysis for the therapeutic decision (Figure 1).
The κ statistic was used for evaluating pairwise agreement about detection of UDN between dermatologists. The comparison of levels of agreement between dermatologists (in terms of κ), as a function of image modality (macroscopic or dermoscopic) was evaluated by means of the Wilcoxon signed rank test for paired observations. Statistical analysis was conducted from November 1, 2012, to June 1, 2013.
A total of 6249 images of nevi were initially collected from 208 volunteers. The sample selected to be presented clinically included 80 individuals (mean age, 42 years [range, 19-80 years]; 33 men and 47 women; 7 with Fitzpatrick skin type I, 33 with type II, 38 with type III, and 2 with type IV), comprising 2089 images (median number of nevi per patient, 26 [range, 8-81]); 7 of these 2089 images were melanoma (in situ and 0.3-, 0.6-, 0.89-, 1.75-, 1.77-, and 2.2-mm thickness). The subsample selected for the presentation of dermoscopic images included 30 of the 80 patients (mean age, 40 years [range, 21-75 years]; 12 men and 18 women; 3 with skin type I, 13 with type II, and 14 with type III), comprising 766 images (median number of nevi per patient, 19 [range, 8-81]); 6 of these 766 images were melanoma (Table).
The analysis of the full set of clinical images required approximately 4 hours (about 7 seconds per image), and analysis of the dermoscopic images required 3 hours (approximately 14 seconds per image). As the nevi in the data set were not biopsied, all patients had 4-year follow-up to assure that new melanoma did not develop from a lesion initially considered as a nevus; 2 of the 80 patients were diagnosed with a new melanoma during the follow-up period. After a detailed analysis, these 2 new melanomas developed either de novo or from a micronevus that was not seen on the clinical image. No melanoma was labeled a nevus in the assessment of the clinical images.
The number of nevi considered to have the ugly duckling sign by the dermatologists in the patient-by-patient experiment was highly variable both in the clinical and dermoscopic images (Figure 2A). The mean number of nevi labeled UDN by the dermatologists was 80 in the clinical images (3.8%) and 42 in the dermoscopic images (5.5%). The median number of UDN detected per patient was 0.8 among the clinical images (mean, 1.0; range, 0.48-2.03), and 1.26 among the dermoscopic images (mean, 1.4; range, 1.00-2.06).
The pairwise agreement between experts (κ) ranged from 0.41 to 0.57 (mean, 0.53) for clinical images and from 0.38 to 0.57 (mean, 0.50) for dermoscopic images (P = .50, Wilcoxon signed rank test). Among the 254 nevi clinically considered by at least 1 dermatologist to be UDN, 54 (21.3%) were considered by at least 5 dermatologists to be UDN and 20 (7.9%) were considered by all the dermatologists to be UDN. Similarly, among the 113 nevi dermoscopically considered by at least 1 dermatologist to be UDN, 32 (28.3%) were considered by at least 5 dermatologists to be UDN and 14 (12.4%) were considered by all the dermatologists to be UDN. The nevi labeled as UDN by the dermatologists who tended to consider few nevi to be UDN were also generally labeled as such by the dermatologists who tended to see more UDN, suggesting a core of consensus on UDN.
The number of nevi considered to be MSN by the 9 dermatologists in the lesion-by-lesion situation was highly variable on both the clinical and dermoscopic images (Figure 2B). The mean number of nevi labeled MSN by the dermatologists was 292 clinically and 132 dermoscopically. The median number of nevi per patient labeled MSN by the dermatologists was 2.2 at the clinical scale (mean, 3.6; range, 1.2-6.3) and was 3.6 at the dermoscopic scale (mean, 1.1; range, 1.7-10.0).
The specificity of IPCA was 0.96 for clinical images and 0.95 for dermoscopic images compared with 0.88 and 0.85, respectively, for LFA. Only 41 of the 292 nevi considered as MSN were also considered UDN (Figure 3). Assuming that, when IPCA is available, only the nevi considered both MSN and UDN are candidates for biopsy, and that when IPCA is not available, all MSN are candidates for biopsy, a whole-body skin examination can potentially reduce the number of nevi removed by a factor of 6.9 compared with LFA alone. As all melanoma from the database were labeled both MSN and UDN, we can estimate that IPCA reduces the number of nevi to biopsy to treat a melanoma by the same factor of 6.9 compared with LFA. Similarly, on dermoscopic examination, IPCA potentially reduced the number of nevi to remove to diagnose a melanoma by a factor 5.3 compared with LFA alone (Figure 3).
This study demonstrates that the identification of UDN by IPCA is a reliable cognitive process. There is good agreement among experts for most UDN, despite variable individual thresholds in the perception of what should be called a UDN. To our knowledge, this study is the first to provide a quantitative estimation of the specificity of IPCA for detection of melanoma. Although all dermatologists have the cognitive ability to perceive a few dominant morphologic subsets of nevi in each individual,4 our study demonstrated that they do not have the same perception threshold for considering that a nevus is different from all others (UDN). The propensity to consider nevi as UDN is likely to be a natural individual cognitive ability, since those who tend to consider more nevi as UDN clinically tend to behave similarly for dermoscopic images. However, it is possible that understanding the intuitive definition of a UDN may be different between observers.
A higher proportion of nevi were considered UDN on dermoscopic examination than on clinical examination, suggesting that the greater number of details in the dermoscopic images provided more opportunity to consider a nevus as a UDN. Intrapatient comparative analysis was more difficult and took much more time (approximately twice as long) for the dermoscopic images than for the clinical ones. Clustering objects in the visual field allowed recognition of objects that did not fit with the others, which is part of the spontaneous human experience in daily life. Conversely, a comparative analysis of the dermoscopic images was an artificial exercise. Our study confirmed the results observed by Scope et al2; namely, that “melanoma are generally apparent as UDs.”2(p63) Even in patients with multiple atypical moles, all the melanomas of our data set were considered by all dermatologists as UDN both clinically and dermoscopically, suggesting that specificity of IPCA using the ugly duckling sign for detecting melanoma was high.
A major result of our study was the experimental demonstration that, when expert dermatologists have access to all nevi of a given individual, they can theoretically decrease the number of nevi biopsied, and the number needed to treat for 1 melanoma, by a factor of 6.9 compared with a decision made on a separate analysis of each nevus alone. It is impossible to determine how physicians’ perceptions work in practice and on what criteria and respective proportion of LFA or IPCA they individually build their final decision. It may well be that physicians’ intuition does not target only lesions that are both MSN and UDN; they may consider a few UDN that are not actually MSN, or may determine that some MSN that are not UDN need to be biopsied. This study demonstrated that dermatologists who cannot access the patient’s entire skin and perform IPCA will not be as effective as they can be; thus, examining only a nevus that concerns the patient, examining a patient who is not undressed, or assessing only the images of suspicious lesions that were transmitted electronically will not achieve the greatest accuracy. Promoting the use of IPCA in the diagnosis of melanoma may improve early detection.
Despite the intention to limit biases, the variability of nevus profiles was greater in the clinical images than in the subset selected for dermoscopic analysis. Although experiments 1 (IPCA) and 2 (LFA) were separated by 1 year, the dermatologists may have recalled certain lesions, especially melanoma. The number of melanomas in the study was low, to closely resemble reality and assess the operational input of IPCA on the specificity and number needed to treat. Another limitation was the inability to determine the sensitivity of IPCA using the ugly duckling sign for detection of melanoma, as the number of melanomas in our series was too small.6 A larger database is required to evaluate the sensitivity of IPCA using the ugly duckling sign.
Including the concept of IPCA using the ugly duckling sign in the education of medical students, the training of practicing physicians, and campaigns targeted to the community could probably improve the accuracy of diagnosis of melanoma. Moreover, our data are important in computer-assisted diagnosis systems and telemedicine, which may have improved accuracy with the use of IPCA.
Accepted for Publication: November 17, 2016.
Corresponding Author: Caroline Gaudy-Marqueste, MD, PhD, Dermatology and Skin Cancer Department, Hôpital Timone, 264 rue Saint Pierre, 13385 Marseille CEDEX 5, France (firstname.lastname@example.org).
Published Online: February 8, 2017. doi:10.1001/jamadermatol.2016.5500
Author Contributions: Drs Fertil and Grob had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Richard, Fertil, Grob.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Wazaefi, Bruneu, Triller, Richard, Fertil, Grob.
Critical revision of the manuscript for important intellectual content: Gaudy-Marqueste, Thomas, Pellacani, Malvehy, Avril, Monestier, Richard, Fertil, Grob.
Statistical analysis: Gaudy-Marqueste, Wazaefi, Fertil, Grob.
Obtained funding: Triller, Fertil.
Administrative, technical, or material support: Wazaefi, Bruneu, Triller, Monestier, Fertil.
Study supervision: Monestier, Fertil, Grob.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported in 2007 and 2012 by grants ANR-07-EMPB-0031 and ANR-12-EMMA-0046 from Agence Nationale de la Recherche (Skinan project).
Role of the Funder/Sponsor: The funding source was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: