Comparison of Diagnostic and Management Sensitivity to Melanoma Between Dermatologists and MelaFind: A Pilot Study

In 2011, malignant melanoma (MM) was estimated to be the fifth most commonly diagnosed cancer in the United States, with 70 230 new MM diagnoses and 8790 MM deaths estimated that year.¹ Early detection of MM is critically important. In this pilot study, we evaluate a handheld imaging device called MelaFind (Mela Sciences Inc), which acts as a noninvasive guide to help dermatologists determine whether they should biopsy a pigmented lesion. Our objectives were to estimate MM management accuracy between dermatologists and MelaFind and to estimate diagnostic sensitivity and specificity to MM among dermatologists. We also assess physician reasoning for performing biopsies of pigmented lesions that prove to be melanomas.

The protocol and consent process were approved through the BRANY (Biomedical Research Alliance of New York) institutional review board. In this cross-sectional reader study, 39 dermatologists were compared with MelaFind in evaluations of 23 MMs and 24 benign pigmented lesions. The cases were selected from a repository of lesions amassed during an acquisition study conducted by MELA Sciences Inc for the US Food and Drug Administration. In the acquisition study, MelaFind was applied to the lesions and produced 1 of 2 suggested courses: “consider biopsy: use clinical judgment” or “consider follow-up: use clinical judgment.” Prior to biopsy of the lesion, photographs of the lesion were taken from 21 inches away (distance image), 8 inches away (close-up image), and dermoscopically. Lesions were biopsied in toto and evaluated by a panel of dermatopathologists who were unaware of the MelaFind recommendations.

Forty-seven lesions (23 MMs and 24 nonmelanomas) were randomly selected from the repository for raters to evaluate. Raters viewed the images and a detailed case history for each lesion but were unaware of the MelaFind recommendations. The raters were asked a series of questions regarding each case, including whether they would biopsy the lesion, their reason for recommending biopsy, and whether they thought it was a melanoma.

Uncertainty in individual estimates was quantified using 1-sided mid- P exact 95% lower confidence bounds (LCBs) for sensitivity and 2-sided mid- P exact 95% confidence intervals (CIs) for specificity.² Average biopsy sensitivity was estimated, and a 2-sided 95% CI was constructed based on the Student [sic] t distribution.

Estimated biopsy sensitivity was 22 of 23 (0.96; 95% LCB, 0.83) for MelaFind, and ranged from 0.48 to 1.00 across study dermatologists. Average biopsy sensitivity of study dermatologists among the 23 melanomas was 0.80 (95% CI, 0.72-0.87). Estimated biopsy specificity was 2 of 24 (0.08; 95% CI, 0.01-0.25) for MelaFind, and ranged from 0.04 to 0.71 across study dermatologists. Average biopsy specificity of study dermatologists was 0.43.

Diagnostic sensitivities and specificities for MM among dermatologists was 0.51 and 0.71, respectively.

For MMs, the most common dermatologist rationale for performing a biopsy was “concern that this lesion is a melanoma” (49%); the second most common was “concern about an aggressive melanocytic lesion other than melanoma” (35%). Other reasons were “I am not concerned about this lesion, but the patient has expressed concern” (8%), and “concern about a nonmelanoma skin cancer” (6%).

In our study, MelaFind performed with a high sensitivity but a low specificity in recommending biopsy for melanomas.

The biopsy specificity for dermatologists in our study (43%) is much higher than the 3.7% specificity found for dermatologists in the study by Monheit et al.³ This disparity in specificity between the 2 studies is likely owing to the larger sample size and higher ratio of nonmelanomas to melanomas in the study by Monheit et al.³

In our study, 2 raters had 100% biopsy sensitivity. Their specificities, however, were low, at 12.5% and 8.3%, respectively. These results indicate that higher biopsy sensitivity is associated with lower specificity for clinically atypical pigmented skin lesions for both dermatologists and MelaFind.

Limitations of our study include that it was internet and image based. Thus, tactile evaluation of the lesions was not possible. In addition, the dermatologists completing our study represent a convenience sample who expressed interest in MelaFind, which could have introduced a selection bias. A final limitation is that our study had a small sample size.

In our study, MelaFind appears to be a very sensitive tool to guide dermatologists in biopsying suspect pigmented lesions. However, users need to be aware that MelaFind, like dermatologists, trades a high sensitivity for a lower specificity, thus resulting in biopsy recommendations for many benign lesions. These findings suggest that MelaFind could be useful for dermatologists. A larger reader study is currently under way to confirm these results.

Correspondence: Dr Chen, Department of Dermatology, Emory University, 101 Woodruff Cir, Atlanta, GA 30322 (schen2@emory.edu).

Accepted for Publication: February 25, 2012.

Author Contributions: Dr Chen had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Gutkowicz-Krusin, Veledar, and Toledano. Analysis and interpretation of data: Wells, Gutkowicz-Krusin, Veledar, Toledano, and Chen. Drafting of the manuscript: Wells. Critical revision of the manuscript for important intellectual content: Wells, Gutkowicz-Krusin, Veledar, Toledano, and Chen. Statistical analysis: Gutkowicz-Krusin, Veledar, and Toledano. Obtained funding: Chen. Administrative, technical, and material support: Wells. Study supervision: Gutkowicz-Krusin and Chen.