Sensitivity was 89.1% (95% CI, 68.58%-92.18%). Specificity was 71.1% (95% CI, 56.9%-82.9%). Diagnostic accuracy was 79.6%. BCC indicates basal cell carcinoma; FN, false negative; FP, false positive; ML, melanocytic lesion; MM, melanoma; NML, nonmelanocytic lesion; SCCIS, squamous cell carcinoma in situ; TN, true negative; and TP, true positive.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Jain M, Pulijal SV, Rajadhyaksha M, Halpern AC, Gonzalez S. Evaluation of Bedside Diagnostic Accuracy, Learning Curve, and Challenges for a Novice Reflectance Confocal Microscopy Reader for Skin Cancer Detection In Vivo. JAMA Dermatol. 2018;154(8):962–965. doi:10.1001/jamadermatol.2018.1668
Reflectance confocal microscopy (RCM) imaging achieves high sensitivity and specificity for the diagnosis of skin cancers.1-5 Of note, these data were mostly derived from blinded analyses performed in a store-and-forward manner by an expert RCM reader on images acquired under a research protocol.1-4 Few studies have explored bedside RCM reader accuracy in the framework of the real-time patient visit.6 Recently, RCM was granted category I Current Procedural Terminology reimbursement codes in the United States, and it is progressing toward clinical use. However, there is a limited pool of experienced RCM readers. Formalized learning approaches to teach a novice reader to interpret RCM images are needed. In this study, we assessed the learning curve, diagnostic accuracy, and pitfalls associated with a novice RCM reader at the bedside.
This study was conducted in a tertiary academic skin cancer center, Memorial Sloan Kettering Cancer Center, New York. The institutional review board of Memorial Sloan Kettering Cancer Center approved the study. Because this study was done in a clinical setting, there was no patient consent for the bedside diagnosis. However, a retrospective protocol, which does not require consent, was used to evaluate the false cases. Over a period of 16 months (August 2015 through December 2016), a novice RCM reader (M.J., a pathologist) imaged 100 consecutive dermoscopically equivocal lesions with RCM at the bedside and made a diagnosis on the basis of RCM findings in a clinical setting using wide-probe and handheld RCM devices (Vivascope, Caliber ID).
Diagnoses made on the basis of histopathologic findings were compared with those made on the basis of RCM to calculate the proportion of true-positive and false-positive findings and diagnostic accuracy with exact 95% CIs. Diagnostic accuracy of RCM conducted on the first 48 lesions (2 of 50 lesions were rejected because of a lack of RCM diagnosis) was compared with the diagnostic accuracy of RCM conducted on the subsequent 50 lesions. After an interval of more than 90 days, the novice RCM reader reevaluated lesions associated with false-positive or false-negative findings, and these lesions were also independently assessed by an RCM expert. The findings of the novice’s reevaluation were compared with those of her original bedside diagnosis, the expert’s assessment, and histopathologic findings.
Of the 98 lesions analyzed, 46 of 98 (47%) were malignant: 26 of 46 (57%) were basal cell carcinoma, 17 of 46 (37%) were melanomas, and 3 of 46 (7%) were squamous cell carcinoma. The remaining 52 of 98 lesions (53%) were benign: 29 of 52 (56%) were melanocytic (common acquired nevi, dysplastic nevi, Spitz, and atypical lesions) and 23 of 52 (44%) were nonmelanocytic lesions. A correct classification was rendered for 78 of 98 lesions (80%; 95% CI, 69%-86%). This included a correct classification of 41 of 46 malignant lesions (89%): 23 of 26 basal cell carcinomas (85%), 15 of 17 melanomas (88%), and 3 of 3 squamous cell carcinomas (100%). A correct classification was also given for 37 of 52 benign lesions (71%): 20 of 21 nevi (95%), 8 of 8 atypical melanocytic lesions (100%), 2 of 7 actinic keratosis (29%), 2 of 5 lichen planus–like keratosis (40%), 1 of 5 acanthomas (25%), 2 of 2 fibrous papules (100%), 1 of 2 dermal fibrosis (50%), and 1 of 2 inflamed follicular cysts (50%) (Figure). Misdiagnosis was made on the basis of RCM findings for 20 of 98 lesions (20%), including false-negative findings for 5 of 20 lesions (25%) and false-positive findings for 15 of 20 lesions (75%). The overall diagnostic accuracy was 80%, with sensitivity and specificity of 89% (95% CI, 69%-92%) and 71% (95% CI, 57%-83%), respectively. The diagnostic accuracy was 79% for the first 48 cases and 80% for the last 50 cases (2-sided P = .91). Reevaluation of these lesions by the novice RCM reader resulted in a correct diagnosis in 15 of 20 lesions (75%); evaluation by the expert RCM reader resulted in a correct diagnosis in 17 of 20 cases (85%). These data, including the potential reasons for misdiagnosis, are shown in the Table.
In this study, the novice RCM reader attained a sensitivity of 89% and a specificity of 71% in routine clinical practice within a short time interval without extensive training. In most cases, misdiagnosis was likely related to the novice’s lack of formal training in and inexperience with performing RCM readings. Another major cause of misdiagnosis was the lack of cellular specificity associated with RCM, which makes it difficult to differentiate basal cell carcinoma tumor nodules from keratosis and even normal skin components, such as hair follicles and epidermis that mimic basal cell carcinoma or to differentiate pagetoid melanocytes from Langerhans cells; this difficulty led to false diagnoses of basal cell carcinoma and melanoma, respectively. Furthermore, 60% of the lesions assessed with the handheld RCM device had a false diagnosis because of a small field of view. Limitations of this study include that it was a single-center, single-observer study with a modest sample size. Because of this, the results may not be broadly generalizable, because the novice reader is a pathologist with an in-depth knowledge of morphology. Studies involving a larger data set are needed. This study highlights a potential approach for formal larger-scale training to expand the existing pool of RCM readers, which is a crucial step for the integration of RCM in clinics to improve skin cancer diagnosis and management.
Accepted for Publication: April 23, 2018.
Corresponding Author: Manu Jain, MD, Dermatology Service, Department of Medicine, Memorial Sloan-Kettering Cancer Center, 16 E 60th St, New York, NY 10022 (firstname.lastname@example.org).
Published Online: July 11, 2018. doi:10.1001/jamadermatol.2018.1668
Author Contributions: Drs Jain and Pulijal had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Jain, Pulijal, Gonzalez.
Drafting of the manuscript: Jain, Pulijal.
Critical revision of the manuscript for important intellectual content: Pulijal, Rajadhyaksha, Halperm, Gonzalez.
Statistical analysis: Pulijal.
Obtained funding: Rajadhyaksha.
Administrative, technical, or material support: Jain, Pulijal, Rajadhyaksha, Halperm.
Supervision: Jain, Halperm, Gonzalez.
Conflict of Interest Disclosures: Dr Rajadhyaksha reported being a former employee of and owning equity in Caliber Imaging and Diagnostics (formerly Lucid Inc). The VivaScope is the commercial version of an original laboratory prototype that was developed by Dr Rajadhyaksha when he was at Wellman Laboratories of Photomedicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA. Drs Halpern and Gonzalez reported having served on the scientific advisory board of Caliber Imaging and Diagnostics. No other disclosures were reported.
Funding/Support: This study was supported in part by grant R01CA199673 from the National Institutes of Health (NIH) and the National Cancer Institute (NCI) and in part by cancer center support grant P30 CA008748 from NIH/NCI to Memorial Sloan-Kettering Cancer Center.
Role of the Funder/Sponsor: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: Melissa Pulitzer, MD, Department of Dermatopathology, Memorial Sloan-Kettering Cancer Center, provided critical review of hematoxylin-eosin images used in the figures; Stephen W. Dusza, DrPH, performed statistical analysis; and Alon Scope, MD, and Patricia Myskowski, MD, provided critical review of the manuscript. No compensation was provided for any of the contributors.
Create a personal account or sign in to: