Concordance and Reproducibility of Melanoma Staging According to the 7th vs 8th Edition of the AJCC Cancer Staging Manual

IMPORTANCE The recently updated American Joint Committee on Cancer (AJCC) classification of cancer staging, the AJCC Cancer Staging Manual, 8th edition (AJCC 8), includes revisions to definitions of T1a vs T1b or greater. The Melanoma Pathology Study database affords a comparison of pathologists’ concordance and reproducibility in the microstaging of melanoma according to both the existing 7th edition (AJCC 7) and the new AJCC 8. OBJECTIVE To compare AJCC 7 and AJCC 8 to examine whether changes to the definitions of T1a and T1b or greater are associated with changes in concordance and reproducibility. DESIGN, SETTING, AND PARTICIPANTS In this diagnostic study conducted as part of the national Melanoma Pathology Study across US states, 187 pathologists interpreting melanocytic skin lesions in practice completed 4342 independent case interpretations of 116 invasive melanoma cases. A consensus reference diagnosis and participating pathologists’ interpretations were classified into the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis class IV (T1a) or class V ( T1b) using both the AJCC 7 and AJCC 8 criteria. MAIN OUTCOMES AND MEASURES Concordance with consensus reference diagnosis, interobserver reproducibility, and intraobserver reproducibility. RESULTS For T1a diagnoses, participating pathologists’ concordance with the consensus reference diagnosis increased from 44% (95% CI, 41%-48%) to 54% (95% CI, 51%-57%) using AJCC 7 and AJCC 8 criteria, respectively. The concordance for cases of T1b or greater increased from 72% (95% CI, 69%-75%) to 78% (95% CI, 75%-80%). Intraobserver reproducibility of diagnoses also improved, increasing from 59% (95% CI, 56%-63%) to 64% (95% CI, 62%-67%) for T1a invasive melanoma, and from 74% (95% CI, 71%-76%) to 77% (95% CI, 74%-79%) for T1b or greater invasive melanoma cases. CONCLUSIONS AND RELEVANCE Melanoma staging in AJCC 8 shows greater reproducibility and higher concordance with a reference standard. Improved classification of invasive melanoma can be expected after implementation of AJCC 8, suggesting a positive impact on patients. However, despite improvement, concordance and reproducibility remain low. JAMA Network Open. 2018;1(1):e180083. doi:10.1001/jamanetworkopen.2018.0083 Key Points Question Do changes to the American Joint Committee on Cancer (AJCC) cancer staging system for melanoma improve concordance and reproducibility for invasive melanomas? Findings In this diagnostic study, melanoma staging in the AJCC Cancer Staging Manual, 8th edition, showed greater reproducibility and higher concordance with a reference standard than melanoma staging in the AJCC Cancer Staging Manual, 7th edition. Meaning Improved classification of invasive melanoma can be expected after implementation of the AJCC Cancer Staging Manual, 8th edition, suggesting a positive impact on patients. + Invited Commentary Author affiliations and article information are listed at the end of this article. Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2018;1(1):e180083. doi:10.1001/jamanetworkopen.2018.0083 May 18, 2018 1/7 Downloaded From: https://jamanetwork.com/ by a University of California Los Angeles User on 01/03/2021


Introduction
Disease subclassification according to the AJCC Cancer Staging Manual by the American Joint Committee on Cancer (AJCC) is the customary and prevalent mode for stratifying patients with melanoma to estimate prognosis, determine appropriate surgical intervention, and assess eligibility for adjuvant therapies and clinical trials. The process presupposes that pathologists' application of the AJCC histopathological criteria to individual cases of melanoma is accurate and reproducible.
However, in the field of melanoma, there are only limited analyses quantifying the degree of reproducibility of AJCC microstaging between pathology observers. 1 Extensive variability has been noted among pathologists in the diagnosis of invasive melanoma. [2][3][4][5][6][7] One of the largest studies, 2 our previously published Melanoma Pathology Study (M-Path) of 187 US pathologists, found less than 50% agreement between pathologists and a consensus-derived reference diagnosis of T1a invasive melanoma, with improvement to 72% concordance for invasive melanoma T1b or greater. Similarly, M-Path findings revealed only 46% interobserver agreement for T1a invasive melanoma, and 77% agreement for T1b or greater melanomas. 2 The previous study 2 was conceived and executed in the context of the AJCC Cancer Staging Manual, 7th edition (AJCC 7) staging system. Across interpretations at 2 points, pathologists' intraobserver reproducibility reached 63% for T1a melanomas and 83% for T1b or greater melanomas. Given the updated classification in the AJCC Cancer Staging Manual, 8th edition (AJCC 8), particularly with changes in definitions of T1a vs T1b or greater, the M-Path database enables a new comparison of pathologist concordance with a reference standard and reproducibility in the microstaging of melanoma according to both the existing AJCC 7 and the current AJCC 8. 8,9 Briefly, in AJCC 8, the depth for stage T1a is established at 0.8 mm, rather than 1.0 mm, and the presence of ulceration continues to contribute to stage modification, but mitoses do not. In addition, the reporting of Breslow thickness is limited to intervals of tenths of a millimeter rather than hundredths.
We assess whether changes in criteria in the newer AJCC 8 are associated with changes in concordance and reliability, and whether observer interpretations of histological alterations within melanocytic lesions are reliable in the context of the demands of microstaging and its consequences per the AJCC schema.

Study Design
The data used in this diagnostic study are derived from the M-Path study, 2 which was described previously. Practicing pathologists from 10 US states who actively interpreted melanocytic skin biopsy lesions as part of their usual clinical practice and planned to continue practicing for a minimum of 2 subsequent years were invited to participate. This study was approved by the institutional review boards of Dartmouth College, the Fred Hutchinson Cancer Research Center, Oregon Health and Science University, and the University of Washington. Informed consent was obtained from every participating pathologist using an online platform.
Each pathologist was randomized to interpret the same set of melanocytic skin biopsy cases on 2 occasions, at least 8 months apart. The study cases (n = 240) were assembled into 5 sets of 48 cases, each represented by a single glass slide. Each set included the full spectrum of melanocytic skin lesions (eg, from benign to invasive melanoma).
Participating pathologists independently reviewed the same cases using the same glass slides.

Participants entered diagnostic interpretations into an online Melanocytic Pathology Assessment
Tool and Hierarchy for Diagnosis (MPATH-Dx) histology form for each case, choosing from a diverse and comprehensive list of more than 50 diagnostic terms. We asked participants to assume that the single glass slide for each case was representative of the entire lesion and that the margin was scheme. 10 Examples of diagnostic terms for each class and suggested treatment recommendations, provided under the assumption that specimen margins are positive, are depicted in Table 1. Because the AJCC 8 criteria changes only affect MPATH-Dx classes IV (T1a) and V (ՆT1b), this article focuses on the distinction between invasive melanomas exclusively.
Before data collection, a panel of 3 experienced dermatopathologists independently reviewed the hematoxylin-eosin-stained glass slides for each case followed by consensus review using a modified Delphi approach. 11,12 This process was used to develop a consensus diagnosis for each of the M-Path study cases. Only 116 cases of invasive melanoma, as defined by the consensus diagnosis, were considered in this analysis. Three cases included in the original M-Path study as class IV were excluded here because classification was based on a treatment recommendation of wide excision but these cases were assessed as melanocytic lesions of uncertain malignant potential.

Statistical Analysis
For each case, the consensus reference diagnosis and the participating pathologists' interpretations were classified into the MPATH-Dx class IV (T1a) or class V (ՆT1b) using both the AJCC 7 and AJCC 8 criteria. 8 For intraobserver concordance among the 118 participants who interpreted the same glass slides on 2 occasions, we calculated the proportion of cases with both interpretations in the same diagnostic class. Confidence intervals for intraobserver concordance rates used a logit transformation and robust standard error that accounted for clustering at pathologist level. Logistic regression models were used to test for a difference in intraobserver concordance between AJCC 7and AJCC 8-based mappings. All P values correspond to 2-tailed tests and differences with P < .05 were considered to be statistically significant. Analyses were performed using Stata statistical software (StataCorp), version 14.

Results
The 116 skin biopsy cases defined as invasive melanoma per the consensus reference diagnosis included 55 cases (47%) of T1a invasive melanoma and 61 cases (53%) of T1b or greater using AJCC 7.

Discussion
This analysis provides data that the new AJCC 8 criteria may lead to improved concordance and reproducibility among pathologists in the classification of invasive melanoma, although the size of this effect is modest. One explanation of the improvement in concordance of pathological staging of

Limitations
Limitations of the study include interpretation of a single slide (although participants were asked to assume the slide was representative), use of a testing environment rather than a practice setting, and inability to obtain second opinions and clinical histories. Also, there is no established method to define a gold-standard diagnosis; therefore, improvement in concordance with an expert-defined reference should not necessarily be interpreted as improvement in accuracy. We chose to use the consensus of 3 experienced pathologists because this approach could be replicated in clinical practice. Finally, the relative proportions of cases used for this study are not representative of the population. 16 Strengths include a large number of participating pathologists reviewing the same glass slides on 2 occasions and the ability to assess both concordance with a reference and reproducibility.

Conclusions
Our results suggest that the changes in the AJCC staging will likely have a positive effect on patients.
The consequences of melanoma staging to patients are substantial. Among these are patients' perceptions of long-term implications to their health as determined by the particular stage assigned at diagnosis, economic consequences of health care services, and the magnitude of surgical interventions indicated by the staging classification (eg, size of wide local resection, eligibility for sentinel lymphatic mapping, and implications for other therapies). In view of these clinical ramifications, even modest improvements of 6% to 10% in diagnostic concordance resulting from changes from AJCC 7 to AJCC 8 are important. However, despite improvement, concordance and reproducibility remain low and suggest that conventional histopathology has been parsed to a degree that falls below the limits of reliability for the demands and consequences of the staging schemata that have evolved over time.