Effect of Gene Expression Classifier Molecular Testing on the Surgical Decision-Making Process for Patients With Thyroid Nodules

Importance  Commercial molecular testing, such as the gene expression classifier (GEC), is now being used in the work up of cytologically indeterminate thyroid nodules. While this test may be helpful in ruling out malignancy in a thyroid nodule, its effect on surgical decision making has yet to be fully defined.

Objective  We aimed to determine the effect and outcome of GEC test results on the decision-making process for patients with thyroid nodules presenting for surgical consultation.

Design, Setting, and Participants  A surgical management algorithm was developed that incorporated individual Bethesda System for Reporting Thyroid Cytopathology classifications, in addition to clinical, laboratory, and radiological findings. We then retrospectively applied this algorithm to 273 consecutive patients with thyroid nodules and GEC test results who had presented for surgical consultation between February 1, 2012, and December 31, 2014.

Interventions  GEC testing.

Main Outcomes and Measures  Changes in management were recorded to identify the effect of GEC testing on the surgical decision-making process. An alteration in management of 20% of cases was considered significant.

Results  Of the 273 consecutive patients assessed by the GEC, mean (SD) age was 50.8 (14.7) years, 204 (74.7%) were female, and the mean (SD) nodule size was 2.4 (1.3) cm. Test results were suspicious for 233 (85.3%); benign for 31 (11.4%); and indeterminate for 8 (2.9%). The GEC test was also positive for medullary thyroid cancer for 1 patient (0.4%). The GEC test was correctly used as a rule-out test in only 127 patients (46.5%) with indeterminate nodules who lacked a clinical indication for surgery. The clinical management plan of only 23 (8.4%) patients was altered as a result of GEC test results, and of these 23 patients who proceeded to surgery, 16 patients (72.7%) were found to be inappropriately overtreated relative to postoperative histopathology analysis. We found that GEC testing did not affect the surgical decision-making process in 250 (91.6%) of our patients. In 146 cases, the use of GEC testing was not clinically indicated, and the test was being overused in patients for whom the results would not change surgical management. The positive predictive value of the GEC test for cytologically indeterminate nodules was 42.1%, and the negative predictive value was 83.3%.

Conclusions and Relevance  The GEC testing did not significantly affect the surgical decision-making process. Gene expression classifier testing is often used incorrectly and is overused in patients for whom the results would not change management. The GEC test demonstrated a lower than expected negative predictive value, and there was evidence of overtreatment among patients whose treatment was altered based on this test.

The clinical importance of nodular thyroid disease primarily rests with the need to exclude nodules that may harbor malignancy. The challenge for any preoperative diagnostic test is to sustain a high sensitivity so that cancers are not missed while simultaneously achieving a high negative predictive value (NPV) to avoid surgery in patients whose nodules are benign. This goal is usually accomplished by obtaining a detailed patient history and performing physical examination and ultrasonography of the neck on the patient. The information obtained will direct subsequent evaluation, including the need for a fine-needle aspiration (FNA) biopsy assessment.¹ When the cytomorphological diagnosis is benign, the thyroid aspirate has an NPV that exceeds 95%, and further diagnostic studies or treatment are usually not required. When the cytomorphological diagnosis is malignant, the thyroid aspirate has a positive predictive value (PPV) that exceeds 97%, and surgery is generally recommended.^1,2 Nonetheless, according to the Bethesda System for Reporting Thyroid Cytopathology,³ approximately one-third of all thyroid aspirates yield indeterminate or suspicious cytologic findings, with an estimated malignancy rate ranging from 5% to 75%.^2,4

Owing to a lack of a definitive diagnosis for indeterminate or suspicious thyroid nodules, diagnostic surgery to establish a histopathological diagnosis and reconcile the behavior of these nodules is required in most cases. However, only 16% to 26% of surgically resected indeterminate nodules and 75% of surgically resected suspicious nodules prove to be malignant.⁵ Such diagnostic operations, with their associated expense and risk, could be avoided if FNA procedures could reliably establish the preoperative diagnosis of a benign nodule. Conversely, unnecessary procedures and multiple surgical procedures could both be avoided if a more accurate preoperative diagnosis of malignant thyroid nodules could be found.

Progress has been made in understanding the molecular mechanisms of thyroid cancer over the past 2 decades.⁶ This understanding has led to the identification and clinical use of novel diagnostic molecular markers as preoperative adjuncts to improve risk stratification and tailor more effective management strategies in patients with indeterminate or suspicious thyroid nodules.⁷ The Afirma gene expression classifier (GEC) (Veracyte) is a proprietary RNA microarray assay developed to improve the diagnosis of patients with cytologically indeterminate thyroid nodules. The GEC test measures the expression of 167 gene transcripts from a cellular aspirate to rule out thyroid cancer by determining which nodules have a benign gene expression profile to obviate the need for diagnostic surgery.⁸ In a large multicenter clinical trial by Alexander et al,⁹ the GEC was shown to have an overall sensitivity and specificity of 92% for indeterminate thyroid aspirates and 52% for suspicious thyroid aspirates, with an overall NPV ranging from 85% to 95%. The NPV of the GEC is therefore used to identify a subgroup of patients with a low risk of malignancy in a population of patients for whom surgery would have otherwise been recommended. Observation or ultrasonography follow-up is recommended in lieu of surgery for these patients as outlined in the recent revision of guidelines for the management of thyroid cancer issued by the National Comprehensive Cancer Network.¹⁰

While the GEC appears promising in differentiating benign from malignant thyroid nodules, its true effect on the preoperative surgical decision-making process has yet to be fully defined. Furthermore, the nonspecificity of the GEC is critical in any clinical practice setting, as the clinical use of a suspicious result remains questionable. Our preliminary results indicate that molecular marker testing is currently being overused and does not alter the surgical management in most patients for whom this testing has been done.¹¹ This is likely owing to the lack of specific guidelines from the professional surgical societies on the proper use of these markers and, more importantly, the fact that current management strategies for nodular thyroid disease take into account a myriad of clinical considerations beyond cytopathological findings.¹ Assessment of diagnostic molecular marker test results in the context of these clinical considerations is necessary to evaluate their true effect on the surgical decision-making process. We therefore chose to determine the effect and the outcome of GEC test results on the decision-making process for patients with thyroid nodules presenting for surgical consultation at our institution.

Under Johns Hopkins University institutional review board approval, we conducted a retrospective medical record review to evaluate all consecutive patients with thyroid nodules who received GEC testing and presented for surgical consultation at Johns Hopkins Hospital between February 1, 2012, and December 31, 2014. During this time, patients were referred for surgical consultation to 10 surgeons with different training backgrounds, 8 of whom were dedicated thyroid surgeons who evaluated 269 patients (98.5%). Only patients who received GEC testing for the thyroid nodule they were being referred for were included in this study. Results of the GEC testing included suspicious for malignancy, benign, or indeterminate. Positivity for medullary thyroid cancer during the screening for nonfollicular cell–derived tumors prevented the material from being scored by the main GEC. Indeterminate GEC test results included FNA samples with inadequate or suboptimal tissue to perform the assay.

As a standard of care at our institution, all outside thyroid FNA specimens were routinely read by one of the cytopathologists at our institution. The final in-house cytologic diagnoses were classified according to the Bethesda System for Reporting Thyroid Cytopathology,³ as described previously.¹² Categorical diagnoses included: (I) insufficient for diagnosis; (II) benign; (III) atypia of undetermined significance (AUS); (IV) suspicious for follicular neoplasm (SFN) or Hürthle cell neoplasm (SHCN); (V) suspicious for malignancy (SFM); and (VI) malignant. Categories III and IV represent the indeterminate classifications, and category V represents the SFM classification.

Prior to initiating the retrospective medical record review, a surgical management algorithm was developed and published in a peer-reviewed journal.¹¹ It incorporated individual Bethesda System for Reporting Thyroid Cytopathology classifications (in-house cytologic diagnosis), in addition to clinical, laboratory, and radiologic findings. The algorithm was created by consensus of the dedicated thyroid surgeons and/or authors to formulate their decision-making process in managing patients with nodular thyroid disease (Figure). Creating a management algorithm allowed us to determine whether the GEC test results had an effect on the surgical decision-making process. The actual management undertaken by the surgeon was then compared with the corresponding recommended management option dictated by the algorithm. If the surgical management recommendation was the same as expected from the algorithm, the GEC test result was considered to have no effect on the decision-making process. Conversely, if the surgical management recommendation was different from what was dictated by the algorithm, the GEC test result was considered to affect the decision-making process unless otherwise reported (eg, a patient with a nodule found to be AUS with nuclear atypia on cytology and a benign GEC test result who was recommended to undergo thyroidectomy for high clinical suspicion of malignancy). An alteration in management of 20% of cases from what was outlined in the algorithm was considered significant. The specific reason(s) for the change in management and extent of treatment were recorded. To evaluate the appropriateness of any change in management, the histopathology diagnosis was correlated with the surgery performed. If the nodule was histologically malignant and the change in management was from a less extensive to more extensive surgery (eg, from hemithyroidectomy to total thyroidectomy), the change was deemed appropriate. Conversely, if the nodule was histologically benign and the change in management was from a more extensive to less extensive surgical procedure, the change was also deemed appropriate. Any other change was deemed inappropriate.

Demographic, clinical, laboratory, and radiologic data were also recorded. For patients who proceeded to surgery, the final histopathology report was reviewed for each case and compared with the preoperative cytology report for direct correlation between samples, as well as to identify the histopathologic nature of the biopsied nodule. Incidental thyroid cancers were not included in the analysis. Data analysis was performed using Microsoft Excel 2013 (Microsoft). All values are expressed as mean (SD), ranges, or absolute numbers.

From February 1, 2012, through December 31, 2014, 273 patients with nodular thyroid disease who were evaluated by the GEC presented for surgical consultation at our institution. The mean (SD) age at presentation was 50.8 (14.7) (range, 17-90) years and 204 patients (74.7%) were female. The mean (SD) nodule size was 2.4 (1.3) (range, 0.4-8.0) cm. One hundred and twenty seven patients (46.5%) had unilobular nodular involvement, and the remaining 146 patients (53.5%) had bilateral multinodular goiters. Seventy-one patients (26%) reported compressive symptoms owing to their nodules, 59 patients (21.6%) were either hypothyroid or hyperthyroid, 11 (4%) had a history of ionizing radiation exposure to the head and neck region, and 12 (4.4%) had a family history of thyroid cancer.

After reviewing the thyroid aspirates at our institution, the in-house FNA diagnoses were as follows: 2 (0.7%) were insufficient for diagnosis; 34 (12.5%), benign; 165 (60.4%), AUS; 32 (11.7%), SFN; 25 (9.2%), SHCN; 9 (3.3%), SFM; and 6 (2.2%), malignant. The corresponding GEC results are summarized in Table 1. Interestingly, of the 51 patients with an in-house thyroid aspirate diagnosis other than AUS, SFN, or SHCN (ie, Bethesda class III and IV³), the outside FNA diagnosis was insufficient for diagnosis in 1 (2%) patient, benign in 12 (24%) patients, and SFM in 4 (8%) patients when the GEC was performed on the thyroid aspirates. Therefore, based on cytology findings only, the GEC test was not clinically indicated in 17 (6.2%) patients.

After surgical consultation, there was no change in surgical management for 231 patients (84.6%). The recommended management was different than what was outlined in the surgical algorithm in 42 patients for the following reasons: GEC test result, 23 patients (8.4%) (Table 2); ultrasonography characteristics of the thyroid nodules (eg, subcentimeter and/or benign-looking or cystic contralateral nodules), 11 (4%); concerning characteristics found on cytology (eg, nuclear atypia, neoplastic cells), 3 (1.1%); outside cytology diagnosis, 1 (0.4%); comorbidities, 3 (1.1%); and patient preference, 1 (0.4%). The GEC test was clinically indicated and correctly used as a rule-out test in patients with cytologically indeterminate nodules (read on the independent cytological assessment specific to Afirma) who lacked a clinical indication for surgery in 127 patients (46.5%).

Of the 273 patients included in this study, 225 (82.4%) proceeded to surgery. Sixty-three (28.0%) underwent hemithyroidectomy, 150 (66.7%) underwent total thyroidectomy, and 12 (5.3%) underwent total thyroidectomy with central compartment neck dissection. All central compartment neck dissections were therapeutic except 1 that was prophylactic for a diagnosis of medullary thyroid cancer. On postoperative histopathology analysis, the biopsied nodules were found to be benign for 75 patients (33.3%); 55 patients (24.4%) were found to have follicular or Hürthle cell adenoma; and 95 patients (42.2%) were found to have thyroid cancer (Table 1). The thyroid cancers included 32 cases (33.7%) of papillary thyroid cancer; 49 (51.6%) cases of follicular-variant papillary thyroid cancer (all encapsulated except 2); 7 cases (7.4%) of follicular thyroid carcinoma; 5 cases (5.3%) of Hürthle cell carcinoma; and 2 cases (2.1%) of medullary thyroid carcinoma. Twenty-seven (12%) additional, incidental subcentimeter papillary and follicular variant thyroid cancers not included in this analysis were identified.

Table 3 summarizes the 23 cases where there was a change in surgical management owing to the GEC test result. Interestingly, the molecular test was not clinically indicated in 5 (22%) patients (4 patients were symptomatic and there was a high clinical suspicion for malignancy in another patient), yet the suspicious GEC result affected the surgeon’s decision to perform a total thyroidectomy in lieu of the recommended hemithyroidectomy. The change in management was owing to concern about the higher probability of malignancy of the suspicious result or to influence from the referring endocrinologist and their interpretation of a suspicious GEC test result. This interpretation of the suspicious GEC test result was more evident during the initial adoption of the test because there was more enthusiasm for total thyroidectomy by the referring endocrinologists. All of these patients had benign disease on postoperative histopathology except a patient who had a subcentimeter-encapsulated follicular variant of papillary thyroid cancer (pT1a N0 Mx).

Correlation between the cytology diagnosis, GEC test results, and histologic surgical diagnosis was determined in all 225 patients who underwent surgery. The overall prevalence of malignancy was 42.2% in this study. Overall, the GEC test labeled 88 malignant nodules (93%) as GEC suspicious. However, 110 histologically benign nodules were also identified as GEC suspicious. Therefore, the sensitivity of the GEC test was 96.7%; specificity, 14.7%; PPV, 44.4%; and NPV, 86.3%. When only examining the indeterminate nodules (in-house reading as Bethesda III and IV), the prevalence of malignancy was 41.2%. The NPV and PPV of the GEC test in Bethesda category III and IV nodules were found to be 83.3% and 42.1%, respectively (Table 4).

In this study, we analyzed 35 successive months of GEC testing in patients being surgically evaluated at our academic institution. We found that GEC testing did not affect the surgical decision-making process in 250 (91.6%) of our patients. In 146 cases, the use of GEC testing was not clinically indicated, and the test was being overused in patients for whom the results would not change surgical management. Moreover, in the 23 patients for whom there was a change in management owing to the GEC test results, the molecular test was not clinically indicated in 5 (22%) patients, yet the suspicious GEC test result affected the surgeon’s decision making (Table 3).

We believe that GEC testing is being overused and does not affect the surgical decision-making process in most cases because there are a lack of specific guidelines from professional surgical societies on the proper use or interpretation of GEC testing. It should also be noted that current management strategies of nodular thyroid disease take into account numerous clinical considerations beyond cytolopathological findings alone. Automatically sending all cytologically indeterminate FNA samples for GEC testing is inappropriate because the cytologic results and clinical presentations will have already provided actionable diagnostic guidance in most cases.¹ In our study, the NPV of the GEC test for patients with a benign in-house cytology diagnosis who underwent surgery was 87.5%, and the PPV was 23.1% for the same population. As a test to rule out thyroid cancer, the GEC should only be used in patients with indeterminate Bethesda cytologic findings and should specifically be limited to patients with minimal clinical suspicion for malignancy and only if the result would help in the decision analysis. Moreover, the GEC was unlikely to rule out a malignancy, nor did it add any benefit to the decision-making process during surgical management in the following scenarios: (1) nodules that yielded material insufficient for diagnosis, benign nodules, nodules SFM, or malignant nodules on cytological evaluation; (2) nodules with strong clinical or radiological findings suggestive of a thyroid malignancy; (3) indeterminate nodules greater than 4 cm, a size at which nodules may harbor an occult thyroid cancer owing to FNA sampling error; (4) nodules causing compressive symptoms, for which surgery would be recommended regardless; and (5) patients who prefer surgery (ie, uncomfortable with conservative management of a nodule regardless of the GEC test result).

The clinician and patient should be aware that, even as a test to rule out malignancy, GEC testing has a false-negative rate of up to 17% in patients who present for surgery, especially if applied in the context of a specialized academic center. Therefore, continued follow-up assessment is crucial. The reliability of the GEC test result depends on the input of cytopathology because it determines the pretest probability of malignancy. This, in turn, inversely influences the NPV of the GEC test. Accordingly, the potential variance of NPV might affect physician reliance on test results for determining the need for surgery. Conversely, a suspicious test result does not mean the patient has cancer, as the PPV for a suspicious GEC test is approximately 42% with a very low specificity. Clinical parameters and patient preference typically define the extent of surgery in this scenario. Although the prevalence of malignancy in this study seems higher than that reported in the Bethesda classification,³ it is largely owing to the homogenous nature of the patients in this study (ie, those with a suspicious GEC test result). It is reasonable to consider that patients with benign-looking indeterminate nodules and indeterminate nodules that are not concerning for malignancy will not be referred for surgical consultation because current guidelines favor conservative monitoring in these patients. Therefore, our group of patients differs from the overall patients with an indeterminate diagnosis in that they were already considered to have a higher risk of malignancy clinically, although some may have been referred for surgical consultation owing to compressive symptoms rather than ruling out malignancy. If all patients at our institution with indeterminate nodules proceeded to surgery, the malignancy rate would have been similar to that of the Bethesda classification.

One of the other limitations of the GEC assay is that FNA samples are subjected to standardized cytological evaluation by the company’s independent cytopathologists at the Thyroid Cytopathology Partners (Austin), and the results are then forwarded to the ordering physician. Clinicians usually have confidence in their institution’s cytopathologists for evaluation of FNA biopsy samples and are familiar with the wording and intonations from a long-standing relationship. In our study, we found 46 cases (16.8%) where there was discordance in the cytologic diagnoses compared with the results obtained from the Thyroid Cytopathology Partners used by Veracyte. This is similar to the study by Alexander et al⁸ who found that there was 14% discordance in diagnoses. Therefore, it may be difficult for some clinicians to rely on outside cytology readings.¹³ In addition, our academic cytopathologists routinely gather quality assurance data on their experience. Conversely, such information is not readily available to the clinician from the Thyroid Cytopathology Partners.⁶

In 2011, Li and colleagues¹⁴ performed a cost-effective modeling analysis of theoretically using the GEC test for cytologically indeterminate thyroid nodules. Assuming that surgery could be avoided in almost 75% of patients with benign thyroid nodules, Li and colleagues concluded that GEC testing would be cost-effective and would reduce health care expenses, despite its direct cost of $4200. Similarly, Najafzadeh et al¹⁵ constructed a simulation model to estimate the clinical and economic outcomes of using a molecular test compared with current practice for diagnosing thyroid nodules. The authors of this study made assumptions in their model by using a molecular marker with 95% sensitivity and specificity. Assuming that surgery could be avoided in almost 93% of patients with benign thyroid nodules, the authors concluded that using molecular testing would result in a saving of $1087 in direct costs per patient, in addition to improving the quality of life for the patient. However, the much lower rate of surgery avoidance that could have been achieved from our study, including others in the medical literature,^8,13 call these analyses into question and suggest that the costs of routine or reflexive implementation of this molecular test for all patients with indeterminate nodules may be substantially higher than initially reported. Including a large group of patients with Bethesda Class III nodules in our study may also lead to higher costs owing to the higher false-positive rate of the GEC test.⁶ Cost-effectiveness studies also need to take into consideration the potential need for future biopsy procedures with expected GEC testing during the follow-up for patients who were found to have an initial benign GEC test result.

In this study, GEC molecular testing as an adjunct to the standard clinical, radiologic, and FNA evaluation of thyroid nodules did not significantly affect the surgical decision-making process. Among patients whose treatment was altered based upon these markers, there was evidence of overtreatment. The indication and effect of such testing needs to be determined at individual institutions based on clinical philosophies for thyroid nodule management. Moreover, the GEC test demonstrated a lower than expected NPV in Bethesda class III and IV indeterminate nodules. This suggests that the NPV of GEC test results are lower than previously reported and calls into question the performance of the test when applied to a patient seeking surgical consultation in a specialized thyroid surgery center.

Finally, clinicians must continue to follow the current management strategies of nodular thyroid disease instead of only relying on any single diagnostic test. Patient characteristics, clinical findings on physical examination, laboratory results, radiological findings, cytological diagnoses, and patient preferences should be all taken into account and applied in a balanced manner. Since the field of diagnostic molecular testing is new and advances are regularly occurring, clinicians need to stay informed of continued research and advances made in the field that may change the understanding of how best to use molecular markers in a clinical setting.

Submitted for Publication: April 2, 2015; final revision received August 27, 2015; accepted October 2, 2015.

Corresponding Author: Ralph P. Tufano, MD, MBA, Division of Head and Neck Endocrine Surgery, Department of Otolaryngology–Head and Neck Surgery, The Johns Hopkins University School of Medicine, 601 N Caroline St, Sixth Floor, Baltimore, MD 21287 (rtufano1@jhmi.edu).

Published Online: November 25, 2015. doi:10.1001/jamaoto.2015.2708.

Author Contributions: Drs Noureldine and Olson had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Noureldine, Olson, Agrawal, Prescott, Zeiger, Tufano.

Acquisition, analysis, or interpretation of data: Noureldine, Olson, Agrawal, Zeiger, Tufano.

Drafting of the manuscript: Noureldine, Zeiger.

Critical revision of the manuscript for important intellectual content: Olson, Agrawal, Prescott, Zeiger, Tufano.

Statistical analysis: Noureldine.

Administrative, technical, or material support: Olson, Agrawal, Prescott, Tufano.

Study supervision: Noureldine, Olson, Agrawal, Prescott, Zeiger, Tufano.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This article was presented at the Annual Meeting of the American Head and Neck Society; April 23, 2015; Boston, Massachussetts.