Is machine learning applied to ultrasonography capable of risk-stratifying thyroid nodules by their genetic status?
In this diagnostic study of 134 lesions among 121 patients, a model developed through automated machine learning was able to identify genetically high-risk thyroid nodules by ultrasonography alone, with a specificity of 97% and positive predictive value of 90%.
The findings suggest that machine learning application to genetic risk stratification of thyroid nodules is feasible, affording an additional diagnostic adjunct to cytogenetics for nodules with indeterminate cytological result.
Thyroid nodules are common incidental findings. Ultrasonography and molecular testing can be used to assess risk of malignant neoplasm.
To examine whether a model developed through automated machine learning can stratify thyroid nodules as high or low genetic risk by ultrasonography imaging alone compared with stratification by molecular testing for high- and low-risk mutations.
Design, Setting, and Participants
This diagnostic study was conducted at a single tertiary care urban academic institution and included patients (n = 121) who underwent ultrasonography and molecular testing for thyroid nodules from January 1, 2017, through August 1, 2018. Nodules were classified as high risk or low risk on the basis of results of an institutional molecular testing panel for thyroid risk genes. All thyroid nodules that underwent genetic sequencing for cytological results with Bethesda System categories III and IV were reviewed. Patients without diagnostic ultrasonographic images within 6 months of fine-needle aspiration or who received definitive treatment at an outside medical center were excluded.
Main Outcomes and Measures
Thyroid nodules were categorized by the model as high risk or low risk using ultrasonographic images. Results were compared using genetic testing.
Among the 134 lesions identified in 121 patients (mean [SD] age, 55.7 [14.2] years; 102 women [84.3%]), 683 diagnostic ultrasonographic images were selected. Of the 683 images, 556 (81.4%) were used for training the model, 74 (10.8%) for validation, and 53 (7.8%) for testing. Most nodules had no mutation (75 [56.0%]), whereas 43 nodules (32.1%) had a high-risk mutation and 16 (11.9%) had an unknown or a low-risk mutation (χ2 = 39.060; P < .001). In total, 228 images (33.4%) were of nodules classified as genetically high risk (n = 43), and 455 (66.6%) were of low-risk nodules (n = 91). The model performed with a sensitivity of 45% (95% CI, 23.1%-68.5%), a specificity of 97% (95% CI, 84.2%-99.9%), a positive predictive value of 90% (95% CI, 55.2%-98.5%), a negative predictive value of 74.4% (95% CI, 66.1%-81.3%), and an overall accuracy of 77.4% (95% CI, 63.8%-97.7%).
Conclusions and Relevance
The study found that the model developed through automated machine learning could produce high specificity for identifying nodules with high-risk mutations on molecular testing. This finding shows promise for the diagnostic applications of machine learning interpretation of sonographic imaging of indeterminate thyroid nodules.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Daniels K, Gummadi S, Zhu Z, et al. Machine Learning by Ultrasonography for Genetic Risk Stratification of Thyroid Nodules. JAMA Otolaryngol Head Neck Surg. Published online October 24, 2019. doi:https://doi.org/10.1001/jamaoto.2019.3073
Browse and subscribe to JAMA Network podcasts!
Customize your JAMA Network experience by selecting one or more topics from the list below.
Create a personal account or sign in to: