[Skip to Content]
[Skip to Content Landing]
Views 2,419
Citations 0
Original Investigation
October 24, 2019

Machine Learning by Ultrasonography for Genetic Risk Stratification of Thyroid Nodules

Author Affiliations
  • 1Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania
  • 2Department of Radiology, Thomas Jefferson University, Philadelphia, Pennsylvania
  • 3Department of Surgery, Lankenau Medical Center, Wynnewood, Pennsylvania
  • 4Beijing Friendship Hospital, Capital Medical University, Beijing, China
  • 5Department of Otolaryngology–Head and Neck Surgery, Thomas Jefferson University, Philadelphia, Pennsylvania
JAMA Otolaryngol Head Neck Surg. 2020;146(1):36-41. doi:10.1001/jamaoto.2019.3073
Key Points

Question  Is machine learning applied to ultrasonography capable of risk-stratifying thyroid nodules by their genetic status?

Findings  In this diagnostic study of 134 lesions among 121 patients, a model developed through automated machine learning was able to identify genetically high-risk thyroid nodules by ultrasonography alone, with a specificity of 97% and positive predictive value of 90%.

Meaning  The findings suggest that machine learning application to genetic risk stratification of thyroid nodules is feasible, affording an additional diagnostic adjunct to cytogenetics for nodules with indeterminate cytological result.


Importance  Thyroid nodules are common incidental findings. Ultrasonography and molecular testing can be used to assess risk of malignant neoplasm.

Objective  To examine whether a model developed through automated machine learning can stratify thyroid nodules as high or low genetic risk by ultrasonography imaging alone compared with stratification by molecular testing for high- and low-risk mutations.

Design, Setting, and Participants  This diagnostic study was conducted at a single tertiary care urban academic institution and included patients (n = 121) who underwent ultrasonography and molecular testing for thyroid nodules from January 1, 2017, through August 1, 2018. Nodules were classified as high risk or low risk on the basis of results of an institutional molecular testing panel for thyroid risk genes. All thyroid nodules that underwent genetic sequencing for cytological results with Bethesda System categories III and IV were reviewed. Patients without diagnostic ultrasonographic images within 6 months of fine-needle aspiration or who received definitive treatment at an outside medical center were excluded.

Main Outcomes and Measures  Thyroid nodules were categorized by the model as high risk or low risk using ultrasonographic images. Results were compared using genetic testing.

Results  Among the 134 lesions identified in 121 patients (mean [SD] age, 55.7 [14.2] years; 102 women [84.3%]), 683 diagnostic ultrasonographic images were selected. Of the 683 images, 556 (81.4%) were used for training the model, 74 (10.8%) for validation, and 53 (7.8%) for testing. Most nodules had no mutation (75 [56.0%]), whereas 43 nodules (32.1%) had a high-risk mutation and 16 (11.9%) had an unknown or a low-risk mutation (χ2 = 39.060; P < .001). In total, 228 images (33.4%) were of nodules classified as genetically high risk (n = 43), and 455 (66.6%) were of low-risk nodules (n = 91). The model performed with a sensitivity of 45% (95% CI, 23.1%-68.5%), a specificity of 97% (95% CI, 84.2%-99.9%), a positive predictive value of 90% (95% CI, 55.2%-98.5%), a negative predictive value of 74.4% (95% CI, 66.1%-81.3%), and an overall accuracy of 77.4% (95% CI, 63.8%-97.7%).

Conclusions and Relevance  The study found that the model developed through automated machine learning could produce high specificity for identifying nodules with high-risk mutations on molecular testing. This finding shows promise for the diagnostic applications of machine learning interpretation of sonographic imaging of indeterminate thyroid nodules.

Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Number of patients
    Johnson Thomas, MD, FACE | Mercy
    Very interesting article.
    How many patients / nodules were there in training, validation and testing set?