Utility of a Deep-Learning Algorithm to Guide Novices to Acquire Echocardiograms for Limited Diagnostic Use | Cardiology | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
Machine Learning–Guided Echocardiogram Image Acquisition

The video demonstrates the interaction of a user with the software to obtain diagnostic echocardiographic images, illustrating the turn-by-turn instructions; the quality meter, which indicates how close the user is to a diagnostic image and dictates when a clip is automatically recorded (termed an auto-capture); and the ability to manually capture the best clip image if the auto-capture is never achieved.

Machine Learning–Guided Echocardiogram Image Acquisition—Sample Echocardiography Study

Example of a nurse-obtained 10-view study demonstrating diagnostic-quality parasternal long-axis views; parasternal short-axis aortic and mitral valve and papillary muscle views; and apical 4-chamber, 5-chamber, 2-chamber, and 3-chamber views; a subcostal 4-chamber view; and an inferior vena cava views.

Knackstedt  C, Bekkers  SC, Schummers  G,  et al.  Fully automated versus standard tracking of left ventricular ejection fraction and longitudinal strain: the FAST-EFS multicenter study.   J Am Coll Cardiol. 2015;66(13):1456-1466. doi:10.1016/j.jacc.2015.07.052 PubMedGoogle ScholarCrossref
Kusunose  K, Abe  T, Haga  A,  et al.  A deep learning approach for assessment of regional wall motion abnormality from echocardiographic images.   JACC Cardiovasc Imaging. 2020;13(2 pt 1):374-381. doi:10.1016/j.jcmg.2019.02.024PubMedGoogle ScholarCrossref
Kusunose  K, Haga  A, Abe  T, Sata  M.  Utilization of artificial intelligence in echocardiography.   Circ J. 2019;83(8):1623-1629. doi:10.1253/circj.CJ-19-0420 PubMedGoogle ScholarCrossref
Madani  A, Arnaout  R, Mofrad  M, Arnaout  R.  Fast and accurate view classification of echocardiograms using deep learning.   NPJ Digit Med. 2018;1:1. doi:10.1038/s41746-017-0013-1 PubMedGoogle ScholarCrossref
Medvedofsky  D, Mor-Avi  V, Amzulescu  M,  et al.  Three-dimensional echocardiographic quantification of the left-heart chambers using an automated adaptive analytics algorithm: multicentre validation study.   Eur Heart J Cardiovasc Imaging. 2018;19(1):47-58. doi:10.1093/ehjci/jew328 PubMedGoogle ScholarCrossref
Narang  A, Mor-Avi  V, Prado  A,  et al.  Machine learning based automated dynamic quantification of left heart chamber volumes.   Eur Heart J Cardiovasc Imaging. 2019;20(5):541-549. doi:10.1093/ehjci/jey137 PubMedGoogle ScholarCrossref
Narula  S, Shameer  K, Salem Omar  AM, Dudley  JT, Sengupta  PP.  Machine-learning algorithms to automate morphological and functional assessments in 2D echocardiography.   J Am Coll Cardiol. 2016;68(21):2287-2295. doi:10.1016/j.jacc.2016.08.062 PubMedGoogle ScholarCrossref
Raghavendra  U, Rajendra Acharya  U, Gudigar  A,  et al.  Fusion of spatial gray level dependency and fractal texture features for the characterization of thyroid lesions.   Ultrasonics. 2017;77:110-120. doi:10.1016/j.ultras.2017.02.003 PubMedGoogle ScholarCrossref
Sengupta  PP, Huang  YM, Bansal  M,  et al.  Cognitive machine-learning algorithm for cardiac imaging: a pilot study for differentiating constrictive pericarditis from restrictive cardiomyopathy.   Circ Cardiovasc Imaging. 2016;9(6):e004330. doi:10.1161/CIRCIMAGING.115.004330 PubMedGoogle Scholar
Zhang  J, Gajjala  S, Agrawal  P,  et al.  Fully automated echocardiogram interpretation in clinical practice.   Circulation. 2018;138(16):1623-1635. doi:10.1161/CIRCULATIONAHA.118.034338 PubMedGoogle ScholarCrossref
Ghorbani  A, Ouyang  D, Abid  A,  et al  Deep learning interpretation of echocardiograms.   NPJ Digit Med. 2020;3:10. doi:10.1038/s41746-019-0216-8 Google ScholarCrossref
Ouyang  D, He  B, Ghorbani  A,  et al.  Video-based AI for beat-to-beat assessment of cardiac function.   Nature. 2020;580(7802):252-256. doi:10.1038/s41586-020-2145-8 PubMedGoogle ScholarCrossref
ECRI. 2020 top 10 health technology hazards executive brief. Published 2019. Accessed March 1, 2020. https://www.ecri.org/landing-2020-top-ten-health-technology-hazards
US Food and Drug Administration. FDA authorizes marketing of first cardiac ultrasound software that uses artificial intelligence to guide user. Published 2020. Accessed February 3, 2021. https://www.fda.gov/news-events/press-announcements/fda-authorizes-marketing-first-cardiac-ultrasound-software-uses-artificial-intelligence-guide-user
Mitchell  C, Rahko  PS, Blauwet  LA,  et al.  Guidelines for performing a comprehensive transthoracic echocardiographic examination in adults: recommendations from the American Society of Echocardiography.   J Am Soc Echocardiogr. 2019;32(1):1-64. doi:10.1016/j.echo.2018.06.004 PubMedGoogle ScholarCrossref
Gallas  BD, Pennello  GA, Myers  KJ.  Multireader multicase variance analysis for binary data.   J Opt Soc Am A Opt Image Sci Vis. 2007;24(12):B70-B80. doi:10.1364/JOSAA.24.000B70 PubMedGoogle ScholarCrossref
Wiegers  SE, Ryan  T, Arrighi  JA,  et al; Writing Committee Members; ACC Competency Management Committee.  2019 ACC/AHA/ASE Advanced Training Statement on Echocardiography (revision of the 2003 ACC/AHA Clinical Competence Statement on Echocardiography): a report of the ACC Competency Management Committee.   J Am Soc Echocardiogr. 2019;32(8):919-943. doi:10.1016/j.echo.2019.04.002 PubMedGoogle ScholarCrossref
Genovese  D, Rashedi  N, Weinert  L,  et al.  Machine learning-based three-dimensional echocardiographic quantification of right ventricular size and function: validation against cardiac magnetic resonance.   J Am Soc Echocardiogr. 2019;32(8):969-977. doi:10.1016/j.echo.2019.04.001 PubMedGoogle ScholarCrossref
Volpato  V, Mor-Avi  V, Narang  A,  et al.  Automated, machine learning-based, 3D echocardiographic quantification of left ventricular mass.   Echocardiography. 2019;36(2):312-319. doi:10.1111/echo.14234 PubMedGoogle ScholarCrossref
Asch  FM, Poilvert  N, Abraham  T,  et al.  Automated echocardiographic quantification of left ventricular ejection fraction without volume measurements using a machine learning algorithm mimicking a human expert.   Circ Cardiovasc Imaging. 2019;12(9):e009303. doi:10.1161/CIRCIMAGING.119.009303 PubMedGoogle Scholar
Sanchez-Martinez  S, Duchateau  N, Erdei  T,  et al.  Machine learning analysis of left ventricular function to characterize heart failure with preserved ejection fraction.   Circ Cardiovasc Imaging. 2018;11(4):e007138. doi:10.1161/CIRCIMAGING.117.007138 PubMedGoogle Scholar
Tabassian  M, Sunderji  I, Erdei  T,  et al.  Diagnosis of heart failure with preserved ejection fraction: machine learning of spatiotemporal variations in left ventricular deformation.   J Am Soc Echocardiogr. 2018;31(12):1272-1284.e9. doi:10.1016/j.echo.2018.07.013 PubMedGoogle ScholarCrossref
Ryan  T, Berlacher  K, Lindner  JR, Mankad  SV, Rose  GA, Wang  A.  COCATS 4 task force 5: training in echocardiography.   J Am Coll Cardiol. 2015;65(17):1786-1799. doi:10.1016/j.jacc.2015.03.035 PubMedGoogle ScholarCrossref
American College of Emergency Physicians. Ultrasound guidelines: emergency, point-of-care, and clinical ultrasound guidelines in medicine. Published 2016. Accessed March 1, 2020. https://www.acep.org/globalassets/new-pdfs/policy-statements/ultrasound-guidelines—emergency-point-of-care-and-clinical-ultrasound-guidelines-in-medicine.pdf
Lewiss  RE. “The ultrasound looked fine”: point-of-care ultrasound and patient safety. Published 2018. Accessed March 1, 2020, 2020. https://psnet.ahrq.gov/web-mm/ultrasound-looked-fine-point-care-ultrasound-and-patient-safety
American College of Cardiology. COVID-19 clinical guidance for the cardiovascular care team. Published 2020. Accessed March 23, 2020. https://www.acc.org/~/media/Non-Clinical/Files-PDFs-Excel-MS-Word-etc/2020/02/S20028-ACC-Clinical-Bulletin-Coronavirus.pdf
American Society of Echocardiography. ASE statement on the protection on patients and the echocardiography service providers during the 2019 novel coronavirus outbreak. Published 2020. Accessed March 23, 2020. https://www.asecho.org/wp-content/uploads/2020/03/ASE-COVID-Statement-FINAL-1.pdf
Cheema  B, Hsieh  C, Adams  D, Narang  A, Thomas  J. Automated guidance and image capture of echocardiographic views using a deep learning-derived technology. American Heart Association; 2019.
Views 15,087
Citations 0
Original Investigation
February 18, 2021

Utility of a Deep-Learning Algorithm to Guide Novices to Acquire Echocardiograms for Limited Diagnostic Use

Author Affiliations
  • 1Bluhm Cardiovascular Institute, Northwestern University, Chicago, Illinois
  • 2Division of Cardiology, Minneapolis Heart Institute, Minneapolis, Minnesota
  • 3Caption Health, Brisbane, California
  • 4Division of Cardiology, Scripps Health, San Diego, California
  • 5Division of Cardiology, MedStar Washington Hospital Center, Washington, DC
  • 6Houston Methodist, Houston, Texas
  • 7Section of Cardiology, The University of Chicago, Chicago, Illinois
  • 8MedStar Health Research Institute, Washington, DC
JAMA Cardiol. Published online February 18, 2021. doi:10.1001/jamacardio.2021.0185
Key Points

Question  Can artificial intelligence guide novice operators to obtain echocardiographic scans with limited diagnostic utility?

Findings  In this diagnostic study, 8 nurses without prior ultrasonography experience used artificial intelligence guidance to scan 30 patients each with a 10-view echocardiographic protocol (240 total patients). Five expert echocardiographers blindly reviewed these scans and felt they were of diagnostic quality for left ventricular size and function in 98.8% of patients, right ventricular size in 92.5%, and presence of pericardial effusion in 98.8%.

Meaning  Artificial intelligence can extend the reach of echocardiography to assess the 4 basic parameters of left ventricular size and function, right ventricular size, and presence of a nontrivial pericardial effusion to sites with limited expertise.


Importance  Artificial intelligence (AI) has been applied to analysis of medical imaging in recent years, but AI to guide the acquisition of ultrasonography images is a novel area of investigation. A novel deep-learning (DL) algorithm, trained on more than 5 million examples of the outcome of ultrasonographic probe movement on image quality, can provide real-time prescriptive guidance for novice operators to obtain limited diagnostic transthoracic echocardiographic images.

Objective  To test whether novice users could obtain 10-view transthoracic echocardiographic studies of diagnostic quality using this DL-based software.

Design, Setting, and Participants  This prospective, multicenter diagnostic study was conducted in 2 academic hospitals. A cohort of 8 nurses who had not previously conducted echocardiograms was recruited and trained with AI. Each nurse scanned 30 patients aged at least 18 years who were scheduled to undergo a clinically indicated echocardiogram at Northwestern Memorial Hospital or Minneapolis Heart Institute between March and May 2019. These scans were compared with those of sonographers using the same echocardiographic hardware but without AI guidance.

Interventions  Each patient underwent paired limited echocardiograms: one from a nurse without prior echocardiography experience using the DL algorithm and the other from a sonographer without the DL algorithm. Five level 3–trained echocardiographers independently and blindly evaluated each acquisition.

Main Outcomes and Measures  Four primary end points were sequentially assessed: qualitative judgement about left ventricular size and function, right ventricular size, and the presence of a pericardial effusion. Secondary end points included 6 other clinical parameters and comparison of scans by nurses vs sonographers.

Results  A total of 240 patients (mean [SD] age, 61 [16] years old; 139 men [57.9%]; 79 [32.9%] with body mass indexes >30) completed the study. Eight nurses each scanned 30 patients using the DL algorithm, producing studies judged to be of diagnostic quality for left ventricular size, function, and pericardial effusion in 237 of 240 cases (98.8%) and right ventricular size in 222 of 240 cases (92.5%). For the secondary end points, nurse and sonographer scans were not significantly different for most parameters.

Conclusions and Relevance  This DL algorithm allows novices without experience in ultrasonography to obtain diagnostic transthoracic echocardiographic studies for evaluation of left ventricular size and function, right ventricular size, and presence of a nontrivial pericardial effusion, expanding the reach of echocardiography to clinical settings in which immediate interrogation of anatomy and cardiac function is needed and settings with limited resources.