Application of Video-Based Deep Learning for Early Diagnosis of Neurological Disorders

The study by Groos et al¹ reported that deep learning (DL)–based assessments of videos of spontaneous movements at gestational corrected age (GCA) 3 months could support early detection of cerebral palsy (CP) in infants with high risk. The study included 557 infants with high risk in 13 hospitals in the United States, Norway, India, and Belgium. Infants had a video from GCA 9 to 18 weeks assessed with the General Movement Assessment and were evaluated for a diagnosis of CP after GCA 12 months. Overall, the DL model had a sensitivity of 71.4% (95% CI, 47.8%-88.7%) and specificity of 94.1% (95% CI, 88.2%-97.6%).¹ Furthermore, the DL method differentiated among infants who developed ambulatory and nonambulatory CP, as well as unilateral and bilateral CP.¹

This study by Groos et al¹ is an important contribution to the existing literature on leveraging video-based DL models to infer a disease based on early symptoms. To represent infant movements in the video, Groos and colleagues¹ used a spatiotemporal skeleton sequence divided into 5-second windows, which the DL model processed to estimate CP risk in that particular window time. The initial layer of the model detected features of movements of a single limb or joint, whereas subsequent layers detected features of whole-body movements. Finally, the model analyzed the biomechanical properties (position, velocity, and body segment length) in 5-second windows to detect whole-body movement features that distinguished infants with CP from infants without CP. In this regard, infants in a video sequence can be described by their motion, skeleton, and spatial characteristics, which could infer the diagnosis of CP.

This interesting approach builds on the efforts of other researchers striving for more accurate detection and evaluation methods for neurological disorders with the help of machine learning. Gait analysis, which is a key aspect of clinical assessments for quantifying functional outcomes following a neurological or musculoskeletal disease, was attempted with DL analyses in various health conditions, such as stroke, Parkinson disease, CP, and spinal cord injury.^2-4 In Parkinson disease, DL-based assessments of gait features using video could allow early discovery of gait anomalies in the course of the disease.⁴ This solution could be more reliable than commonly used clinical tests, such as the Hoehn and Yahr scale or the Unified Parkinson Disease Rating Scale. Similarly, single-camera videos and pose estimation models based on DL networks could be used to quantify clinically relevant gait metrics in individuals poststroke, such as measuring reduced walking speeds, decreased cadence, prolonged swing time, and reduced stance time on the paretic side.² Likewise, a study by Bandini et al³ proposed a DL algorithm for automatically detecting hand-object interactions in egocentric videos recorded by participants with cervical spinal cord injury during their daily activities at home. This technique allows the assessment of neurological function in an everyday life environment, which may better indicate the actual hand function. Although DL for quantification of human behavior through videography is still in its infancy, applications are promising and offer novel metrics that, once validated with clinical criterion standards, will constitute clinically valid tools for early diagnosis and surveillance of recovery and prognosis of various neurological disorders.

Published: July 11, 2022. doi:10.1001/jamanetworkopen.2022.21343

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Massaad E et al. JAMA Network Open.

Corresponding Author: Elie Massaad, MD, MMSc, Department of Neurosurgery, Massachusetts General Hospital, Harvard Medical School, 55 Fruit St, Boston, MA 02114 (emassaad@mgh.harvard.edu).

Conflict of Interest Disclosures: None reported.