High-Throughput, Contact-Free Detection of Atrial Fibrillation From Video With Deep Learning | Atrial Fibrillation | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Figure.  Experimental Setup
Experimental Setup

A, Example video recording of 5 patients arranged in 1 of the 32 different heart rhythm permutations with extracted facial photoplethysmographic (FPPG) signals and reference electrocardiogram (ECG) traces. B, The 5-participant binary (atrial fibrillation [AF]/sinus rhythm [SR]) matrix.

Table.  Diagnostic Accuracy of Facial Photoplethysmographic in Detecting Atrial Fibrillation
Diagnostic Accuracy of Facial Photoplethysmographic in Detecting Atrial Fibrillation
1.
Freedman  B, Camm  J, Calkins  H,  et al; AF-Screen Collaborators.  Screening for atrial fibrillation: a report of the AF-SCREEN international collaboration.  Circulation. 2017;135(19):1851-1867. doi:10.1161/CIRCULATIONAHA.116.026693PubMedGoogle ScholarCrossref
2.
Yan  BP, Lai  WHS, Chan  CKY,  et al.  Contact-free screening of atrial fibrillation by a smartphone using facial pulsatile photoplethysmographic signals.  J Am Heart Assoc. 2018;7(8):e008585. doi:10.1161/JAHA.118.008585PubMedGoogle Scholar
3.
Sun  Y, Thakor  N.  Photoplethysmography revisited: from contact to noncontact, from point to imaging.  IEEE Trans Biomed Eng. 2016;63(3):463-477. doi:10.1109/TBME.2015.2476337PubMedGoogle ScholarCrossref
4.
Poh  MZ, Poh  YC, Chan  PH,  et al.  Diagnostic assessment of a deep learning system for detecting atrial fibrillation in pulse waveforms.  Heart. 2018;104(23):1921-1928. doi:10.1136/heartjnl-2018-313147PubMedGoogle ScholarCrossref
Research Letter
November 27, 2019

High-Throughput, Contact-Free Detection of Atrial Fibrillation From Video With Deep Learning

Author Affiliations
  • 1Division of Cardiology, Department of Medicine and Therapeutics, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
  • 2Division of Cardiology, Department of Medicine and Therapeutics, Prince of Wales Hospital, Hong Kong SAR, China
  • 3Heart and Vascular Institute, The Chinese University of Hong Kong, Hong Kong SAR, China
  • 4Heart Research Institute, Charles Perkins Centre, and Concord Hospital Cardiology, University of Sydney, Australia
  • 5Cardiio Inc, Cambridge, Massachusetts
JAMA Cardiol. 2020;5(1):105-107. doi:10.1001/jamacardio.2019.4004

Approaches for atrial fibrillation (AF) detection can screen only 1 patient at a time.1 In 2018,2 we demonstrated a novel method of AF detection by analyzing facial photoplethysmographic (FPPG) signals without physical contact using a smartphone camera.2 In this proof-of-concept study, we prospectively evaluated the feasibility of high-throughput AF detection by analyzing FPPG signals3 from multiple patients concurrently using a single digital camera and a pretrained deep convolutional neural network (DCNN).4

Methods

After institutional approval from the Joint Chinese University of Hong Kong–New Territories East Cluster Clinical Research Ethics Committee and individual written informed consent, 20 patients (mean [SD] age, 76.6 [7.6] years; 12 men [60%]) with permanent AF and 24 control individuals (mean [SD] age, 56.8 [20.2] years; 14 men [58.3%]) in sinus rhythm (SR) were recruited. A digital camera (50D; Canon) was used to film 5 patients sitting in a row 150 cm away (Figure). We recorded 64 videos (1-minute duration, 24 FPS), each capturing 5 patients simultaneously in 32 different heart-rhythm permutations based on a 5-participant binary (AF/SR) matrix and repeated once using patients chosen at random. Patients were instructed to keep their head stationary and not talk. The FPPG signals from patients were automatically extracted from the videos, resampled to 30 Hz, and analyzed in segments of 512 samples using Cardiio Deep Rhythm (Cardiio Inc), a DCNN previously trained for detecting AF from PPG waveforms.4 Pulse irregularity in more than 50% FPPG segments for each patient was considered positive for AF. The investigator applying DCNN was blinded to the reference electrocardiogram (ECG) and participant binary matrix.

We calculated Cohen κ coefficients to assess the agreement and conducted a generalized linear mixed model with AF/SR as dependent variable, using a logit link, to control for seating position and heart-rhythm permutation as fixed effects and patients as random effect. All P values less than .05 were considered significant, and all P values were 2-sided.

Results

On average, each patient appeared in 7 videos. Of the 44 patients, the test-retest reliability of FPPG was 95.4% with κ = 0.91 (95% CI, 0.78-1.00). The generalized linear mixed model demonstrated very good reliability and consistent results between patients (intraclass correlation coefficient = 0.88; P = .004), irrespective of seating position and heart-rhythm permutation. A total of 320 individual FPPG signals were analyzed (Table). Agreement between FPPG and ECG was excellent (95.9%), with κ = 0.92 (95% CI, 0.88-0.96) varying between 0.84 and 1.00 across seating positions. Overall sensitivity was 93.8% (95% CI, 88.9%-96.6%); specificity, 98.1% (95% CI, 94.6%-99.4%); positive predictive value, 98.0% (95% CI, 94.2%-99.4%); and negative predictive value, 94.0% (95% CI, 89.6%-96.6%) in discriminating AF from SR. Heart rhythms of all 5 patients were correctly identified in 79.7% of videos (n = 51 of 64) while only 1 patient was misclassified in the remaining 20.3% (n = 13 of 64). Possible causes for false-positive results (n = 3) included motion artifact and ectopic beats, whereas slow AF with ventricular rates less than 60 bpm was the main possible cause for false-negative results (n = 9 of 10). Receiver operating characteristic analysis yielded a C statistic of 0.99; thresholds of 0.33, 0.91, and 0.67 were optimized for sensitivity (sensitivity 100% and specificity 85%), specificity (sensitivity 84.4% and specificity 99.4%), and both sensitivity and specificity (sensitivity 95.6% and specificity 96.2%), respectively.

Discussion

To our knowledge, this is the first study to demonstrate detection of AF with high accuracy from multiple patients concurrently with a single camera. Our findings raise the possibility that this low-cost approach may be scalable for high-throughput AF screening. The proposed approach circumvents the need to perform serial 1-on-1 screening and requires minimal effort from patients, potentially saving time and reducing work for clinical staff. Limitations include the need for patients to stay still for a minute and confirmatory ECG for suspected AF. In practice, the positive predictive value will inevitably be lower because AF prevalence in the general population is lower than in this proof-of-concept study. Importantly, this technology must be deployed in a manner that respects patient privacy (eg, having a clearly designated seating area in the clinic for opt-in screening). Further studies are warranted to evaluate performance in a real-world clinical and home setting.

Back to top
Article Information

Corresponding Author: Bryan P. Yan, MD, Division of Cardiology, Department of Medicine and Therapeutics, Prince of Wales Hospital, Shatin, NT, Hong Kong SAR, China(bryan.yan@cuhk.edu.hk).

Accepted for Publication: August 22, 2019.

Published Online: November 27, 2019. doi:10.1001/jamacardio.2019.4004

Author Contributions: Dr Yan had access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Yan, Lai, Chan, M. Poh.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Yan, Au, M. Poh.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Lai, Chan, Freedman.

Obtained funding: Yan.

Administrative, technical, or material support: Lai, Chan, Y. Poh, M. Poh.

Supervision: Yan.

Conflict of Interest Disclosures: Drs Y. Poh and M. Poh are cofounders and employees of Cardiio Inc and have an ownership stake in the company. No other disclosures were reported.

Funding/Support: This study was supported by a grant to Drs Yan and Freedman from the Hong Kong Research Grants Council–General Research Fund (14118314). Cardiio Inc provided a digital single-lens reflex camera for study purposes.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the patients for granting permission to publish this information.

References
1.
Freedman  B, Camm  J, Calkins  H,  et al; AF-Screen Collaborators.  Screening for atrial fibrillation: a report of the AF-SCREEN international collaboration.  Circulation. 2017;135(19):1851-1867. doi:10.1161/CIRCULATIONAHA.116.026693PubMedGoogle ScholarCrossref
2.
Yan  BP, Lai  WHS, Chan  CKY,  et al.  Contact-free screening of atrial fibrillation by a smartphone using facial pulsatile photoplethysmographic signals.  J Am Heart Assoc. 2018;7(8):e008585. doi:10.1161/JAHA.118.008585PubMedGoogle Scholar
3.
Sun  Y, Thakor  N.  Photoplethysmography revisited: from contact to noncontact, from point to imaging.  IEEE Trans Biomed Eng. 2016;63(3):463-477. doi:10.1109/TBME.2015.2476337PubMedGoogle ScholarCrossref
4.
Poh  MZ, Poh  YC, Chan  PH,  et al.  Diagnostic assessment of a deep learning system for detecting atrial fibrillation in pulse waveforms.  Heart. 2018;104(23):1921-1928. doi:10.1136/heartjnl-2018-313147PubMedGoogle ScholarCrossref
×