Estimating the Burden of Influenza-like Illness on Daily Activity at the Population Scale Using Commercial Wearable Sensors | Geriatrics | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Overview of Data Collection and Analysis
Overview of Data Collection and Analysis

ID indicates identification; and ILI, influenza-like illness.

Figure 2.  Mean Step Differentials and Histogram of Total Influenza-Like Illness (ILI) Burden Across Participants Measured in Lost Steps
Mean Step Differentials and Histogram of Total Influenza-Like Illness (ILI) Burden Across Participants Measured in Lost Steps

A, Mean step differentials during ILI events and matched control periods. The step differentials are the mean across participants of the per-participant difference between the estimated counterfactual step counts and the observed step counts. Shaded areas indicate SE above and below the line. B, Total steps lost during days 0 to 10 during ILI events and matched control periods.

Figure 3.  Association of Confounder Variables With the Mean Step Differentials
Association of Confounder Variables With the Mean Step Differentials

Shaded areas indicate SE for each group. GED indicates General Educational Development certification; and ILI, influenza-like illness.

Figure 4.  Mean Step Differentials Between Participants With Influenza-like Illness (ILI) and Controls by Symptom Experienced
Mean Step Differentials Between Participants With Influenza-like Illness (ILI) and Controls by Symptom Experienced

Shaded areas indicate SE for each group.

Table.  Estimated Total Burden and Maximal Burden on Day 1 of ILI in Terms of Steps Lost for the Whole US Population
Estimated Total Burden and Maximal Burden on Day 1 of ILI in Terms of Steps Lost for the Whole US Population
1.
World Health Organization. WHO coronavirus (COVID-19) dashboard. Accessed February 1, 2022. https://covid19.who.int/
2.
Kochanek  KD, Xu  JQ, Arias  E. Mortality in the United States, 2019.   NCHS Data Brief. 2020;(395):1-8. PubMed
3.
Garten  R, Blanton  L, Elal  AIA,  et al.  Update: influenza activity in the United States during the 2017-18 season and composition of the 2018-19 influenza vaccine.   MMWR Morb Mortal Wkly Rep. 2018;67(22):634-642. doi:10.15585/mmwr.mm6722a4PubMedGoogle ScholarCrossref
4.
Tsai  Y, Zhou  F, Kim  IK.  The burden of influenza-like illness in the US workforce.   Occup Med (Lond). 2014;64(5):341-347. doi:10.1093/occmed/kqu022PubMedGoogle ScholarCrossref
5.
Bilcke  J, Coenen  S, Beutels  P.  Influenza-like-illness and clinically diagnosed flu: disease burden, costs and quality of life for patients seeking ambulatory care or no professional care at all.   PLoS One. 2014;9(7):e102634. doi:10.1371/journal.pone.0102634PubMedGoogle ScholarCrossref
6.
Houts  CR, Patrick-Lake  B, Clay  I, Wirth  RJ.  The path forward for digital measures: suppressing the desire to compare apples and pineapples.   Digit Biomark. 2020;4(suppl 1):3-12. doi:10.1159/000511586PubMedGoogle ScholarCrossref
7.
Konty  KJ, Bradshaw  B, Ramirez  E, Lee  WN, Signorini  A, Foschini  L.  Influenza surveillance using wearable mobile health devices.   Online J Public Health Inform. 2019;11(1):e249. doi:10.5210/ojphi.v11i1.9758Google ScholarCrossref
8.
Shapiro  A, Marinsek  N, Clay  I,  et al.  Characterizing COVID-19 and influenza illnesses in the real world via person-generated health data.   Patterns (N Y). 2020;2(1):100188. doi:10.1016/j.patter.2020.100188PubMedGoogle ScholarCrossref
9.
Radin  JM, Wineinger  NE, Topol  EJ, Steinhubl  SR.  Harnessing wearable device data to improve state-level real-time surveillance of influenza-like illness in the USA: a population-based study.   Lancet Digit Health. 2020;2(2):e85-e93. doi:10.1016/S2589-7500(19)30222-5PubMedGoogle ScholarCrossref
10.
Hunter  J, Kainkaryam  R, Drysdale  E,  et al. Using wearables for influenza-like illness detection: the importance of design. Presented at: Machine Learning for Mobile Health Workshop, 33rd Annual Conference on Neural Information Processing System (NeurIPS 2020); December 12, 2020; Vancouver, British Columbia, Canada.
11.
Centers for Disease Control and Prevention. Estimated flu-related illnesses, medical visits, hospitalizations, and deaths in the United States—2018-2019 flu season. Accessed October 5, 2021. https://www.cdc.gov/flu/about/burden/2018-2019.html
12.
Evidation Health Inc. Evidation. Accessed October 5, 2021. https://my.evidation.com/
13.
Deering  S, Grade  MM, Uppal  JK,  et al.  Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.   JMIR Res Protoc. 2019;8(1):e10974. doi:10.2196/10974PubMedGoogle ScholarCrossref
14.
Kumar  S, Tran  J, Lee  WN, Bradshaw  B, Foschini  L, Juusola  J.  Longitudinal data from activity trackers show that those with greater inconsistency in activity levels are more likely to develop more severe depression.   Value Health. 2018;21:S191. doi:10.1016/j.jval.2018.04.1282Google ScholarCrossref
15.
Budd  A, Blanton  L, Grohskopf  L,  et al. Influenza. In: Roush  SW, Baldy  LM, Kirkconnell-Hall  MA, eds.  Manual for the Surveillance of Vaccine-Preventable Diseases. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases; 2017.
16.
Wasserstein  RL, Schirm  AL, Lazar  NA.  Moving to a world beyond “p < 0.05”.   Am Stat. 2019;73:S1-S19. doi:10.1080/00031305.2019.1583913Google ScholarCrossref
17.
Wasserstein  RL, Lazar  NA.  The ASA’s statement on P values: context, process, and purpose.   Am Stat. 2016;70:129-133. doi:10.1080/00031305.2016.1154108Google ScholarCrossref
18.
Bates  D, Mächler  M, Bolker  B, Walker  S.  Fitting linear mixed-effects models using lme4.   J Stat Softw. 2015;67:1-48. doi:10.18637/jss.v067.i01Google ScholarCrossref
19.
Stuart  EA, Lee  BK, Leacy  FP.  Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research.   J Clin Epidemiol. 2013;66(8)(suppl):S84-S90, 90.e1. doi:10.1016/j.jclinepi.2013.01.013PubMedGoogle ScholarCrossref
20.
Smith  J, Sweetman  A.  Estimating the causal effects of policies and programs.   Can J Econ. 2016;49:871-905. doi:10.1111/caje.12217Google ScholarCrossref
21.
Mezlini  AM, Das  S, Goldenberg  A.  Finding associations in a heterogeneous setting: statistical test for aberration enrichment.   bioRxiv. Preprint posted online March 25, 2020. doi:10.1101/2020.03.23.002972Google Scholar
22.
United States Census Bureau. Educational attainment in the United States: 2019: table 1. educational attainment of the population 18 years and over, by age, sex, race, and Hispanic origin: 2019. March 30, 2020. Accessed October 5, 2021. https://www.census.gov/content/census/en/data/tables/2019/demo/educational-attainment/cps-detailed-tables.html
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 4,124
    Citations 0
    Original Investigation
    Infectious Diseases
    May 12, 2022

    Estimating the Burden of Influenza-like Illness on Daily Activity at the Population Scale Using Commercial Wearable Sensors

    Author Affiliations
    • 1Evidation Health Inc, Menlo Park, California
    • 2Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle
    JAMA Netw Open. 2022;5(5):e2211958. doi:10.1001/jamanetworkopen.2022.11958
    Key Points

    Question  How can the true burden of influenza-like illnesses (ILIs) be estimated given that most cases of ILIs are mild and go undocumented ?

    Findings  This cohort study of 15 122 adults who reported ILI symptoms and had data from wearable sensors at symptom onset found an overall reduction in mobility equivalent to 15% of the active US population becoming completely immobilized for 1 day. More than 60% of this reduction occurred among persons who had sought no medical care.

    Meaning  This study suggests that the burden of ILIs is much greater than had previously been understood.

    Abstract

    Importance  The severity of viral infections can vary widely, from asymptomatic cases to complications leading to hospitalizations and death. Milder cases, despite being more prevalent, often go undocumented, and their public health burden is not accurately estimated.

    Objective  To estimate the true burden of influenza-like illness (ILI) in the US population using a surrogate measure of daily steps lost as measured by commercial wearable sensors.

    Design, Setting, and Participants  This cohort study modeled data from 15 122 US adults who reported ILI symptoms during the 2018-2019 influenza season (before the COVID-19 pandemic) and who had a sufficient density of wearable sensor data at symptom onset. Participants’ minute-level step data as measured by commercial wearable sensors were collected from October 1, 2018, through June 30, 2019. Minute-level activity time series were transformed into day-level time series per user, indicating the total number of steps daily.

    Main Outcomes and Measures  The primary end point was the number of steps lost during the period of 4 days before symptom onset (the latent phase) through 11 days after symptom onset (the symptomatic phase). The association between covariates and steps lost during this interval was also examined.

    Results  Of the 15 122 participants in this study, 13 108 (86.7%) were women, and the median age was 32 years (IQR, 27-38 years). For their ILI event, 2836 of 15 080 participants (18.8%) sought medical attention, and only 61 (0.4%) were hospitalized. Over the course of an ILI lasting 10 days, the mean cumulative loss was 4437 steps (95% CI, 4143-4731 steps). After weighting, there was an estimated overall nationwide reduction in mobility equivalent to 255.2 billion steps (95% CI, 232.9-277.6 billion steps) lost because of ILI symptoms during the study period. This finding reflects significant changes in routines, mobility, and employment and is equivalent to 15% of the active US population becoming completely immobilized for 1 day. Moreover, 60.6% of this reduction in steps (154.6 billion steps [95% CI, 138.1-171.2 billion steps]) occurred among persons who sought no medical care. Age and educational level were positively associated with steps lost.

    Conclusions and Relevance  These findings suggest that most of the burden of ILI in this study would have been invisible to health care and public health reporting systems. This approach has applications for public health, health care, and clinical research, from estimating costs of lost productivity at population scale, to measuring effectiveness of anti-ILI treatments, to monitoring recovery after acute viral syndromes such as during long COVID-19.

    Introduction

    Nearly 2 years into the COVID-19 pandemic, SARS-CoV-2 has infected more than 370 million people worldwide and caused more than 5.64 million deaths.1 However, the global burden of the pandemic is suspected to be much higher because milder infections, the most prevalent, are more likely to go unreported. This type of underreporting is not specific to COVID-19; it also applies to other viral infections, such as influenza-like illness (ILI).

    Influenza-like illness is highly prevalent in the US and in the rest of the world. Every year, ILI is among the leading causes of mortality, severely affecting the youngest and the oldest individuals.2 Even beyond the severe cases, however, ILI places a considerable burden on society. Previous work has measured this burden in terms of quality-adjusted life-years, financial costs, work productivity, and/or absenteeism,3,4 but these measures often rely on self-reported estimates of varying quality. Furthermore, an important portion of that burden might remain unobserved because milder cases of ILI can be underrepresented or overlooked in studies that recruit participants solely from clinical care institutions or that analyze medical records or claims data.5

    Commercial wearable sensors provide unique opportunities to understand objective behavioral and physiological changes associated with respiratory infections such as ILI and COVID-19 in clinical settings. Participants who consent to data sharing can allow for the unobtrusive and continuous collection of data without the burden of manual, subjective, self-reported input regarding the severity of disease. By their nature of being continuously produced over time (such as hour-level or day-level step counts), these data can complement measures of total disease burden and be used to study the dynamics of the burden over the course of the disease.6 Previous studies have shown the ability of wearable sensor data to track ILI-associated behavioral and physiological changes in activity (such as resting heart rate, sleep, and step counts),7,8 reflect statewide ILI prevalence,9 and even forecast future ILI events at the individual level based solely on abnormal activity data.10

    We used large-scale wearable sensor data to quantify the burden of ILI at a national level. Specifically, we studied a cohort corresponding to a large population of individuals from whom both wearable sensor data and frequent ILI survey data were collected. We first present a method for assessing ILI burden at the individual level and then reweight the resulting estimates to reflect the US population, using estimates of ILI burden published by the US Centers for Disease Control and Prevention (CDC).11 Our aims were to assess whether the burden of ILI increases with markers of increased disease severity (medical visits, hospitalization, and symptoms) and to explore associations between covariates and the ILI burden trajectory from onset to recovery.

    Methods
    Data Collection

    The Achievement application (Evidation Health Inc) is a mobile consumer platform that encourages users to develop healthy habits—such as walking, meditating, and logging meals—and provides incentives for them to participate in research by completing surveys and sharing their data from commercial-grade wearable sensors.12-14 Since 2017, the application has been used for voluntary monitoring of annual waves of ILI virus infections.7

    We developed an ILI survey and administered it during the 2018-2019 influenza season (eFigure 2 in the Supplement). Every 2 weeks, surveys queried members as to whether they had had any ILI symptoms in the preceding 14 days. Members who reported symptoms were then asked to describe specific symptoms that they experienced, the date of symptom onset, the recovery date (if available), health care–seeking behavior, diagnosis of ILI, lost workdays, household illness, and vaccination history days.

    We also collected minute-level step data from October 1, 2018, through June 30, 2019, from participants who agreed to link their wearable sensor data to the Achievement platform. Minute-level activity time series were transformed into day-level time series per user, indicating the total number of steps daily. Several quality-control criteria were applied to ensure that analyses included only participants using a wearable sensor who had dense activity data coverage throughout the influenza season (particularly the days surrounding the ILI) and only participant-days with sufficient sensor wear time (eAppendix in the Supplement).

    The survey also captured the demographic characteristics of age, sex, race and Hispanic ethnicity, and educational level achieved. This study received expedited review and independent institutional review board approval from the Solutions institutional review board after data collection. This institutional review board waived the requirement to reobtain informed consent from participants owing to the retrospective design of the study. The data used by the investigators were already deidentified.

    Statistical Analysis

    Our data set enabled robust characterization of the time course of behavior during ILIs (Figure 1). Participants’ self-reported descriptions of ILI events (survey responses) were paired with their activity time series, passively recorded from their wearable fitness tracker. Influenza-like illness onset dates were used to index the activity time series, with ILI day 0 marking self-reported symptom onset. There were 3 time periods of interest:

    1. The latent phase: ILI days −4 through −1. The incubation period of the ILI is 1 to 4 days, according to the CDC.15

    2. The symptomatic phase: ILI days 0 to 11. This choice was based on our data, in which the last day with a statistically significant16,17 change in steps was noted on day 10.

    3. The baseline, or healthy, phase: the remaining days in the time series.

    In addition, we created matching control events. Each individual had 1 ILI event and 1 control event whose dates corresponded to the ILI event of another randomly sampled participant, which ensured that the distribution of control event dates matched the distribution of ILI dates.

    We fitted a longitudinal multilevel regression model (LME4 package for R, version 3.60 [R Group for Statistical Computing]18) to days during the baseline period. The model was trained on all data excluding ILI and control periods (days −4 to 11 around each ILI event). This baseline model aimed to quantify typical healthy behavior by accounting for both group-level trends (eg, on average, people walk less during winter months than in the spring and fall) and individual differences between participants (eg, weekly step activity patterns differ between office workers and hospital nurses). Fixed effects were specified for each week of the year separately (as a categorical variable), which captured seasonal behavioral patterns at the group level. We also specified random effects (participant level) for a 5° polynomial fitting the week of the year and a random intercept (participant level) for each day of the week to account for individual differences in weekly activity patterns.

    Next, the baseline model was used to estimate each participant’s behavior for all days of the influenza season, including all 3 time periods of interest. Multilevel models make unique covariates for each individual. The covariates obtained are analogous to counterfactuals19 (ie, the number of steps for each individual if they had not had an ILI event). We then computed step differentials as the difference between the observed activity and the estimated activity for each participant-day. These step differentials corresponded to the association of the ILI with activity level for each participant-day.

    We were interested in the mean step differentials across participants, which are akin to the mean treatment effect in the causal inference literature.20 The baseline model mean absolute error was 2375 steps per day for the baseline period it was trained on and 2477 steps per day for the unseen control periods.

    We also calculated the step differentials between the ILI and control periods (days −4 to 11) per individual. These differentials allowed us to estimate the ILI burden on each individual in our data for each day and to verify that the baseline model covariates contained no observable bias. Statistically significant differences in step differentials between cases and controls were assessed by means of the Wilcoxon signed rank test, the t test, and a novel statistical test for heterogeneous effects.21 All tests were 2-tailed; statistical significance was assessed after correction for multiple hypotheses (at false discovery rate–corrected P < .05). We also assessed the potential association of covariates, such as age, sex, race and Hispanic ethnicity, and disease symptoms, with ILI burden.

    Because our sample of participants was not representative of the overall US population (ie, it was not a random probability sample and differed notably from the US population in distributions for sex and age), we attempted to adjust our quantification of ILI burden through poststratification. We reweighted our sample to match the relative proportions and absolute size of the US population by age group, sex, race and Hispanic ethnicity, and educational level, as reported by the US Census.22

    For each day of the ILI period, we used a linear model with the step differentials as outcomes and the demographic factors used for restratification (age, sex, race and Hispanic ethnicity, and educational level), along with the CDC’s disease severity levels (attended medical care and hospitalized) as covariates. Using this model, we could estimate the ILI burden for each group or stratum corresponding to every possible combination of covariates. Within each group, the burden was summed over ILI days 0 to 10 (the only days showing statistically significant differences between cases and controls). These burden covariates were then reweighted by the US Census estimates of population size per demographic category and CDC estimates of ILI prevalence among demographic groups, and finally they were summed to create an estimate of the total US ILI burden. This procedure of estimating the national ILI burden was repeated on the control periods for comparison, and 95% CIs were estimated by bootstrapping (100 times) (see the eAppendix in the Supplement for additional validation details).

    Finally, we tested whether any demographic covariates—such as age category, sex, race and ethnicity category, educational level category, medical attention (yes or no), and hospitalization (yes or no)—were associated with the daily or total ILI burden. We fitted a regression model with the daily or total ILI burden as an outcome and the covariates as covariates. We also performed univariate tests and visually inspected the association of each variable separately.

    Results
    Participants

    After applying the selection criteria, demographic information, daily steps data from wearable sensors, and ILI survey data were available for 15 122 participants. Participants were a median age of 32 years (IQR, 27-38 years), 13 108 (86.7%) were women, 2014 (13.3%) were men, 13 140 (86.9%) were White, and nearly all (14 244 of 14 339 [99.3%]) had at least a high school diploma (eTable 1 in the Supplement). For their ILI event, 2836 of 15 080 participants (18.8%) sought medical attention, and only 61 (0.4%) were hospitalized.

    Baseline Modeling

    We trained the linear mixed model on all data except the ILI periods and the matching control periods. We then used the model to estimate the step count for each individual for each day of the ILI and control periods.

    Figure 2A shows the mean differences between the estimated and observed step counts. As expected, the step differential mean values were near zero for control periods and the days before symptoms in the ILI period (indicating typical healthy behavior). In contrast, the step differential mean values deviated substantially from zero during the symptomatic ILI periods, showing the expected abnormal behavior during the ILI event. The difference in step differentials between case and control periods was statistically significant only for days 0 to 10 of symptom onset, with the step differentials that corresponded to an ILI event being negative and lower than those corresponding to control periods. The mean outcome of an ILI event was thus a reduction in the number of steps on days 0 to 10 of symptom onset. There was no significant difference between the groups in number of steps during days −4 to −1 (the latent phase) or on day 11 and beyond.

    Despite the association between the ILI event and steps overall, there was a great deal of heterogeneity (Figure 2B); 41% of participants (6229 of 15 122) showed little to no decrease (<1000 steps lost) or an overall increase in steps during days 0 to 10 of their ILI event; this proportion was 53% (7073 of 13 460) for the control periods. However, when we focused only on the examples with more than 15 000 steps lost, such cases represented 23% (3442 of 15 122) of the ILI examples and only 12% (1671 of 13 460) of the control examples.

    Overall ILI Burden

    Across our sample, a mean of 924 steps (95% CI, 861-988 steps) were lost on the worst day of the ILI event, corresponding to day 1 after symptom onset (Figure 2A). In comparison, the estimated loss during the control periods did not change significantly (−34 steps lost [95% CI, −95 to 26 steps lost]). Over the course of the ILI (days 0-10), the mean cumulative loss was 4437 steps (95% CI, 4143-4731 steps). For control periods, the burden was closer to zero, with −41 steps lost (95% CI, −312 to 229 steps lost).

    We reweighted the sample using self-reported age, sex, race and Hispanic ethnicity, and educational level to approximate the overall US population aged 18 years or older. Using this approach, we estimated that a total of 255.2 billion steps (95% CI, 232.9-277.6 billion steps) were lost during ILI events in the US during the study period (Table; eTable 2 in the Supplement). This finding reflects significant changes in routines, mobility, and employment and is equivalent to 15% of the active US population becoming completely immobilized for 1 day. The corresponding analysis for the matched control periods yielded no significant change (3.7 billion steps lost [95% CI, −25.4 to 32.8 billion steps lost]). Of the 255.2 billion steps lost owing to ILI, 60.6% (154.6 billion steps [95% CI, 138.1-171.2 billion steps]) occurred among individuals who had not sought medical attention.

    Association of Covariates With ILI Burden

    In both univariate tests and logistic regression analyses, neither sex nor race and ethnicity had an association with ILI burden, whereas age and educational level did. As shown in Figure 3A and C, the ILI step burden increased with age and with the level of education.

    Participants who were seen by a physician had a higher ILI burden compared with the rest of the sample (Figure 3B and D). This finding would be expected, as more severe ILI episodes likely led to requests for medical assessment. This finding was true whether the influenza diagnosis was positive or negative. Similarly, participants who reported being hospitalized had a higher ILI burden, although the SE was large, given the small number of reported hospitalizations (n = 61).

    Influenza-like illness burden was also associated with both the number of workdays missed and self-reported overall health (Figure 3E and F). As expected, participants who took more time off lost more steps than those who reported working through their illness. Participants who self-reported very good to excellent health had a higher number of steps at baseline and lost more steps during ILI than participants with poorer health.

    Finally, we investigated the association of different symptoms with the trajectory of ILI burden. In Figure 4, we show 4 different patterns of association between symptoms and disease trajectory (see eFigure 1 in the Supplement for the full list of symptoms). Participants reporting nausea or vomiting had a higher burden (more steps lost) in the initial phase of the ILI (days 0-2) but converged to the same level of step differential in the remaining days compared with participants who did not report nausea or vomiting. In contrast, participants reporting a sore throat exhibited a lower burden (less steps lost) on the first 2 days; however, they exhibited a higher burden on days 3 to 10 and a slower recovery compared with participants without a sore throat. Participants who reported headache had a higher burden across the duration of the disease. Finally, participants who reported shortness of breath showed a similar mean burden on days 0 to 2 but a higher burden on days 3 to 10 (slower recovery).

    Discussion

    We present an approach for quantifying the population burden of ILI based on changes in daily activities obtained from wearable sensors linked to survey responses from the 2018-2019 influenza season. Our results show that wearable sensors can be used to estimate the burden of ILI at the individual and national levels in terms of the number of steps taken daily. We additionally investigated how different demographic covariates might modify this ILI burden and how different symptoms are associated with different trajectories of ILI burden throughout the course of the disease.

    The measure of burden used in this study was the mean number of steps lost. This is a directly quantifiable, objective measure with passively collected data by wearable sensors. This measure has an advantage compared with existing approaches that rely more on self-reported estimates of ILI burden, such as quality of life–based and productivity surveys.3,4 Our approach also included medically unattended cases, which are ignored by measures relying on health care institutions or claims data. In addition, our measure of ILI burden captures the disease burden among more segments of the population uncounted in work-centric measures, including unemployed and retired persons. However, if the goal is to measure loss of productivity due to ILI, then our measure can be seen as an imperfect proxy. For example, a proportion of our participants did not lose a meaningful number of steps and did not take time off from work despite the ILI. For such participants, our approach would not adequately assess the real burden of ILI in terms of productivity or quality of life. This finding will be especially true in a post–COVID-19 world, where remote work is a possibility for many occupations.

    Some of the associations found in this study are difficult to interpret with the data available. For example, we observed that the ILI burden was correlated with the level of education. This finding might be explained by reduced job precarity among the more highly educated and the ability to take days off from work to recover at home, leading to a larger reduction in the number of steps taken. The ILI burden was also correlated with time off from work, which can be interpreted in 2 ways: either participants with milder disease presentation did not need to take time off from work, or using steps to measure the ILI burden underestimates illness severity among participants who are less inclined to take time off from work because of socioeconomic reasons.

    A potential improvement on this work in the future would be to refine our baseline model estimating step counts. We plan to explore the use of deep learning and of individual-level models considering autocorrelation to potentially achieve less noisy covariates of baseline steps. A better estimate of activity in the absence of ILI will allow us to better quantify its effects. The individual burden of ILI can be associated with many other factors that we did not collect data for (eg, specific occupations and chronic conditions). Modeling those factors can further refine our models and lead to a more accurate measure of the individual and population ILI burden.

    Limitations

    This study has the following limitations: the accuracy limitations of the currently used models, the inability to clearly interpret the association between our burden and education or work productivity loss, and potentially unmeasured confounders (such as socioeconomic situation) that could bias our estimations. Further studies on independent data sets will be valuable in confirming our estimates, especially if these data sets are more representative and require less reweighting.

    Conclusions

    In this study, an estimated 255.2 billion steps were lost because of ILI symptoms, representing 15% of the active US population being immobilized for 1 day. A similar approach to that taken in this study could be used in many settings. In public health settings, this measure of burden could be used to measure the effect of other viral infections, such as COVID-19, and to monitor over time the effectiveness of policies and interventions to reduce their spread. In health care settings, the same approach could also be applied to quantifying the burden of other chronic diseases, including COVID-19 postviral syndrome (long COVID-19) or seasonal disorders. Finally, with adequate controlling for confounding factors, the approach could be used in clinical research to measure effectiveness of therapies.

    Back to top
    Article Information

    Accepted for Publication: March 15, 2022.

    Published: May 12, 2022. doi:10.1001/jamanetworkopen.2022.11958

    Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2022 Mezlini A et al. JAMA Network Open.

    Corresponding Author: Aziz Mezlini, PhD, Evidation Health Inc, 2-113 Kenwood Ave, York, ON M6C2S4, Canada (amezlini@evidation.com).

    Author Contributions: Drs Mezlini and Shapiro had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Mezlini and Shapiro are co–first authors.

    Concept and design: Mezlini, Shapiro, Caddigan, Ramirez, Foschini.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Mezlini, Shapiro, Caddigan, Althoff, Foschini.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Mezlini, Shapiro, Daza, Caddigan, Althoff, Foschini.

    Obtained funding: Foschini.

    Administrative, technical, or material support: Shapiro, Caddigan, Ramirez, Foschini.

    Supervision: Ramirez, Foschini.

    Conflict of Interest Disclosures: Drs Mezlini, Shapiro, Daza, Caddigan, Ramirez, and Foschini are employees of Evidation Health Inc, developer of the Achievement health and research platform. No other disclosures were reported.

    Funding/Support: This work was supported by Evidation Health Inc.

    Role of the Funder/Sponsor: Evidation Health Inc had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We thank Catherine Wang, BS, previously with Evidation Health Inc, Benjamin Bradshaw, BS, Evidation Health Inc, Andrew Trister, MD, PhD, Gates Foundation, Kevin Konty, PhD, New York City Department of Health, and Yuedong Wang, PhD, University of California–Santa Barbara, for their scientific guidance and manuscript and analysis reviews. They were not compensated for their contributions.

    Additional Information: The data sets analyzed in this study are not publicly available but can be shared for scientific collaboration by contacting the corresponding author.

    References
    1.
    World Health Organization. WHO coronavirus (COVID-19) dashboard. Accessed February 1, 2022. https://covid19.who.int/
    2.
    Kochanek  KD, Xu  JQ, Arias  E. Mortality in the United States, 2019.   NCHS Data Brief. 2020;(395):1-8. PubMed
    3.
    Garten  R, Blanton  L, Elal  AIA,  et al.  Update: influenza activity in the United States during the 2017-18 season and composition of the 2018-19 influenza vaccine.   MMWR Morb Mortal Wkly Rep. 2018;67(22):634-642. doi:10.15585/mmwr.mm6722a4PubMedGoogle ScholarCrossref
    4.
    Tsai  Y, Zhou  F, Kim  IK.  The burden of influenza-like illness in the US workforce.   Occup Med (Lond). 2014;64(5):341-347. doi:10.1093/occmed/kqu022PubMedGoogle ScholarCrossref
    5.
    Bilcke  J, Coenen  S, Beutels  P.  Influenza-like-illness and clinically diagnosed flu: disease burden, costs and quality of life for patients seeking ambulatory care or no professional care at all.   PLoS One. 2014;9(7):e102634. doi:10.1371/journal.pone.0102634PubMedGoogle ScholarCrossref
    6.
    Houts  CR, Patrick-Lake  B, Clay  I, Wirth  RJ.  The path forward for digital measures: suppressing the desire to compare apples and pineapples.   Digit Biomark. 2020;4(suppl 1):3-12. doi:10.1159/000511586PubMedGoogle ScholarCrossref
    7.
    Konty  KJ, Bradshaw  B, Ramirez  E, Lee  WN, Signorini  A, Foschini  L.  Influenza surveillance using wearable mobile health devices.   Online J Public Health Inform. 2019;11(1):e249. doi:10.5210/ojphi.v11i1.9758Google ScholarCrossref
    8.
    Shapiro  A, Marinsek  N, Clay  I,  et al.  Characterizing COVID-19 and influenza illnesses in the real world via person-generated health data.   Patterns (N Y). 2020;2(1):100188. doi:10.1016/j.patter.2020.100188PubMedGoogle ScholarCrossref
    9.
    Radin  JM, Wineinger  NE, Topol  EJ, Steinhubl  SR.  Harnessing wearable device data to improve state-level real-time surveillance of influenza-like illness in the USA: a population-based study.   Lancet Digit Health. 2020;2(2):e85-e93. doi:10.1016/S2589-7500(19)30222-5PubMedGoogle ScholarCrossref
    10.
    Hunter  J, Kainkaryam  R, Drysdale  E,  et al. Using wearables for influenza-like illness detection: the importance of design. Presented at: Machine Learning for Mobile Health Workshop, 33rd Annual Conference on Neural Information Processing System (NeurIPS 2020); December 12, 2020; Vancouver, British Columbia, Canada.
    11.
    Centers for Disease Control and Prevention. Estimated flu-related illnesses, medical visits, hospitalizations, and deaths in the United States—2018-2019 flu season. Accessed October 5, 2021. https://www.cdc.gov/flu/about/burden/2018-2019.html
    12.
    Evidation Health Inc. Evidation. Accessed October 5, 2021. https://my.evidation.com/
    13.
    Deering  S, Grade  MM, Uppal  JK,  et al.  Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.   JMIR Res Protoc. 2019;8(1):e10974. doi:10.2196/10974PubMedGoogle ScholarCrossref
    14.
    Kumar  S, Tran  J, Lee  WN, Bradshaw  B, Foschini  L, Juusola  J.  Longitudinal data from activity trackers show that those with greater inconsistency in activity levels are more likely to develop more severe depression.   Value Health. 2018;21:S191. doi:10.1016/j.jval.2018.04.1282Google ScholarCrossref
    15.
    Budd  A, Blanton  L, Grohskopf  L,  et al. Influenza. In: Roush  SW, Baldy  LM, Kirkconnell-Hall  MA, eds.  Manual for the Surveillance of Vaccine-Preventable Diseases. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases; 2017.
    16.
    Wasserstein  RL, Schirm  AL, Lazar  NA.  Moving to a world beyond “p < 0.05”.   Am Stat. 2019;73:S1-S19. doi:10.1080/00031305.2019.1583913Google ScholarCrossref
    17.
    Wasserstein  RL, Lazar  NA.  The ASA’s statement on P values: context, process, and purpose.   Am Stat. 2016;70:129-133. doi:10.1080/00031305.2016.1154108Google ScholarCrossref
    18.
    Bates  D, Mächler  M, Bolker  B, Walker  S.  Fitting linear mixed-effects models using lme4.   J Stat Softw. 2015;67:1-48. doi:10.18637/jss.v067.i01Google ScholarCrossref
    19.
    Stuart  EA, Lee  BK, Leacy  FP.  Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research.   J Clin Epidemiol. 2013;66(8)(suppl):S84-S90, 90.e1. doi:10.1016/j.jclinepi.2013.01.013PubMedGoogle ScholarCrossref
    20.
    Smith  J, Sweetman  A.  Estimating the causal effects of policies and programs.   Can J Econ. 2016;49:871-905. doi:10.1111/caje.12217Google ScholarCrossref
    21.
    Mezlini  AM, Das  S, Goldenberg  A.  Finding associations in a heterogeneous setting: statistical test for aberration enrichment.   bioRxiv. Preprint posted online March 25, 2020. doi:10.1101/2020.03.23.002972Google Scholar
    22.
    United States Census Bureau. Educational attainment in the United States: 2019: table 1. educational attainment of the population 18 years and over, by age, sex, race, and Hispanic origin: 2019. March 30, 2020. Accessed October 5, 2021. https://www.census.gov/content/census/en/data/tables/2019/demo/educational-attainment/cps-detailed-tables.html
    ×