Telemedicine intensive care unit (tele-ICU) mortality literature search flow diagram. LOS indicates length of stay.
Effect of telemedicine intensive care unit (ICU) coverage on ICU mortality. Weights are calculated from random-effects analysis. CI indicates confidence interval; OR, odds ratio.
Effect of telemedicine intensive care unit coverage on in-hospital mortality. Weights are calculated from random-effects analysis. CI indicates confidence interval; OR, odds ratio.
Cumulative influence of study on meta-analysis of intensive care unit mortality. Horizontal lines depict 95% confidence intervals; vertical dashed line, the pooled odds ratio of 0.80.
Effect of telemedicine intensive care unit (ICU) coverage on ICU length of stay (LOS). Weights are calculated from random-effects analysis. CI indicates confidence interval; SMD, standardized mean difference.
Effect of telemedicine intensive care unit coverage on hospital length of stay (LOS). Weights are calculated from random-effects analysis. CI indicates confidence interval; SMD, standardized mean difference.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Young LB, Chan PS, Lu X, Nallamothu BK, Sasson C, Cram PM. Impact of Telemedicine Intensive Care Unit Coverage on Patient Outcomes: A Systematic Review and Meta-analysis. Arch Intern Med. 2011;171(6):498–506. doi:10.1001/archinternmed.2011.61
Although remote intensive care unit (ICU) coverage is rapidly being adopted to enhance access to intensivists, its effect on patient outcomes is unclear. We conducted a meta-analysis to examine the impact of telemedicine ICU (tele-ICU) coverage on mortality and length of stay (LOS).
We conducted a systematic review of studies published from January 1, 1950, through September 30, 2010, using PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Global Health, Web of Science, the Cochrane Library, and conference abstracts. We included studies that reported data on the primary outcomes of ICU and in-hospital mortality or on the secondary outcomes of ICU and hospital LOS.
We identified 13 eligible studies involving 35 ICUs. All the studies used a before-and-after design. The studies included 41 374 patients (15 667 pre–tele-ICU and 25 707 post–tele-ICU patients). Tele-ICU coverage was associated with a reduction in ICU mortality (pooled odds ratio, 0.80; 95% confidence interval [CI], 0.66-0.97; P = .02) but not in-hospital mortality for patients admitted to an ICU (pooled odds ratio, 0.82; 95% CI, 0.65-1.03; P = .08). Similarly, tele-ICU coverage was associated with a reduction in ICU LOS (mean difference, −1.26 days; 95% CI, −2.21 to −0.30; P = .01) but not hospital LOS (mean difference, −0.64; 95% CI, −1.52 to 0.25; P = .16).
Tele-ICU coverage is associated with lower ICU mortality and LOS but not with lower in-hospital mortality or hospital LOS.
Staffing of intensive care units (ICUs) with physicians specially trained to care for critically ill patients (ie, intensivists) is associated with improved survival.1-3 As a result, an array of organizations, including the Leapfrog Group, has promoted around-the-clock intensivist staffing.4,5 Advocates have estimated that as many as 50 000 lives and $4.3 billion might be saved annually in the United States through more consistent intensivist staffing.6,7 Because many hospitals lack the patient volume or financial resources to justify hiring dedicated intensivists and because of a shortage of trained intensivists,8-11 hospitals increasingly are adopting telemedicine ICU (tele-ICU) coverage to compensate for a lack of on-site intensivist expertise.
Tele-ICU coverage typically uses a combination of videoconferencing technology, telemetry, and an electronic medical record to allow off-site intensivists and/or critical care nurses to assist in the treatment of critically ill patients.12 Individual hospitals typically choose to implement tele-ICU in accordance with their needs and resources. The most comprehensive systems use a team of physicians and critical care nurses who continuously monitor patients' noninvasive and invasive hemodynamic factors, order laboratory and radiographic studies, initiate preventive treatments (eg, deep venous thrombosis prophylaxis), and aid in diagnosis and treatment plans.13-16 Although robust evidence supports the benefits of on-site intensivist staffing,1-3 the impact of tele-ICU coverage is less clear.17,18 Thorough evaluation of the impact of tele-ICU coverage is important given that installation of these systems can cost hospitals millions of dollars plus additional ongoing costs for upkeep.19 At the same time, an estimated 200 to 400 hospitals, under tremendous pressure to “do something” to improve intensivist coverage, have implemented tele-ICU coverage, and more than 1 million patients have already received care in hospitals equipped with these systems despite a lack of clear understanding of the benefits of this technology.20
Accordingly, we performed a systematic review and meta-analysis of the published and unpublished literature with the primary objective of examining the impact of tele-ICU coverage on ICU and in-hospital mortality. Our secondary objective was to evaluate the impact of these systems on ICU and in-hospital length of stay (LOS). We also examined key hospital, ICU, and patient factors that may mitigate or enhance the effectiveness of tele-ICU coverage. Finally, we lay out key gaps in the existing knowledge and directions for future research.
We conducted a systematic literature review in accordance with published guidelines.21,22 We identified studies of hospital inpatients that (1) evaluated the effect of tele-ICU coverage, (2) performed comparisons between a control and an intervention group, and (3) provided quantitative data on at least 1 of the primary outcomes (ICU or in-hospital mortality) in this review. Secondary outcomes were ICU and hospital LOS.
For the purposes of the review, we defined tele-ICU coverage as the application of telemedicine to hospital critical care units; this included any telecommunication system installed in the ICU to facilitate real-time access to critical care specialists (eg, intensivists or critical care nurses) located elsewhere.23-26 This definition covers a variety of tele-ICU models ranging from exclusively reactive systems, where tele-ICU staff was available via telephone on an as-needed consultative basis,27 to the more commonly used proactive systems, where patients were continuously monitored in real time via videoconferencing, telemetry, and access to the electronic medical record.28,29
With the assistance of a trained medical research librarian, we searched for relevant studies published from January 1, 1950, through September 30, 2010, using PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Global Health, Web of Science, and the Cochrane Library. The search was not limited to English-language articles and used keywords and Medical Subject Heading terms with a Boolean search strategy (appendix A; available from the authors by request). The initial search yielded 721 articles (Figure 1).
To identify unpublished studies,21,30 we conducted a keyword search of abstracts and presentations from relevant major scientific conferences from 2006 through 2010. In particular, we reviewed abstracts from the annual meetings of the American College of Cardiology, American College of Chest Physicians, American College of Emergency Physicians, American Telemedicine Association, American Thoracic Society, International Society for Pharmacoeconomics and Outcomes Research, and Society of Critical Care Medicine. The keyword search yielded an additional 2683 abstracts and presentations that were potentially relevant to our study.
Removal of duplicate reports resulted in the identification of 3133 unique studies of potential relevance. The titles and abstracts of each study were independently reviewed and assessed for eligibility by 2 of us (L.B.Y. and P.M.C.). We eliminated 2957 studies that did not pertain to tele-ICU coverage as defined for our purposes or did not report original research (eg, case reports and commentaries).
The remaining 176 studies (59 articles and 117 abstracts) were considered potentially eligible and reviewed in full. Studies were included if (1) they evaluated tele-ICU coverage, (2) they reported ICU or in-hospital mortality in a manner that allowed us to abstract the number of deaths and number of patients at risk for death, and (3) they provided mortality data for an ICU under tele-ICU coverage and for a control ICU. Controls could be similar ICUs concurrently lacking tele-ICU coverage or the same ICU before tele-ICU implementation. Disagreement about inclusion was resolved by discussion between the reviewers. In cases of multiple articles that appeared to present overlapping data from the same patient populations,27,31-40 we contacted the study authors and included data from the most comprehensive article while excluding the duplicate study. For 12 studies with incomplete data (eg, missing details on ICU characteristics or LOS data), we contacted the authors on at least 3 occasions during a 3-week period to request additional data. Authors of 9 studies27,29,32,37,40-44provided supplemental data sufficient for study inclusion, 1 was unable to share his data,45 and 1 did not respond.46 Ultimately, 13 studies that met the inclusion criteria were included in this analysis.
Data abstraction was independently performed by 2 of us (P.M.C. and L.B.Y.) using a prepiloted, standardized abstraction form (appendix B; available from the authors by request). For each study, we collected information on 57 variables of potential interest that covered issues of study design and setting, physical ICU and tele-ICU organization and function, and study outcomes. Key study design information included (1) whether the study authors were affiliated with a tele-ICU vendor, (2) number of ICUs and ICU beds in the study, (3) number of hospitals and ICU beds in the study, and (4) dates and duration of the control and intervention periods. Important variables related to ICU organization and function included (1) ICU type (eg, medical or surgical); (2) physical ICU staffing (eg, open to patient's attending physician or closed to physicians not on the ICU staff); (3) severity measure used and measured severity of patients' conditions; (4) intensivist availability at physical ICUs; (5) model of tele-ICU coverage (proactive, reactive, or mixed); (6) tele-ICU structure (a single consulting “hub” covering multiple ICU “spokes,” a “parent” hospital covering a “child” site, or a network of clinicians who monitor from multiple sites); and (7) tele-ICU hours of operation. We also abstracted data on the primary outcomes (ICU and in-hospital mortality) and the secondary outcomes (ICU and hospital LOS). More specifically, for each study we recorded the number of deaths and number of patients “at risk” for both the intervention and control groups; we abstracted analogous data on LOS.
Study quality was assessed using the Newcastle-Ottawa Scale for assessing the quality of nonrandomized studies in meta-analyses.47 The scale addresses 3 domains of quality (selection, comparability, and exposure), with scores ranging from 0 (lower quality) to 9 (higher quality). Two reviewers (L.B.Y. and P.M.C.) independently rated each study, and variations in ratings were reconciled via discussion.
The 4 distinct study outcomes included ICU mortality, in-hospital mortality, ICU LOS, and hospital LOS, with all studies providing data on ICU or in-hospital mortality (or both). It was our intention to examine unadjusted and adjusted outcomes reflecting the important role of patient severity in our analysis. However, the underlying studies relied on different methods of severity adjustment (eg, Simplified Acute Physiology Score, version II [SAPS] and Acute Physiology and Chronic Health Evaluation [APACHE]) and provided these data in varying degrees of detail. Thus, our analyses were based solely on unadjusted mortality and LOS. Likewise, it was our initial intention to examine outcomes (mortality and LOS) at the study and ICU level. However, although all studies reported at least 1 relevant outcome at the study level, only 8 studies reported outcomes at the ICU level. Thus, evaluation of the impact of tele-ICU coverage on mortality and LOS was conducted at the study level only.
All included studies used a before-and-after design; therefore, we examined the effect of tele-ICU coverage on ICU and in-hospital mortality using odds ratios (ORs) with 95% confidence intervals (CIs) compared with the preintervention period of no tele-ICU coverage. For ICU and hospital LOS, study results were expressed as standardized mean differences with 95% CIs. Since all studies reported at least 1 relevant outcome but few reported all 4 outcomes, analyses for each outcome were, by necessity, limited to the studies that reported that outcome. Meta-analysis for each outcome was performed using a DerSimonian and Laird random-effects model with inverse-variance weights.48 We evaluated between-study heterogeneity with I2 statistics49 and publication bias using the Begg test. We examined the cumulative influence of each study on the pooled estimate over calendar time and also the influence of each study on the overall pooled estimate by omitting each estimate one at a time.
We used several strategies recommended by the Cochrane Collaboration to address potential heterogeneity (eg, meta-regression and performance of subgroup analysis).50 In particular, we decided a priori to examine the impact of the monitoring system among (1) higher- and lower-quality studies, (2) studies with and without author affiliation with a tele-ICU vendor, (3) hospitals with and without intensivists on staff, (4) ICUs with and without an academic affiliation, and (5) open and closed ICUs. Although we were able to perform these analyses, many of the analyses in these subgroups continued to have high heterogeneity as assessed by the I2 statistic. Furthermore, few eligible studies were even within most of the subgroups of interest. This situation limited our confidence in the validity of these results, and therefore, we present subgroup analyses only according to author affiliation with a tele-ICU vendor and according to study quality. These results are principally presented for illustration and example. All statistical tests were 2 sided and were evaluated at a significance level of .05 because there were 2 primary end points. We used commercially available software (Stata, version 10.1; StataCorp, College Station, Texas) to conduct all analyses.
A total of 13 studies27-29,32,37,40-44,51-53 encompassing 35 ICUs (9 medical, 8 surgical, 8 mixed, 7 other, and 3 not specified) from 27 hospitals were included in our study (Figure 1 and Table 1). These studies included a total of 41 374 ICU patients (15 667 preintervention and 25 707 postintervention). Seven studies had been published,27-29,32,51-53 and 6 were conference abstracts.37,40-44 Although our screening criteria allowed for concurrent or preintervention control groups, all identified studies used a before-and-after design. Four studies had no ties to tele-ICU vendors,29,32,52,53 5 had clear industry affiliation,28,41,42,44,51 and 4 did not report this information.27,37,40,43 The mean study quality (Newcastle-Ottawa Scale) was moderate (median, 6; range, 3-9), and interrater reliability between the 2 reviewers was high (Cronbach α, 0.94).
There were notable differences in the settings where the studies were conducted and the manner in which remote monitoring was implemented (Table 2). For example, of the 35 individual ICUs examined in these 13 studies, 9 ICUs were within or affiliated with academic medical centers,28,29,40,51,53 18 were in community hospitals,27,32,37,42,53 and studies for the remaining 8 ICUs did not report presence or absence of academic affiliation. The ICUs where remote monitoring was implemented also differed with regard to the staffing models used. In particular, 16 ICUs were reported as open,28,32,42,52,53 8 were reported as closed or mixed,29,44,51,53 and the remaining 11 were not characterized.27,32,37,40,41,43
There were important differences in how tele-ICU monitoring was used. Thirteen ICUs received around-the-clock monitoring,28,32,43,53 11 were monitored primarily during evenings and weekends,29,42,51 and 1 used remote monitoring on an “as needed” basis27; the remaining 10 ICUs did not report the hours for their tele-ICU monitoring.37,40,41,44,52
Twelve of the 13 studies, encompassing 34 ICUs, reported ICU mortality among 40 541 patients (15 311 preintervention and 25 230 postintervention). Collectively, implementation of tele-ICU coverage was associated with a significant reduction in ICU mortality (pooled OR, 0.80; 95% CI, 0.66-0.97; P = .02) (Figure 2). There were 10 studies encompassing 30 ICUs that reported data on in-hospital mortality among 32 575 patients (13 574 preintervention and 19 001 postintervention). Pooled analysis revealed that tele-ICU monitoring was not associated with a statistically significant reduction in in-hospital mortality for patients admitted to an ICU (pooled OR, 0.82; 95% CI, 0.65-1.03; P = .08) (Figure 3).
As expected, statistical heterogeneity was significant (P < .001) and considerable among studies for ICU mortality (I2 = 77.1%) and in-hospital mortality (I2 = 84.9%). However, there was no evidence of publication bias as assessed by the Begg test (P = .58 for ICU mortality; P = .42 for in-hospital mortality). As each study was added in chronological order, the cumulative effect on the pooled results moved toward the null for ICU mortality (Figure 4) and in-hospital mortality (results not shown).
Of the 13 studies included in our analysis, ICU LOS was reported for 7 studies (18 ICUs) representing 14 395 patients, and hospital LOS was examined in 6 studies (17 ICUs) representing 14 232 patients. Implementation of tele-ICU coverage was associated with a significant reduction in ICU LOS (mean reduction of 1.26 days; 95% CI, −2.21 to −0.30 days; P = .01) (Figure 5) but was not associated with a significant reduction in hospital LOS (mean reduction of 0.64 days; 95% CI, −1.52 to 0.25 days; P = .16) (Figure 6).
Heterogeneity and limited sample size in most key subgroups limited the interpretation of metaregression or subgroup analyses. Results were similar in most subgroup analyses with statistically nonsignificant interaction terms (P > .05), including studies with and without tele-ICU vendor affiliations and among higher- and lower-quality studies, with higher-quality studies being defined as those with a Newcastle-Ottawa Scale score of 7 or greater (Table 3 and Table 4); however, we did find that the reduction in ICU LOS was statistically significant among studies with tele-ICU vendor affiliations compared with studies without tele-ICU vendor affiliations (P < .001).
In a systematic literature review and meta-analysis of 13 studies encompassing more than 41 000 patients, we found that tele-ICU coverage was associated with a significant reduction in ICU mortality and LOS but did not significantly reduce in-hospital mortality or hospital LOS. Our study also revealed a striking degree of variation in how tele-ICU coverage was defined, the hospitals where it was evaluated, and the impact that tele-ICU coverage appeared to have. In aggregate, our results highlight the need for more rigorous evaluation of tele-ICU coverage.
A number of our findings merit further discussion. First, it is important to discuss our finding that tele-ICU care was associated with a significant reduction in ICU mortality but not in-hospital mortality. The potential causes for our disparate findings are important and complex. Tele-ICU care is typically a multifaceted intervention that includes upgrades to electronic health records, enhanced quality improvement programs, and monitoring of patients by off-site intensivists with a focus on the patients, staff, and technology of the ICU.54 Thus, it is possible that tele-ICU care might reduce the mortality for ICU patients while the patients are in the ICU, but this benefit could be eroded once patients are transferred to the floor.
It is also possible that a portion of or all the reduction in mortality observed with tele-ICU care results from changes in triage and medical decision making and thus the severity and acuity of illness among the patients who were treated in the ICU before and after tele-ICU implementation. In particular, implementation of tele-ICU care may lead to changes in decision making about which patients should and should not be admitted to the ICU in the first place.55 More specifically, tele-ICU care may lead to more judicious decisions about which patients are likely to benefit from ICU admission and which patients should immediately receive palliative care.56 Finally, the statistically significant reduction in ICU mortality, but not in-hospital mortality, may reflect the fact that more studies reported ICU mortality than in-hospital mortality, and we therefore had lower statistical power for the latter.
Second, it is important to consider our finding that tele-ICU care was associated with a 1.26-day reduction in ICU LOS but no reduction in hospital LOS. This finding is logical and mechanistically plausible given the nature of tele-ICU coverage. In particular, if the addition of tele-ICU care leads to more consistent attempts at ventilator weaning, this might reduce ICU LOS and ventilator days.57,58 If the availability of around-the-clock intensivist oversight leads to a greater willingness to transfer patients out of the ICU on weekends and evenings, this too could contribute to reductions in ICU LOS. A 1.26-day reduction in ICU LOS translates into a 10% to 30% relative reduction in ICU LOS and may represent an important mechanism for improving ICU throughput in an era of limited bed capacity and intensivists.59,60
Third, the benefit of tele-ICU on mortality and LOS appeared moderately greater in studies that were published by authors associated with tele-ICU vendors, such as VISICU (Baltimore, Maryland). Although these interaction effects were not statistically significant, the point estimates alone are cause for concern and highlight the importance of independent research. Future studies involving tele-ICU should consistently disclose potential financial conflicts of interest and funding sources.61,62
Fourth, there is a high degree of heterogeneity in the studies included in our analysis, which is manifest in numerous nontrivial ways. The analyses included in our study consisted of evaluations of tele-ICU coverage that were implemented in open and closed ICUs despite evidence that closed ICUs with dedicated intensivist staffing typically have much better outcomes.2 If closed ICU staffing already delivers something resembling optimal outcomes, it is reasonable to assume that adding tele-ICU coverage to closed ICUs might offer little benefit, whereas addition of tele-ICU coverage to open ICUs might offer significant benefit. It also seems likely that the benefits of tele-ICU coverage differ according to an array of other factors, including baseline ICU mortality, patient severity, underlying diagnosis, staff acceptance of tele-ICU monitoring, and the number of hours per day that tele-ICU monitoring is available. In the most rigorous evaluation of tele-ICU coverage reported to date, Thomas et al53 found that only 31% of physicians of patients eligible for tele-ICU coverage delegated full authority to the tele-ICU team and allowed the tele-ICU team to intervene only for life-threatening emergencies in the remaining 69%. In our analysis, few studies presented data on these important confounders in a consistent and transparent enough way to allow for more detailed analyses at the ICU level. The heterogeneity that we observed in the design of the underlying studies included in our analysis contributed to the wide CIs for all 4 of our study end points (ICU and in-hospital mortality and ICU and hospital LOS). These wide CIs are far from trivial. Our best estimate is that tele-ICU coverage reduced ICU mortality by 20% and ICU LOS by 1 day (approximately 30%, assuming an average ICU LOS of 3 days) but had no effect on in-hospital mortality or hospital LOS. However, our wide CIs suggest that tele-ICU coverage could have a much larger effect or no effect whatsoever. Thus, it is again important to reiterate the pressing need for further study.
A number of limitations to our analysis merit brief mention. Although there are a number of guidelines for the reporting of observational studies,63,64 we often found important information missing. Perhaps the most visible shortcoming was a lack of consistent measurement, reporting, and adjustment for patient severity. Similarly, although some studies reported data at the level of the individual hospital or ICU, many provided only aggregate study-level outcomes. As a result, we were limited in our ability to examine differences in the impact of tele-ICU coverage in our subgroup and meta-regression analyses. Finally, cost data were seldom reported; therefore, we were unable to draw meaningful conclusions regarding the cost or cost-effectiveness of tele-ICU.
In conclusion, we found that, although tele-ICU coverage was associated with significant reductions in ICU mortality and LOS, no improvement in in-hospital mortality or hospital LOS occurred. We identified several factors that are likely to mitigate the effectiveness of tele-ICU coverage but few definitive data as to which patients, ICUs, and hospitals are most likely to benefit from this technology. Given the significant resources required for tele-ICU implementation, further evaluation is dearly needed.
Correspondence: Lance Brendan Young, PhD, MBA, Center for Research in the Implementation of Innovative Strategies in Practice, Iowa City Veterans Affairs Medical Center (Mail Stop 152), Iowa City, IA 52242 (Lance.Young@va.gov).
Accepted for Publication: October 15, 2010.
Author Contributions: Drs Cram and Young had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Young, Sasson, and Cram. Acquisition of data: Young and Cram. Analysis and interpretation of data: Young, Chan, Lu, Nallamothu, and Sasson. Drafting of the manuscript: Young, Chan, Lu, Sasson, and Cram. Critical revision of the manuscript for important intellectual content: Young, Chan, Lu, Nallamothu, and Cram. Statistical analysis: Lu, Sasson, and Cram. Obtained funding: Cram. Study supervision: Sasson and Cram.
Financial Disclosure: None reported.
Funding/Support: This study is based on work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, and the Veterans Affairs Midwest Rural Health Resource Center.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the views or policy of the Department of Veterans Affairs or the US government.
Previous Presentations: A poster based on this article was presented at the American Thoracic Society International Conference; May 16, 2010; New Orleans, Louisiana; and an oral presentation was delivered at the VA Health Services Research and Development (HSR&D) National Meeting; February 17, 2011; National Harbor, Maryland.