Association of Telemedicine for Remote Monitoring of Intensive Care Patients With Mortality, Complications, and Length of Stay

Context Telemedicine technology, which can enable intensivists to simultaneously monitor several intensive care units (ICUs) from an off-site location, is increasingly common, but there is little evidence to support its use.

Objective To assess the association of remote monitoring of ICU patients (ICU telemedicine [tele-ICU]) with mortality, complications, and length of stay (LOS).

Design, Setting, and Patients Observational study conducted in 6 ICUs of 5 hospitals in a large US health care system to assess the use of tele-ICU. The study included 2034 patients in the preintervention period (January 2003 to August 2005) and 2108 patients in the postintervention period (July 2004 to July 2006).

Main Outcome Measures Hospital and ICU mortality, complications, and hospital and ICU survivors' LOS, with outcomes adjusted for severity of illness.

Results Local physicians delegated full treatment authority to the tele-ICU for 655 patients (31.1%) and authority to intervene only in life-threatening events for the remainder. Observed hospital mortality rates were 12.0% (95% confidence interval [CI], 10.6% to 13.5%) in the preintervention period and 9.9% (95% CI, 8.6% to 11.2%) in the postintervention period (preintervention to postintervention decrease, 2.1%; 95% CI, 0.2% to 4.1%; P = .03); observed ICU mortality rates were 9.2% (95% CI, 8.0% to 10.5%) in the preintervention period and 7.8% (95% CI, 6.7% to 9.0%) in the postintervention period (preintervention to postintervention decrease, 1.4%; 95% CI, −0.3% to 3.2%; P = .12). After adjustment for severity of illness, there were no significant differences associated with the telemedicine intervention for hospital mortality (relative risk, 0.85; 95% CI, 0.71 to 1.03) or for ICU mortality (relative risk, 0.88; 95% CI, 0.71 to 1.08). There was a significant interaction between the tele-ICU intervention and severity of illness (P < .001), in which tele-ICU was associated with improved survival in sicker patients but with no improvement or worse outcomes in less sick patients. There were no significant differences between the preintervention and postintervention periods for hospital or ICU LOS.

Conclusion Remote monitoring of ICU patients was not associated with an overall improvement in mortality or LOS.

Care provided in intensive care units (ICUs) is notable for its high cost¹ and high rates of morbidity and mortality.² Because onsite staffing by intensivists has been associated with lower morbidity and mortality,³ experts recommend that intensivists care for ICU patients.⁴ However, there is a shortage of intensivists,⁵ which has led to the use of telemedicine technology to allow intensivists to remotely and simultaneously care for patients in several ICUs (ICU telemedicine [tele-ICU]), thus extending their reach.^6,7

Remote monitoring may be a partial solution for the intensivist shortage, but it is expensive, its use is increasing, and there are few data in the peer-reviewed literature evaluating its effect on morbidity and mortality. Furthermore, it is not known whether remote monitoring by intensivists would benefit all or only a subset of ICU patients.^3,4 Therefore, we conducted this study to assess the effect of a tele-ICU intervention on mortality, complications, and length of stay in 6 ICUs in a large health care system by measuring these outcomes before and after implementation of the tele-ICU.

We studied the use of a tele-ICU intervention in a large, nonprofit health care system located in the Gulf Coast region of the United States. The tele-ICU was implemented to care for all patients in 7 ICUs (one of these, a neurotrauma ICU, was not included in this study).

When describing the ICUs, we use the term “closed” to indicate units in which intensivists were the only physicians allowed to admit patients to the unit and “open” to indicate units in which multiple different physician types (including intensivists) had admission privileges. The study ICUs and hospitals included a closed medical ICU (16 beds) and closed shock trauma/general surgical ICU (11 beds) in a large tertiary care teaching hospital; 2 open medical-surgical ICUs (12 beds and 8 beds) in 2 small community hospitals; and 2 open medical-surgical ICUs (32 beds and 8 beds) in 2 large urban hospitals. In the closed units, intensivist-led multidisciplinary teams conducted rounds among all patients at least once a day. These patients never had nonintensivists as attending physicians. In the open units, patients were cared for by several types of physicians who also saw their patients daily. One of the open units had intensivist coverage for approximately 50% of the patients, and another had coverage for 10% or less of the patients; the remaining patients had no intensivist coverage.

The tele-ICU intervention had a staggered implementation, with the first ICU starting the intervention in March 2004 and the last in September 2005. The study was approved by the institutional review board of the University of Texas Health Science Center at Houston, with a waiver of the requirement for informed consent.

The tele-ICU system consisted of a remote office located in the administrative offices of the health care system, separate from all the hospitals. It was equipped with audiovisual monitoring and was staffed by 2 intensivists from noon to 7 AM Monday through Friday and 24 hours a day on Saturday and Sunday. In addition, 4 registered nurses and 2 administrative technicians worked there 24 hours a day, 7 days a week. Each intensivist collaborated with 2 nurses and 1 technician to monitor half of the ICU beds. The tele-ICU used proprietary eICU technology (VISICU Inc, Baltimore, Maryland). Each physician and nurse sat at a computer workstation having either 5 monitors (for nurses) or 6 monitors (for physicians). This workstation provided real-time vital signs with graphic trends; audiovisual connections to patients' rooms; early warning signals regarding abnormalities in a patient's status (Smart Alerts, VISICU Inc); and access to laboratory values, imaging studies, and the medication administration record. Tele-ICU physicians conducted rounds based on subjective assessments of illness severity: patients in the “red” category (severely ill, hemodynamically unstable) were evaluated every hour; those in the “yellow” category were evaluated every 2 hours; and those in the “green” category were evaluated every 4 hours. The tele-ICU and the outlying units did not share a unified electronic medical record. Progress notes were faxed daily from monitored units to the tele-ICU, and orders from the tele-ICU were entered into the computer workstation, then printed in the monitored units.⁸

Physicians who cared for patients in the monitored units determined how much authority the tele-ICU had to care for their patients. “Minimal delegation” meant that the tele-ICU could intervene only for life-threatening situations (as judged by the tele-ICU staff); “full delegation” meant that the tele-ICU could give routine orders, change treatment plans, and intervene for life-threatening situations. A middle category existed but was not used. To help interpret our results, we collected data on physician delegation (the number of physicians who selected full vs minimal delegation) as well as the number and types of interventions initiated by the tele-ICU.

This observational study consisted of 2 independent groups: patients in the preintervention period (January 2003 to August 2005) and those in the postintervention period (July 2004 to July 2006). Because severity of illness and its relation to health status was expected to affect outcomes, the Simplified Acute Physiology Score II (SAPS II)⁹ was likewise measured for all patients.

The primary hypotheses were that ICU mortality and hospital mortality would be less in the postintervention period than in the preintervention period. Assuming no clustering effects of patient outcomes among units, the required sample size was 1612 per group, yielding a total of 3224 patients. However, assuming the intracluster correlation coefficient for binary outcomes to be at most 0.01, the required sample size was 2355 per group, for a total of 4732 patients.¹⁰

The final sample comprised 2034 patients (49.1%) in the preintervention and 2108 patients (50.9%) in the postintervention conditions, for a total of 4142 patients. This sample had 80% power to detect a reduction from 10.5% to 7.7% at a 5% level of significance (2-sided), even if the unit and SAPS II effects accounted for as much as 20% of the variance, again assuming no clustering effects.

However, as reported below, the models used for determining required sample size—the 2-group preintervention vs postintervention model and the main-effects model adjusting for unit and SAPS II score—showed severe lack of fit to the data, necessitating the use of models containing interactions and quadratic terms. These more complicated models were not anticipated and therefore were not considered in the original sample size determination. They nevertheless proved to be highly significant. We did not conduct a poststudy power analysis.¹¹

The number of records per ICU was proportional to each ICU's number of discharges, with half allocated for the preintervention period and half for the postintervention period. For a given unit, the preintervention sample size was accrued by sampling discharges (survivors and nonsurvivors) starting the day prior to implementation of the tele-ICU intervention and collecting all records of discharged patients from that day backward until the requisite number of records was collected. The postintervention sampling of admissions began on average 95 days (range, 60-120 days) after implementation to allow for start-up problems to be addressed. Postintervention sampling then continued until the requisite sample size was collected. The time required to collect the sample size thus varied by ICU, depending on the number of discharges or admissions per day. Overall, records were sampled from January 2003 to July 2006.

Registered nurses with intensive care experience and not employed by one of the ICUs or VISICU were trained to abstract data from medical records. To test reliability of data abstraction, 357 records were reviewed twice. The reliability of abstraction was good, with mean agreement of 93.7% for the variables required to calculate mortality and the 8 complications. The nurses were not told if the records they reviewed were in the preintervention or the postintervention period, but it was possible for them to have read something in the medical record that would have unblinded them to intervention group.

We collected data on patient demographics (age, sex, race/ethnicity), SAPS II score,⁹ ICU mortality, hospital mortality, ICU length of stay (LOS), hospital LOS, and 8 ICU complications. Race/ethnicity categories were determined by the patient or family and were collected to determine if the proportion of racial/ethnic groups changed from the preintervention to the postintervention period.

The 8 complications were (1) ventilator-associated pneumonia based on either the definition of Kollef¹² or the diagnosis of ventilator-associated pneumonia written in a physician note, plus the administration of antibiotics; (2) vascular catheter–related bloodstream infection based on either the Centers for Disease Control and Prevention definition used in the National Nosocomial Infections Surveillance System or the diagnosis written in a physician note, plus the administration of antibiotics; (3) upper gastrointestinal tract bleeding as determined by clinical bleeding and hemodynamic compromise or need for transfusion; (4) acute renal failure indicated by new renal replacement therapy; (5) unplanned extubation; (6) need for cardiopulmonary resuscitation; (7) venous thromboembolic disease as determined by the presence of either deep-vein thrombosis or pulmonary embolism on the radiologist report of an appropriate imaging study; and (8) readmission to ICU during the hospitalization. We also performed a 60-day audit of all tele-ICU decisions in 4 of the participating ICUs.

Exploratory analyses (bar plots, box plots, density plots, and quantile-quantile plots) were conducted on age, sex, race/ethnicity, and SAPS II score to assess distributional characteristics, identify extreme values, and assess possible differences in the background variables between the preintervention and postintervention conditions within each unit. Exploratory local regression¹² was used to review relations among all the variables.

After each statistical analysis, regression diagnostics, including residual, leverage, and influence diagnostics, were conducted. Models were compared to local regression fits to assess the adequacy of fit of the SAPS II score.¹³ Bootstrap analyses were conducted to assess the influence of the inclusion of patients with rarer high SAPS II scores on the regression models. Missing data were assumed to be missing completely at random—a datum's being missing was unrelated to the possible value of the datum itself or to any other values in the data set.¹⁴

The hospital- and SAPS II–adjusted risk ratios were computed from multilevel Poisson regression, with the ICUs as a random effect and the SAPS II score and intervention as fixed effects. However, the main-effect adjustments, although informative and presented herein, did not provide a complete account of the data. Therefore, a more elaborate statistical analysis was based on the following strategy of model construction. An initial, plausible, sufficiently complex model was proposed and fitted to the data. This initial model was then reduced in a stepwise procedure using the consistent Akaike information criterion (CAIC),¹⁵ a goodness-of-fit index favoring parsimony, to eliminate terms unnecessary to account for the data. This CAIC-reduced model was then examined for adequate fit using regression diagnostics.

If the CAIC-reduced model was satisfactory, it was considered the final model, and the model-construction process was finished. If, however, the reduced model showed lack of fit, an alternative initial model was proposed and the procedure repeated. Once a final model was obtained, sequential significance tests were conducted and P values, predicted values with 95% confidence intervals (CIs), and other subsidiary statistics such as relative risk (RR) were calculated.

The statistical models for complications, ICU mortality, and hospital mortality were mixed-effect logistic regressions with random unit effects and fixed SAPS II score and tele-ICU intervention effects. The initial model was a main-effects model for the tele-ICU intervention effect, adjusted for the unit and SAPS II score effects. This initial model, reduced by the CAIC, proved adequate in accounting for complications but inadequate in accounting for the ICU or hospital mortality.

For mortality, a more inclusive initial model was proposed that contained additional effects for tele-ICU intervention, moderated by unit (unit × intervention), by SAPS II score (SAPS II × intervention), or jointly by both unit and SAPS II score (unit × SAPS II × intervention). The final models contained a unit effect and a SAPS II × intervention interaction. The standard errors and 95% CIs for estimated mortality rates and RRs were obtained by simulation.¹⁶

Because patients who die before discharge have markedly different clinical and resource utilization characteristics than those who survive, and because data truncated by death cannot technically be treated as missing,¹⁴ only those patients who survived were analyzed for LOS. Thus, the estimates of the effects of the tele-ICU intervention on ICU LOS or hospital LOS are conditional on the patient's being alive at transfer to the ICU or discharge from the hospital.

After examining several distributions for LOS, the log-normal distribution was chosen. As in the case of mortality outcomes, the unit effect was considered random, whereas the SAPS II and telemedicine effects were considered fixed. A linear mixed-effects model with a quadratic term for SAPS II score proved necessary to account for the LOS data. An initial model containing the squared SAPS II score and all interactions was reduced by the CAIC to a final model that contained an effect for linear and quadratic components of SAPS II score and their interactions with the tele-ICU intervention.

Relative LOS, the ratio between postintervention LOS and preintervention LOS, was calculated from the final model. As previously discussed, sequential significance tests were obtained from the final model, and standard errors and 95% CIs were obtained by simulation.

All analyses were conducted in R.¹⁷ The RODBC package¹⁸ was used for data management, the chron package¹⁹ for date arithmetic, the MASS package²⁰ for the CAIC, the locfit package¹³ for local regression, and the lme4 package²¹ for generalized linear multilevel models. Documentation and automatic report generation from R into LaTeX documents was conducted using Sweave.²²

The initial sample consisted of 4167 patients. Elimination of cases with missing data yielded a final sample of 4142 patients, of whom 2034 were preintervention and 2108 were postintervention. The final sample sizes by unit/ICU type were 259 (open 8-bed medical-surgical), 681 (closed 16-bed medical), 216 (closed 11-bed surgical), 771 (open 8-bed medical-surgical), 1647 (open 32-bed medical-surgical) and 568 (open 12-bed medical-surgical).

Patient age (P = .13), sex (P = .20), and SAPS II score (P = .25) were similar in the preintervention and postintervention periods, but race/ethnicity showed changes from the preintervention to the postintervention period (P = .006), with more whites and African Americans and fewer Hispanics and individuals of other race/ethnicity in the postintervention than in the preintervention period (Table 1). Almost two-thirds of the patients in the postintervention group had physicians who chose minimal delegation to the tele-ICU (n = 1393 [66.1%]). For these patients, the tele-ICU intervened only for life-threatening situations. Physicians delegated full treatment authority to the tele-ICU for 655 patients (31.1%).

During a 60-day period for 2 of the closed and 2 of the open ICUs, the tele-ICU physicians initiated 1446 orders to change care. In the closed units 5.3 orders were initiated per day, and 7% of these were high-level interventions (code supervision, ventilator management, and acute management of hypertension, arrhythmia, or hypotension). In the open units 18.5 orders were initiated per day, and 26% were high-level interventions.

The observed hospital mortality rates (Table 2) were 12.0% (95% CI, 10.6% to 13.5%) in the preintervention period and 9.9% (95% CI, 8.6% to 11.2%) in the postintervention period (preintervention to postintervention decrease, 2.1%; 95% CI, 0.2% to 4.1%; P = .03). After adjustment for severity of illness, there were no significant differences associated with the telemedicine intervention for hospital mortality (RR, 0.85; 95% CI, 0.71 to 1.03) (Figure).

Although the telemedicine intervention did not exhibit a main effect (P = .07), it was moderated by the SAPS II score (P < .001). The interaction of SAPS II score with the telemedicine intervention is exhibited in the upper left panel of eFigure 1. The 2 fitted curves intersect at a SAPS II score of 39 (95% CI, 27 to 48), which corresponds to a risk of 8.9%. For those patients with SAPS II scores of 39 or greater (31.4% of the sample), the mortality risk was higher and increased more rapidly for patients in the preintervention group than for those in the postintervention group. But for patients with SAPS II scores less than 39 (68.6% of the sample), the mortality risk was higher for patients in the postintervention group than for those in the preintervention group. The differential effect of the telemedicine intervention as a function of SAPS II score is further elucidated in the bottom panel of eFigure 1, which reexpresses its upper plot as RR, along with its 95% point-wise confidence ribbon. Comparing patients with SAPS II scores less than or greater than 39 is not a subgroup analysis but a computed result from the final logistic regression model.

The observed ICU mortality rates (Table 2) were 9.2% (95% CI, 8.0% to 10.5%) in the preintervention period and 7.8% (95% CI, 6.7% to 9.0%) in the postintervention period (preintervention to postintervention decrease, 1.4%; 95% CI, −0.3% to 3.2%; P = .12). Adjusted ICU mortality was also not significant (RR, 0.88; 95% CI, 0.71 to 1.08), but the interactions with SAPS II score paralleled those for hospital mortality and are displayed in the right panels of eFigure 1. The RR favors the preintervention group for SAPS II scores less than 44 (95% CI, 31 to 54; 76% of the sample). The RR favors the postintervention group for SAPS II scores of 44 or greater (24% of the sample).

The observed complication rates were 17.9% (95% CI, 16.3% to 19.6%) for the preintervention period and 19.2% (95% CI, 17.5% to 20.9%) for the postintervention period (Table 2), yielding a preintervention to postintervention increase of 1.3% (95% CI, −1.1% to 3.7%; P = .31). The CAIC-reduced model revealed that the rate of complications was associated with SAPS II score (P < .001) but not with tele-ICU implementation (P < .15).

Of the 4142 patients, the 3688 patients (89.0%) who survived to discharge were analyzed for hospital LOS. The observed mean hospital LOS (Table 2) was 9.8 days (95% CI, 9.4 to 10.2 days) preintervention and 10.7 days (95% CI, 10.2 to 11.1 days) postintervention, yielding a preintervention to postintervention increase of 0.9 days (95% CI, 0.2 to 1.5 days; P = .006). Adjusted for unit (random) effects and SAPS II score (P < .001), the intervention did not exhibit a main effect (P = .10) but was moderated by the linear (P < .09) and quadratic (P < .002) components of the SAPS II score (eFigure 2). For patients with SAPS II scores less than 49 (95% CI, 36 to 61; 85.1% of the sample), patients in the preintervention group had LOS values less than or equal to those for patients in the postintervention group. For patients with SAPS II scores of 49 or greater, representing 14.9% of the sample, those in the postintervention group had a 20% to 80% reduction in LOS. Comparing patients with SAPS II scores less than and greater than 49 is not a subgroup analysis but a direct result from the final regression model. The differential changes in LOS as a function of SAPS II score are further elucidated in the bottom panel of eFigure 2, which reexpresses its upper plot as relative LOS, along with its 95% point-wise confidence ribbon. A relative LOS equal to 1 implies no difference between the groups; greater than 1 favors the preintervention group, and less than 1 favors the postintervention group.

The observed mean LOS for the 3789 patients (91.5%) who survived to transfer was 4.3 days (95% CI, 4.0 to 4.5 days) for the preintervention period vs 4.6 days (95% CI, 4.3 to 4.9 days) for the postintervention period, yielding a preintervention to postintervention increase of 0.3 days (95% CI, −0.1 to 0.7 days; P = .13). Results for ICU LOS parallel those for hospital LOS and are displayed in the right panels of eFigure 2. The relative LOS favors the preintervention group for SAPS II scores less than 60 (95% CI, 46 to 72; 92.3% of the sample). The RR favors the postintervention group for SAPS II scores of 60 or greater (7.7% of the sample).

We detected no association between implementation of telemedicine technology and adjusted hospital or ICU mortality, LOS, or complications. Our results differ from those from a previous report⁶ and should be interpreted in light of 3 factors that may influence the effectiveness of a tele-ICU: (1) how the tele-ICU was used by the remote intensivists to alter care in the monitored units; (2) its acceptance by physicians in the monitored units; and (3) integration of the information systems of the tele-ICU and the monitored units.

Interestingly, our regression models suggested a decreased risk for more severely ill patients (SAPS II scores ≥39) after tele-ICU implementation. Thus, proponents of tele-ICU systems might suggest that the mortality reduction we observed among the sickest patients was because the sickest patients were those most likely to have unexpected changes in their medical condition (eg, arrhythmia, hypotension, sepsis, hypoxia) that required rapid intervention. The tele-ICU can provide this rapid response because of the constant monitoring (including computer-issued alerts sent to the remote intensivist to report changes in key physiological parameters) and availability of nurse and physician intensivists. The 60-day surveillance data suggested that tele-ICU physicians frequently provided such timely and critical interventions, especially in the open ICUs.

Clinician acceptance of the technology is also important. The fact that almost two-thirds of the patients in our study had physicians who chose minimal delegation to the tele-ICU may have contributed to our inability to find no mortality benefit, even though the tele-ICU physicians could still intervene in life-threatening situations. Had more physicians delegated full responsibility to the tele-ICU, less ill patients might have benefited from the routine care provided by the tele-ICU.

The lack of an overall mortality reduction may also be explained by the degree of integration of the tele-ICU and remote units. Physicians and nurses in this tele-ICU had access to real-time vital signs and wave tracings, laboratory values, imaging studies, and the medication administration record. However, the tele-ICU and the monitored units did not share clinical notes or computerized physician order entry within a common electronic record. These notes were instead faxed to the tele-ICU daily. Greater integration of clinical information might have resulted in a larger effect on mortality.

Our study has other strengths and weaknesses. We surveyed the ICUs regularly and found no quality improvement projects focused on the sickest patients (such efforts could explain the lower mortality in those patients). Our sample of patients and ICUs was larger and more heterogeneous than those in previous reports,⁶ and our data were collected by an explicit and reliable medical record review (in contrast to administrative data). Implementation of the tele-ICU occurred sequentially throughout the system over an 18-month period, and we waited several weeks before starting postintervention data collection. This allowed for managers of the tele-ICU and each unit to work out startup problems associated with implementation of such a complex technology and mediated any seasonal changes in mortality.

We have no data on survival following hospital discharge. It is possible that the tele-ICU intervention led to the sickest patients being transferred more quickly to hospice or other sites of care, where they died. Such transfers may have been an appropriate improvement in care, but without data on survival following hospital discharge we do not know if the 20% to 50% reduction in hospital mortality among the sickest patients translates to the same reduction in long-term mortality.

Although the nurse data collectors were not told if a medical record was from the preintervention or postintervention period, something written by a caregiver could have indicated that the record was from the tele-ICU period. The lack of association between the tele-ICU intervention and complications was not anticipated but can be explained by the above-noted lack of acceptance and integration of the tele-ICU. Another possible explanation is detection and surveillance bias. The presence of the tele-ICU may have led to increased surveillance for, and documentation of, complications by the physicians and nurses in the monitored units.

Implementation of a tele-ICU was not associated with a reduction in overall hospital mortality for patients in these 6 ICUs. The lack of apparent benefit may be attributable to low decisional authority granted to the tele-ICU as well as to varied effects across different types of patients. Given the expense of tele-ICU technology, the conflicting evidence about its effectiveness, and the existence of other effective quality improvement interventions for ICUs,²³ further use of this technology should proceed in the context of careful monitoring of patient outcomes and costs.

Corresponding Author: Eric J. Thomas, MD, MPH, University of Texas Medical School at Houston, 6410 Fannin UPB 1100.45, Houston, TX 77030 (eric.thomas@uth.tmc.edu).

Author Contributions: Dr Thomas had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Thomas, Weavind, Patel.

Acquisition of data: Thomas, Wueste, Weavind.

Analysis and interpretation of data: Thomas, Lucke, Wueste, Patel.

Drafting of the manuscript: Thomas, Wueste, Patel.

Critical revision of the manuscript for important intellectual content: Thomas, Lucke, Wueste, Weavind, Patel.

Statistical analysis: Lucke.

Obtained funding: Thomas.

Administrative, technical, or material support: Wueste, Weavind, Patel.

Study supervision: Thomas, Wueste.

Funding/Support: This study was funded by grant R01 HS015234-01 from the Agency for Healthcare Research and Quality and by grant UL1 RR024148 01 from the National Center for Research Resources, National Institutes of Health (Clinical and Translational Sciences Award).

Role of the Sponsors: The Agency for Healthcare Research and Quality and the National Center for Research Resources had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.