Effect of a Postdischarge Virtual Ward on Readmission or Death for High-Risk Patients: A Randomized Clinical Trial

Importance  Hospital readmissions are common and costly, and no single intervention or bundle of interventions has reliably reduced readmissions. Virtual wards, which use elements of hospital care in the community, have the potential to reduce readmissions, but have not yet been rigorously evaluated.

Objective  To determine whether a virtual ward—a model of care that uses some of the systems of a hospital ward to provide interprofessional care for community-dwelling patients—can reduce the risk of readmission in patients at high risk of readmission or death when being discharged from hospital.

Design, Setting, and Patients  High-risk adult hospital discharge patients in Toronto were randomly assigned to either the virtual ward or usual care. A total of 1923 patients were randomized during the course of the study: 960 to the usual care group and 963 to the virtual ward group. The first patient was enrolled on June 29, 2010, and follow-up was completed on June 2, 2014.

Interventions  Patients assigned to the virtual ward received care coordination plus direct care provision (via a combination of telephone, home visits, or clinic visits) from an interprofessional team for several weeks after hospital discharge. The interprofessional team met daily at a central site to design and implement individualized management plans. Patients assigned to usual care typically received a typed, structured discharge summary, prescription for new medications if indicated, counseling from the resident physician, arrangements for home care as needed, and recommendations, appointments, or both for follow-up care with physicians as indicated.

Main Outcomes and Measures  The primary outcome was a composite of hospital readmission or death within 30 days of discharge. Secondary outcomes included nursing home admission and emergency department visits, each of the components of the primary outcome at 30 days, as well as each of the outcomes (including the composite primary outcome) at 90 days, 6 months, and 1 year.

Results  There were no statistically significant between-group differences in the primary or secondary outcomes at 30 or 90 days, 6 months, or 1 year. The primary outcome occurred in 203 of 959 (21.2%) of the virtual ward patients and 235 of 956 (24.6%) of the usual care patients (absolute difference, 3.4%; 95% CI, −0.3% to 7.2%; P = .09). There were no statistically significant interactions to indicate that the virtual ward model of care was more or less effective in any of the prespecified subgroups.

Conclusions and Relevance  In a diverse group of high-risk patients being discharged from the hospital, we found no statistically significant effect of a virtual ward model of care on readmissions or death at either 30 days or 90 days, 6 months, or 1 year after hospital discharge.

Trial Registration  clinicaltrials.gov Identifier: NCT01108172

Quiz Ref IDReadmissions after hospital discharge are common,¹ costly, and believed by many to be an indicator of suboptimal health care.² In the United States, hospitals are now financially penalized if their readmission rates for Medicare patients are deemed excessive.³ These penalties are set to increase to up to 3% of a hospital’s Medicare reimbursement in 2015. This “pay-for-performance” program has been implemented despite limited evidence supporting the hypothesis that hospitals can implement initiatives that will successfully and consistently reduce readmissions.

Although several single-center studies have demonstrated that a hospital’s readmission rate is modifiable,^4-6 the authors of a recent systematic review were unable to identify a single intervention or a bundle of interventions that reliably reduced the risk of readmission in a manner that could be considered generalizable.⁷ In contrast, the authors of another recently published systematic review concluded that complex interventions could reduce readmissions.⁸

Quiz Ref IDThe virtual ward model of care, first pioneered in the United Kingdom⁹ and since piloted elsewhere,^10,11 is an intuitively appealing way of providing care to community-dwelling patients with complex needs. The virtual ward takes many elements of hospital care that are appreciated by patients or clinicians (eg, an interprofessional team, a daily team meeting, a single point of contact for patients, etc) and incorporates them into community-based care, with a goal of improving health outcomes and patient experience while also providing better value for money. Despite the virtual ward’s conceptual appeal and its increasingly common implementation, the model of care has not yet been rigorously evaluated.

We therefore performed a randomized trial to determine whether a virtual ward could improve health outcomes and reduce readmissions after hospital discharge in a high-risk population.

We conducted a parallel-group randomized trial with patients randomized at hospital discharge in a 1:1 ratio to either the virtual ward model of care or usual care. Patients were eligible if they were aged 18 years or older, being discharged from the general internal medicine ward of any of the 4 participating hospitals, at high risk of readmission (as determined by LACE¹² [length of stay, acuity of the admission, comorbidities, and emergency department visits in the previous 6 months] score ≥10), and resided within the boundaries of the Toronto Central Local Health Integration Network (see study protocol in the Supplement). Patients were excluded if they were being discharged to a rehabilitation or complex continuing care facility, if neither they nor anyone they could designate could speak English, if they had been previously enrolled in the study, or if they did not wish to participate.

Quiz Ref IDPatients in the control group received usual care. At all 4 participating hospitals, this generally included a typewritten, structured discharge summary, given to the patient at the time of discharge and also sent to the patient’s primary care physician, a prescription when indicated, counseling from the resident physician or other members of the health care team, arrangements for home care as needed, and recommendations or appointments for follow-up care with the patient’s primary care and specialist physicians. Follow-up at a postdischarge clinic was not a routine practice at any of the hospitals but could have been arranged by the discharging physicians at their discretion (Figure).

In addition to receiving usual care, patients assigned to the virtual ward group were admitted to the virtual ward on the day they were discharged home. These patients were informed that they would be contacted by a virtual ward team member the day after discharge. They were also provided with written information about what kind of services would be available and with a telephone number to call if they needed help. This telephone number was answered by a clerical staff member who directed the call to the appropriate team member during weekday business hours or the call was sent to the virtual ward physician’s pager after hours. We modified the virtual ward team complement slightly during the course of the study, but it always consisted of care coordinators (similar to case managers), a part-time pharmacist, a part-time nurse or nurse practitioner, a full-time physician, and a clerical assistant. Most of the staff worked for the Toronto Central Community Care Access Centre, which is responsible for either directly providing or contracting publicly funded home care services in Toronto.

The virtual ward team met each morning to discuss newly admitted and current patients and to design and execute individualized care plans. Each virtual ward patient’s primary care physician received a letter by fax informing him/her that his/her patient had been admitted to the virtual ward. In addition, the virtual ward physician was strongly encouraged to speak to the primary care physician on the telephone soon after the patient was admitted, whenever necessary during admission, and before virtual ward discharge. The care plan began to be developed at the daily interprofessional team meeting on the day after the patient was discharged from the hospital, followed by the care coordinator seeing the patient at home within a few days of discharge. Subsequently, as needed, patients could be assessed by telephone, at home, in a clinic at the hospital where the virtual ward team was based, or if necessary, at an alternate location (eg, in the family physician’s office).

During the virtual ward admission, patients were discussed regularly at the daily interprofessional team meetings. Consistent with the way publicly funded home care is delivered in Ontario, many services, including personal support, nursing, and occupational therapy, were provided by staff working for independent agencies contracted by the Toronto Central Community Care Access Centre. Patients were discharged from the virtual ward when the team believed they were ready for discharge or when it was clear that they were unwilling to further engage with the team. We did not prospectively collect data to characterize the intensity of care provided by the virtual ward team, but we did retrospectively collect such data from 30 randomly selected virtual ward patients during each of 5 six-month periods.

The primary outcome was the composite of readmission to any hospital or death within 30 days of discharge. Secondary outcomes included the composite of readmission to the hospital or death within 90 days, 6 months, and 1 year of discharge, as well as readmission, death, emergency department visits, and nursing home admission (for patients not residing in a nursing home prior to the index hospital admission) within 30 days, 90 days, 6 months, and 1 year of discharge. Outcome data were collected by contacting patients (or other individuals the patient had designated as the contact) by telephone. Patient-reported emergency department visit or readmission to hospital were verified by examining hospital records. When patients could not be contacted at all, hospital records at each of the 7 hospitals that care for adults in the Toronto Central Local Health Integration Network were reviewed. For a large subset of the patients who consented to data linkage, we also compared outcomes obtained by telephone with outcomes ascertained by a review of administrative data housed at the Institute for Clinical Evaluative Sciences.

We hypothesized that the virtual ward could reduce readmissions by approximately one-third, a relative risk reduction that is only slightly larger than that which was observed in the landmark trial of the Care Transitions Intervention, which was a much less intensive intervention.⁴ We estimated that the primary outcome rate in the control group would be 15%.¹³ Allowing for 10% loss to follow-up, we determined we would need 1510 patients for the study to have 80% power. The data and safety monitoring board (DSMB) met on October 29, 2012, when primary outcome data were available for 1464 patients. The DSMB calculated that the conditional power at that time was approximately 50% but noted that a 6-month continuation of the trial would increase the adaptive conditional power to 95%.¹⁴ Of note, this was not based on a prespecified rule but rather on the DSMB’s expert judgment. The trial steering committee accepted the DSMB’s recommendation and continued recruitment. At a subsequent meeting on May 9, 2013, the DSMB recommended unblinding the principal investigator, who together with the trial steering committee decided to stop trial enrollment.

A research coordinator at each of the 4 study sites identified eligible patients and obtained written informed consent during their hospital admission or oral informed consent when obtained over the telephone. Patients were randomized to either the virtual ward or usual care using a computer-generated randomization list either when discharge was imminent or immediately after discharge. The randomization list was stratified by discharge site and homelessness and used random permuted blocks of size 2 and 4.

Given the nature of the intervention, it was not possible to blind patients or clinicians. The statistician and the DSMB were unaware of treatment assignment. The research coordinators who ascertained outcomes were also unaware of treatment assignment, and patients were asked not to tell the coordinator to which treatment group they had been assigned. However, a small proportion of patients made comments during the interviews that unblinded the coordinator.

The primary analysis was an intention-to-treat, unadjusted comparison of proportions, using a χ² test. The significance threshold was 0.05 and testing was 2-sided.

Because of the very small amount of missing outcome data (eg, 0.42% for the primary outcome), these patients were excluded from the analysis of the relevant outcome. To examine potential subgroup effects, we constructed a logistic regression model with each of the following covariates: hospital from which the patient was discharged, discharge location (nursing home vs other), time of enrollment relative to virtual ward initiation, LACE index score,¹² age, sex, reason for hospital admission (heart failure vs other), and risky alcohol use (defined as a response of ≥1 to the question “How many times in the past year have you had X or more drinks in a day?” for which X is 5 for men and 4 for women),¹⁵ as well as interaction terms for each subgroup of interest. To calculate an odds ratio to estimate the treatment effect of the virtual ward for each subgroup, we constructed a logistic regression model with each of the covariates listed above and a single interaction term examining the subgroup of interest. However, all reported P values for interactions were obtained from a fully adjusted model with all the interactions included simultaneously. This was done to avoid the risk of a spurious finding from fitting multiple models. The subgroup analyses were all prespecified, and we excluded 44 individuals with missing covariate data from the subgroup analyses. To compare agreement between data collected within the study and administrative data, we calculated a Cohen κ. Statistical analyses were conducted using either R (version 3.0.2) or SAS (version 9.3).

The study was approved by the research ethics board at each of the participating institutions.

Between June 2010 and May 2013, 30 143 patients were assessed for potential eligibility at the 4 participating hospital sites. Of the 6559 eligible patients, 1932 were randomly assigned to 1 of the 2 groups and were included in the analysis (Figure). There were no important differences between the 2 treatment groups (Table 1).

Within the 30-day period following discharge, 24.6% of patients assigned to usual care and 21.2% of patients assigned to the virtual ward had been readmitted to hospital or died (absolute difference, 3.4%; 95% CI, −0.3% to 7.2%; P = .09; Table 2). There were 47 deaths in the usual care group and 40 deaths in the virtual ward group (4.9% vs 4.2%; absolute difference, 0.7%; 95% CI −1.1% to 2.6%; P = .50). Agreement between data collected by the research coordinators and administrative data was excellent (Cohen κ, 0.88; 95% CI, 0.86-0.91 for readmission and 0.94; 95% CI, 0.91-0.98 for death).

By 90 days after discharge, 38.0% of patients assigned to usual care and 37.1% of patients assigned to the virtual ward had been readmitted to hospital or died (absolute difference, 0.9%, 95% CI, −3.4% to 5.2%, P = .72). There were no statistically significant between-group differences in any of the outcomes at 6 months or 1 year.

None of the interaction tests between the intervention and prespecified subgroups were statistically significant: P = .60 for hospital discharge site, P = .73 for discharge location (nursing home vs other), P = .59 for time of enrollment relative to virtual ward initiation, P = .32 for LACE score, P = .84 for age, P = .61 for sex, P = .40 for reason for hospital admission (heart failure vs other), and P = .34 for risky alcohol use. No tests of interaction were performed for homeless patients and patients screening positive for drug use because of the small numbers in these subgroups (Table 3).

A retrospective analysis of virtual ward activity data for 150 randomly selected patients indicates that the intensity of care provided by the virtual ward team was relatively high. Patients were discussed at the interprofessional team meetings an average of 6.3 times (SD, 2.1) and received an average of 2.8 home visits (SD, 0.95; Table 4). Of note, these data do not include health care or personal care services provided by home care contractors or care provided by physicians not associated with the virtual ward and likely underestimate the amount of care by virtual ward team members because some care may not have been charted. The mean length of stay in the virtual ward was 35.5 days (SD, 27.0 days).

Quiz Ref IDIn this randomized trial involving patients at high risk of readmission or death, we found that a virtual ward model of care, instituted for several weeks after hospital discharge, did not reduce the composite outcome of readmission or death at 30 days after discharge from the hospital. Although our data are not inconsistent with a small absolute benefit at 30 days, outcome data at 90 days suggest that any benefits were not sustained. As a consequence, given the per-patient costs of our intervention, it is highly unlikely that a virtual ward model of care structured similarly to ours would represent an efficient use of health care resources.

Quiz Ref IDThere are several potential reasons the virtual ward model of care we implemented did not reduce readmissions. First, it was difficult for virtual ward team members to communicate with many patients’ primary care physicians. Many primary care physicians were not easily available by telephone or e-mail, which made collaborative care difficult. Second, the multiplicity of different information technology systems available made it difficult for virtual ward team members to know what care had previously been provided to a patient, as well as what care was concurrently being provided. This may have been a particular challenge in our virtual ward, which was based out of an ambulatory hospital different from the ones from which patients were discharged. Third, publicly funded home care in Ontario operates under a purchaser-provider paradigm, which has the potential to increase fragmentation. For example, the virtual ward team could not easily communicate directly with a personal support worker providing care to a virtual ward patient. Fourth, in large part because we implemented a single virtual ward serving patients discharged from several hospitals, the intervention we tested began after discharge and not during the acute care hospitalization. Several observers have suggested that interventions designed to improve posthospital outcomes should begin in the acute care setting.¹⁶ Fifth, the virtual ward model as we implemented it may have simply been too weak a model. For example, a more intensive model (eg, one with an even greater number of home visits, particularly physician home visits) or one that relied more on remote monitoring technology may have been more successful. Finally, some have argued that only a relatively small proportion of readmissions are preventable, at least within the context of the health care and social support systems as they are currently structured.¹⁷ Some proportion of patients who are frequently readmitted to hospital may be characterized as being “hospital dependent.”¹⁸

There are several possible reasons that the results of our study differ from the results of earlier work.^4-6 First, our trial was larger than most previously reported trials designed to evaluate postdischarge interventions, and large trials are less likely to produce a false-positive result.¹⁹ Second, we included patients from several different hospital sites and based the postdischarge centralized intervention out of an ambulatory hospital. Although this was a deliberate design choice that we believed would increase the generalizability of our intervention, postdischarge interventions based out of the discharging hospital site could be more likely to improve postdischarge health outcomes because clinician and information continuity might be better maintained.^20,21 Third, the population of patients included in our study differed substantially from those included in other trials. For example, the patients included in the randomized trial of Project RED⁵ were approximately 20 years younger than individuals included in our trial and included many patients without health care insurance, which is generally a nonissue in Canada. The patients included in the randomized trial of the Care Transitions Intervention⁴ were likely substantially healthier than patients included in our trial, as evidenced by the 30-day readmission rate of 11.3% in the control group of that trial compared with the 30-day readmission rate of 21.3% in our control group. An initial evaluation of a large, multicenter quality improvement initiative known as Project BOOST (Better Outcomes for Older adults through Safe Transitions) suggested a modest reduction in readmissions (absolute reduction, 2.0%; P = .054) among participating sites.²² However, the fact that only 11 of 30 participating hospitals provided data for the analysis makes it difficult to demonstrate convincingly that all hospitals can substantially reduce their readmission rates.^23,24

Conceptually, there are many similarities between the virtual ward model of care and the medical home model.^25,26 One important difference is that the medical home model has largely been intended to transform primary care for all patients, whereas the virtual ward model has been designed for patients who are very likely to be admitted to hospital in the near future. This difference is apparent when comparing health care utilization rates—for example, in a largely negative observational study of the medical home model, hospitalization rates were approximately 1% per month,²⁷ more than an order of magnitude lower than the 30-day readmission rate in our study.

Our study had several strengths. First, we specifically designed the intervention so that it would be generalizable to other health care settings. We worked within the existing structure of the health care system, and rotated staff and physicians so that the results of the trial would not be dependent on a small group of individuals whose high quality of care might not be replicable. Second, we evaluated the virtual ward model of care using a randomized controlled trial, which is the best way to minimize the risk of selection bias, and the approach recommended by many experts, even for evaluations of complex interventions.²⁸ Evaluations that minimize the risk of bias are especially important when the interventions being evaluated are expensive or have the potential to cause harm.

Our study also has several limitations. First, although our trial was larger than several other randomized trials evaluating postdischarge interventions, our results are sufficiently imprecise that we are unable to confidently exclude a small reduction in the primary outcome at 30 days. However, there was no difference at 90 days, 6 months, or 1 year. Second, because of the nature of the intervention and the importance of context, our findings may not be generalizable to all health systems. For example, it remains possible that a virtual ward model of care in a differently structured health care system (eg, one with integrated home-based, primary, hospital, and emergency care, as well as a single electronic health record) might reduce hospital use in high-risk patients in a cost-effective manner. Third, many potentially eligible patients could not be included in the trial because they were discharged on holidays, evenings, or weekends. It is possible that the virtual ward may be more effective in this subgroup than in patients who are discharged during the daytime on weekdays.

Our study showed that the virtual ward model of care did not reduce the primary outcome of readmission or death, or either component individually, in a diverse group of high-risk patients being discharged from hospital.

Corresponding Author: Irfan A. Dhalla, MD, MSc, Department of Medicine, University of Toronto, 30 Bond St, Toronto, ON, Canada, M5B 1W8 (dhallai@smh.ca).

Author Contributions: Dr Dhalla and Mr Thorpe had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Dhalla, O’Brien, Morra, Thorpe, Wong, Mehta, Abrams, Ko, Bell, Gruneir, Lewis, Daub, Rochon, Laupacis.

Acquisition, analysis, or interpretation of data: Dhalla, Morra, Thorpe, Mehta, Frost, Van Rooyen, Bell, Gruneir, Daub, Anderson, Hawker, Laupacis.

Drafting of the manuscript: Dhalla, Van Rooyen.

Critical revision of the manuscript for important intellectual content: Dhalla, O’Brien, Morra, Thorpe, Wong, Mehta, Frost, Abrams, Ko, Bell, Gruneir, Lewis, Daub, Hawker, Rochon, Laupacis.

Statistical analysis: Dhalla, Thorpe.

Obtained funding: Dhalla, Morra, Mehta, Bell, Gruneir, Daub, Anderson, Rochon, Laupacis.

Administrative, technical, or material support: Dhalla, O’Brien, Morra, Wong, Mehta, Frost, Abrams, Ko, Van Rooyen, Hawker.

Study supervision: Dhalla, O’Brien, Morra, Mehta, Frost, Rochon, Laupacis.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: This research was funded by the Canadian Institutes of Health Research, the Ontario Ministry of Health and Long-Term Care, the Green Shield Canada Foundation, the University of Toronto Department of Medicine, and the AcademicFunding Plan Innovation Fund.

Role of the Funder/Sponsor: None of the funder or sponsors had any role in the design of the study, the conduct of the study, the collection, management, analysis or interpretation of the data, or the preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We would like to thank the various groups and committees that made this trial possible: the Data and Safety Monitoring Board (Dave Sackett [chair], Bill Ghali, Finlay McAlister, Lehana Thabane, Mark Rochon), the Virtual Ward Steering Committee (Stacey Daub [cochair]; Gillian Hawker [cochair], Howard Abrams, Irfan Dhalla, Phil Ellison, Marnie Escaf, Francoise Ko, Andreas Laupacis, Anne Marie MacLeod, Rajin Mehta, Heather McPherson, Dante Morra, Ophyr Mourad, Jim O’Neill, Paula Rochon, Fredrika Scarth, Tom Stewart), the Virtual Ward Operations Committee (Francoise Ko [chair], Jamie Arthur, Cris Barrett, Leslie Beard, Irfan Dhalla, Paul Griffin, Catharine McManamon, Tara O’Brien, Dipti Purbhoo, Norm Umali, Brian Wong), and the Virtual Ward Physician Committee (Bob Hyland [chair], Irfan Dhalla, David Frost, Francoise Ko, Rajin Mehta, Tara O’Brien, Mirek Otremba, Tia Pham, Brian Wong, Rob Wu). Additionally, we thank the following individuals for their help with various aspects of the study and manuscript: Miin Alikhan, Michelle Ang, Leslie Ashley, Cailin Bator, Mahmood Beheshti, Wee-Shian Chan, Kendra Delicaet, Stephanie DeMasi, Pamela De Verno, Lorraine Greaves, Judith Hall, Stephen Hwang, Trevor Jamieson, Tara Kiran, Natascha Kozlowski, Wendy Levinson, Nelson Lo, Braden Manns, Ophyr Mourad, Tim Pauley, Lina Pham, Dipti Purbhoo, Laura Pus, Fredrika Scarth, Steve Shadowitz, Arthur Slutsky, Kim Tran, Filomena Valle-Leutri, Carl van Walraven, Rob Wu, and Winnie Yau. We are particularly grateful to the health care professionals who were part of the virtual ward team: Jane Ascroft, Bess Diamantopoulos, Lydia Dimitrievska, Effie Galanis, Michelle Herman, Jeannette Hilliges, Joanne Hunter, Jacqueline Lyn, Anne-Marie Murphy, Grace Sangle, Elisabete Spinasse, Tharexa Sribilan, Norm Umali, Milan Barboza, Pascal Bastien, Mark Bonta, Tina Borschel, Savannah Cardew, Wee Shian Chan, Irfan Dhalla, David Frost, Stephen Hwang, Trevor Jamieson, Pieter Jugovic, Peter Kopplin, Shoba Kumar, Andreas Laupacis, Ken Locke, Rajin Mehta, Josiah Moffatt, Punam Mony Singh, Tara O’Brien, Navindra Persaud, Samir Sinha, Graham Slaughter, Sharon Straus, Brie Volpini, and Brian Wong. Most of all, we are grateful to the patients who volunteered to be part of this study. No one mentioned herein received additional remuneration for participating in the research or helping in the editorial process.