Postdischarge Virtual Visits for Low-risk Surgeries: A Randomized Noninferiority Clinical Trial

Key Points

Question  Does video-based postdischarge virtual follow-up provide noninferior care compared with in-person follow-up?

Findings  In this randomized clinical trial that included 432 adults, the postdischarge hospital encounter proportion after minimally invasive appendectomy or cholecystectomy was not significantly different (12.8% for virtual vs 13.3% for in person). Virtual visits provided equal amount of time with the clinician but significantly decreased the overall time commitment for postdischarge visits.

Meaning  Video-based postdischarge virtual visits were not associated with increased use of care and saved time for the patient.

Importance  Postdischarge video-based virtual visits are a growing aspect of surgical care and have dramatically increased in the setting of the coronavirus disease 2019 (COVID-19) pandemic.

Objective  To evaluate the outcomes of all-cause 30-day hospital encounter proportion among patients who have a postdischarge video-based virtual visit follow-up compared with in-person follow-up.

Design, Setting, and Participants  Randomized, active, controlled noninferiority trial in an urban setting, including patients from a small community hospital and a large, tertiary care hospital. Patients who underwent minimally invasive appendectomy or cholecystectomy by a group of surgeons who cover emergency general surgery at these 2 hospitals were included. Patients undergoing elective and nonelective procedures were included.

Interventions  Patients were randomized in a 2:1 fashion to video-based virtual visit or in-person visit.

Main Outcomes and Measures  The primary outcome is the percentage of patients with 30-day hospital encounter, and we hypothesized that there would not be a significant increase in the 30-day hospital encounter proportion for patients who receive video-based virtual postdischarge care compared with patients who receive standard (in-person) care. Hospital encounter includes emergency department visit, observation, or inpatient admission.

Results  A total of 1645 patients were screened; 289 patients were randomized to the virtual group and 143 to the in-person group. Fifty-three patients crossed over to the in-person follow-up group. The percentage of patients who had a hospital encounter was noninferior for virtual visits (12.8% vs 13.3% for in-person, Δ 0.5% with 1-sided 95% CI, −∞ to 5.2%). The amount of time patients spent with the clinician (mean of 8.4 minutes virtual vs 7.8 minutes in-person; P = .30) was not different, but the median overall postoperative visit time was 27.5 minutes shorter (95% CI, −33.5 to −24.0).

Conclusions and Relevance  Postdischarge video-based virtual visits did not increase hospital encounter proportions and provided shorter overall time commitment but equal time with the surgical team member. This information will help surgeons and patients feel more confident in using video-based virtual visits.

Trial Registration  ClinicalTrials.gov Identifier: NCT03258177

Best-practice postdischarge care of patients undergoing operative procedures requires time and financial contributions from patients and health care systems. Patients who reside in remote locations or have low socioeconomic status are particularly susceptible.¹ Telehealth encompasses a wide range of technologies and tactics. The live video domain involves “live, 2-way interaction between a person and a clinician using audiovisual communications technology.”² Live video virtual technology offers patients the ability to interact with their clinician, visually and verbally, to assure that all concerns are addressed, while keeping the construct of a traditional in-person visit. Expanding access to telehealth services through live video virtual care may offset some costs associated with postdischarge care, yet the safety of this model has not been established.^3,4 The convenience of virtual visits may improve compliance with follow-up appointments, decreasing postdischarge use of urgent care facilities and emergency departments (EDs).

Virtual care technology has been evolving, its uptake influenced by many factors, including changing regulations and reimbursement. It was anticipated to become commonplace over the next decade.⁵ When the coronavirus disease 2019 (COVID-19) crisis hit the United States, there was rapid uptake in use of virtual visits, fueled by an altered risk-to-benefit ratio in the setting of widespread social distancing. Changes that allowed for reimbursement for telemedicine⁶ and waived previously existing requirements that out-of-state clinicians be licensed in the state where they were providing care⁷ further bolstered the rapid uptake. In this setting, understanding the safety of telemedicine is critical for informing patients, clinicians, and health systems.

Quiz Ref IDThe aim of this study was to assess the noninferiority of postdischarge virtual care visits as measured by the rate of 30-day hospital encounters (all-cause) for patients undergoing a minimally invasive appendectomy or cholecystectomy. We hypothesized that there would not be an increase in the 30-day hospital encounter proportion for video-based virtual postdischarge care compared with standard (in-person) care. We secondarily hypothesized that: (1) total visit time (wait time and time spent with clinician) would be less for virtual visit patients and (2) that a greater percentage of virtual visit patients would complete a postdischarge follow-up appointment.

This randomized, active-control, noninferiority trial began in August 2017 and continued until in-person postoperative clinic visits were discontinued owing to COVID-19 in March 2020. Patients were recruited from 2 hospitals: 1 tertiary care level 1 trauma center (>850 beds), Carolinas Medical Center, and a 196-bed community hospital in an urban setting, Carolinas Medical Center–Mercy. Patients were eligible for inclusion if they (1) underwent a minimally invasive (MIS) appendectomy or cholecystectomy (including laparoscopic and robotic), (2) had surgery performed by the group of surgeons who cover nonassigned emergency general surgery (EGS) call, (3) were between the ages of 18 and 90 years, (4) spoke English, (5) had a North Carolina or South Carolina residence, and (6) had an email address and the technology required to complete a visit. This group of surgeons is dedicated to the coverage of nonassigned EGS patients, share a group coverage model, and perform elective surgical procedures. Patients were excluded if they had (1) postoperative length of stay of more than 3 days; (2) inability to complete virtual visit (lack of access to a device or internet); (3) high risk for postoperative complications (perforated appendicitis or cholecystitis); (4) medical condition, laboratory finding, or physical disability that precluded participation (determined by principal investigators or operating surgeon); (5) discharge with drains; (6) narcotic (opioid) medication for chronic pain; and/or (7) discharge to location other than home. After trial commencement, the protocol was updated to (1) exclude patients with active cocaine use and (2) remove the requirement that virtual visit patients will automatically crossover to in-person if admitted to the hospital before their scheduled virtual visit. The formal trial protocols can be found in Supplement 1.

Virtual visits were completed by either an advanced practice professional (APP), such as a nurse practitioner or physician’s assistant, or a surgeon: surgical team member will be used to refer to both roles. Initially, surgical team members were only licensed in North Carolina, but 1 clinician obtained licensing in South Carolina during the trial. A daily report identified potentially eligible patients within the previous and upcoming 48-hour windows. The study was approved by the Atrium Health institutional review board. Patients were approached by a research team member to offer participation and obtain signed informed consent. Research team members who approached, enrolled, and randomized patients included the principal investigators (K.H. and C.R.), research coordinator (N.K.), another APP who performed virtual visits, and research associates specifically trained in the study.

Quiz Ref IDPer protocol, patients randomized to virtual visit would crossover to in-person office visit for (1) unresolved technical problems, (2) inability to perform full evaluation, (3) need/request for prescription pain medication, (4) malignant neoplasm on pathology, or (5) additional care advised by physician. Other reasons for crossover included patient request and scheduling conflicts.

Patient demographics and comorbidities were obtained from the electronic medical record (EMR) and employment, living situation, and highest level of education were obtained via survey at enrollment. Research associates obtained postdischarge visit time for in-person visits. For virtual visits, the certified medical assistant recorded arrival time, and the surgical team member recorded patient time in the waiting room and the start and completion times of their visit. Thirty-day all-cause hospital encounters were abstracted from the EMR by the Center for Outcomes Research and Evaluation. Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at Atrium Health.^8,9

The research coordinator (N.K.) reviewed the EMR at 30 days (or last surgical follow-up if >30 days) to monitor adverse events. Patients were emailed a survey 30 days after surgery to assess for adverse events, including readmission to other facilities. Electronic reminders were sent if surveys were not returned within a week, and 2 attempts were made to contact the patient by telephone.

Patients were randomized to virtual or in-person visit using computer-generated (J.Z.) permuted block randomization with a random block size of 6 and 9 in a 2:1 fashion using the REDCap randomization module. The allocation sequence was concealed. Blinding of randomization after assignment was not possible, but primary outcome was abstracted by an independent data collection team. In-person visits were scheduled by the discharging nurse during business hours or via telephone call by the patient otherwise. Virtual-visit patients were given a brochure detailing how to access the platform and create a secure login and a telephone number to call to schedule an appointment. Patients who had not called within 3 business days after discharge were called up to 2 times by clinic staff to schedule the appointment.

The American Well (Amwell) platform was used, which offers both an app for phones and an online portal accessible on personal computers with video capabilities. The virtual visit began with the patient logging in at the prescheduled time and completing a video-based visit with the certified medical assistant who confirmed patient location, identification, and reviewed demographics and home medications. Patients were then placed in the virtual waiting room until the surgery team member picked them up and a separate video-based virtual visit was performed. At visit end, patients were sent educational material on their diagnosis via MyAtrium (part of MyChart; Epic)¹⁰ or US postal mail.

The primary outcome measure was 30-day postdischarge hospital encounter proportion, which included all-cause ED visits, inpatient admissions, and observation admissions, at any hospital or freestanding ED within the Atrium Health system. The primary outcome was electronically pulled for each patient using unique patient identifiers from clinical encounters within the EMR. The secondary outcome measures were total visit time, time spent with surgical team, and percentage of patients who completed a postdischarge visit. Esri’s ArcMap 10.5.1 with a network analyst extension was used to determine the quickest 1-way travel time (minutes) and distance (miles) from the patient’s billing address to the hospital where they had surgery. The analysis was conducted assuming the patients traveled at the posted speed limit. No time of day variable was used in this analysis.

Sample size estimation was performed prior to study initiation based on the primary outcome of postdischarge 30-day hospital encounter and a historical 20% incidence for usual in-person care. Target enrollment was 752 patients (501 virtual care and 251 in-person) to detect an a priori chosen noninferiority margin of 8% comparing virtual care vs in-person and achieving a power of 0.8 (α = .05). Interim analysis at 50% enrollment with review by the data safety monitoring board (DSMB) was preplanned.

Continuous data were compared with analysis of variance (including t test) or Kruskal-Wallis test. Categorical data were compared using Pearson χ² test or Fisher exact test. The primary analysis was an intention-to-treat (ITT) analyses, and a secondary per protocol (patients who completed their allocated intervention) analysis was performed for both primary and secondary outcomes. One-sided (for primary outcome) and 1-sided (for all other outcomes) 95% confidence interval for effect estimate between 2 groups was calculated. All other tests were 2-sided (significance level of .05). Statistical analysis was performed using SAS Enterprise Guide, version 7.1 (SAS Institute Inc) and R package (The R Foundation). The interim statistical analysis (November 2019) was reviewed by the DSMB, which recommended continuing to accrue patients. The trial was terminated in March 2020 prior to reaching target enrollment in the setting of COVID-19 secondary to the new higher risk of in-person encounters (DSMB members were in agreement). Owing to uncertainty about when it would be safe to re-enroll patients and the concern that it would not be clinically equivalent to compare pre-COVID outcomes with post-COVID outcomes (as clinical infrastructures have changed in our system), final study analysis and conditional probability analysis were performed so that the learnings could be shared with the surgical community, which has rapidly adopted the use of virtual visits.

Of the 1645 patients screened, 788 were eligible for study inclusion (Figure). Of the patients enrolled, 295 patients were randomized to the virtual visit arm (5 following randomization screen failures and 1 randomization error without consent) and 147 patients to the in-person arm (4 following randomization screen failures). Demographics were not different between the groups (Table 1). Most patients had commercial insurance/Medicare and 44% of patients had high school/some college as their highest level of education. Unplanned surgery was more common, and a little more than half of the patients had a cholecystectomy (Table 2).

A postoperative visit was completed for 289 patients (67%) overall, and 90 patients (21%) scheduled an appointment but were a “no show” to that visit, and this was not different between the 2 groups (Table 3). A total of 53 patients crossed over to the in-person follow-up group. The amount of time that patients spent with the surgical team member was not different in ITT and per protocol analysis. Quiz Ref IDThe median patient postoperative visit (47.5 vs 20.0 minutes), travel (12.7 vs 0.0 minutes), and total commitment (66.0 vs 14.0 minutes) time were significantly longer for in-person visits compared with virtual visits (Table 3). For patients who completed their allocated intervention, the median total commitment time saved was 63.9 minutes for virtual visits compared with in-person follow-up (Table 3). The all-cause 30-day hospital encounter incidence was not different between the 2 groups (12.8% vs 13.3%; P = .89). In the noninferiority analysis, the proportional difference in all-cause hospital encounter for virtual visits compared with in-person visits was −0.48% (1-sided 95% CI, −∞ to 5.2%). Patients who did not complete their allocated virtual visit or crossed over to in-person visits had an all-cause 30-day hospital encounter incidence of 14.3%. Most encounters were ED visits, and there were no 30-day mortalities.

Given currently detected proportions of hospital encounters in the 2 groups, assuming the observed treatment effect for the remainder of the trial, and allow data variability, we simulated 10 000 times to assess the probability of observing a difference of proportions less than 8%. We found 9608 of 10 000 (96%; 95% CI, 95.7%-96.5%) tests showed that the difference of hospital encounter incidence between 2 groups will be less than 8% if we continue the study to complete the target enrollment. This demonstrates a 4% probability (95% CI, 3.5%-4.3%) that the noninferiority margin could exceed the prespecified 8% proportional difference.

Quiz Ref IDIn this randomized clinical trial (RCT) of patients who underwent MIS appendectomy or cholecystectomy, we found no statistical difference in all-cause 30-day hospital encounter proportion, including inpatient admission, observation status, or ED encounter, between postdischarge follow-up using video-based virtual vs in-person follow-up. Although we did not reach target enrollment, noninferiority was demonstrated for postdischarge virtual visits in our study sample and was further supported via a simulation model. The all-cause 30-day encounter proportion in our study was lower than our historical cohort, likely reflecting ongoing improvements in care over time. To our knowledge, this is the first RCT using video-based virtual visits for immediate postdischarge encounters in general surgery patients in the United States. Our findings are similar to those of other studies that have evaluated virtual visits but were not RCTs, introducing potential bias in the patients who chose to participate in virtual visits. Prior studies were underpowered to detect differences in health care use. Telephone-only follow-ups in Veterans Affairs patients¹¹ after laparoscopic inguinal hernia repair identified no postoperative complications related to telephone follow-ups.¹¹ In a broader Veterans Affairs general surgery population comparing preintervention patients (n = 29) and postintervention patients (n = 171), there was no difference in proportion of ED presentation or readmission.¹² To our knowledge, the largest study to date¹³ in general surgery patients compared 485 preintervention patients with 233 eligible patients who chose to use videoconference calls for postoperative follow-up and with no difference in readmission, reoperation, or ED visits.¹³ All of these studies acknowledged that the voluntary, nonrandomized nature was a study limitation, and 2 specifically suggested that large RCTs would be helpful.^11-13

Randomized clinical trials that have compared virtual visits with in-person visits have been outside the postoperative time period or pilot studies. Remote video communication was evaluated for efficacy and patient satisfaction in a RCT for surveillance visits (>90 days after prostatectomy) in a urology clinic, finding equivalent efficiency (in time), and satisfaction but decreased costs for video visits.¹⁴ In a pilot study of 24 randomized orthopedic trauma patients, Skype follow-up for 2 of 4 total postdischarge visits was used with no difference in complications and no rehospitalizations.¹⁵

We did not reach target enrollment and terminated the study in the setting of the COVID-19 pandemic. To our knowledge, this remains the largest RCT of video-based virtual visits in general surgery patients. More than 1600 patients were screened, and a little more than half were potentially eligible. Quiz Ref IDThe most common exclusions were non–English-speaking patients, medical exclusions, and living in a state where our surgical team members were not licensed. Of the remaining patients, one-third declined to participate. The barriers to including non–English-speaking patients were unanticipated. Interpreters were readily available, but converting the third-party platform to other languages is a challenge that we continue to work through. Another 50 patients were not eligible owing to a technology barrier. It is important as telemedicine is adopted in the United States that careful attention is paid to the ability to provide care to all patients and avoid creating or worsening disparities.¹⁶

Regulations regarding physician and APP licensing at the state level present barriers to virtual care. More than 100 patients were not eligible owing to residing in a state in which clinicians were not licensed. In these cases, the burden of travel is on the patient, potentially causing patients to not follow up if the burden is too great. These barriers were temporarily lifted during the COVID-19 pandemic.¹⁷ The Interstate Medical Licensure Compact is an expedited pathway that allows physicians to get licensure in multiple states for participating states.¹⁸ The Interstate Medical Licensure Compact became operational in April 2017 and has 29 active states. North Carolina and neighboring states are not participants, creating barriers to providing care to patients across state lines in our region. Future work to ensure that regulation of telemedicine does not unduly burden the patient will be critical to providing patient-centered and best practice care.

Our study focused on providing postdischarge care to patients who underwent low-risk minimally invasive procedures; the postdischarge phase of care was identified partly because of its neutrality regarding physician professional fees, which are covered in the global period. Payer reimbursement for telemedicine visits is in evolution. Historically, there has not been payer parity for medical care provided via telemedicine in North Carolina, although that has temporarily changed in the setting of the COVID-19 environment.⁶ Future legislation to ensure payer parity, particularly outside of the global period, will be critical to expanding telemedicine.

Virtual visits eliminated travel time and significantly decreased wait time for postdischarge appointments but did not decrease the amount of time patients were able to spend with the surgery team member. The time savings associated with virtual visits is one potential avenue to decrease disparities in access to care, as patients with transportation challenges, difficulty getting time off of work, or need for childcare may realize greater benefits.¹⁹ However, there was no difference in percentage of patients completing a postdischarge appointment, suggesting that factors other than travel time likely affect patient decisions to complete recommended follow-up care. Patients and clinicians should be reassured that the critical visit portion, time together discussing medical needs, is preserved.

The strength of this study is the randomized control study design and its setting within a large health system with a robust electronic data warehouse. Limitations include that we did not reach target enrollment, inability to include non–English-speaking patients, and the exclusion of patients who were not able or unwilling to be allocated to virtual visits. It is possible that some patients had encounters outside our system, and we attempted to minimize this possibility with 30-day patient surveys. Additionally, our study focused on low-risk, minimally invasive procedures and may not be generalizable to higher-risk surgeries. Future studies to better understand the association between patient factors, including primary language, and use and perceptions of telemedicine are needed.

To our knowledge, this is the first randomized clinical trial investigating the use of video visits compared with in-person visits for immediate postdischarge surgical follow-up after low-risk, minimally invasive surgery. Although we did not reach target enrollment, our current and simulated results did not cross the noninferiority margin, favoring noninferiority for postdischarge virtual visits. These results can inform future policy decisions regarding use of video-based telehealth in surgical patients.

Corresponding Author: Caroline E. Reinke, MD, MSHP, Carolinas Medical Center, 1025 Morehead Medical Plaza, Ste 300, Charlotte, NC 28204 (caroline.e.reinke@atriumhealth.org).

Accepted for Publication: October 23, 2020.

Published Online: January 13, 2021. doi:10.1001/jamasurg.2020.6265

Author Contributions: Ms Kaiser and Dr Zhao had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Harkey, Zhao, Matthews, Kelz, Reinke.

Acquisition, analysis, or interpretation of data: Harkey, Kaiser, Zhao, Hetherington, Gutnik, Kelz, Reinke.

Drafting of the manuscript: Harkey, Kaiser, Reinke.

Critical revision of the manuscript for important intellectual content: Harkey, Zhao, Hetherington, Gutnik, Matthews, Kelz, Reinke.

Statistical analysis: Zhao.

Obtained funding: Reinke.

Administrative, technical, or material support: Harkey, Kaiser, Hetherington, Gutnik, Reinke.

Supervision: Harkey, Matthews, Kelz, Reinke.

Conflict of Interest Disclosures: Dr Kelz reported research funding from the National Institutes of Health. No other disclosures were reported.

Funding/Support: This work was funded by the American College of Surgeons Franklin H. Martin Faculty Research Fellowship.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We thank the Carolinas Center for Surgical Outcomes Science and Acute Care Surgery research teams who participated in approaching and enrolling patients; Charity Patterson, PhD, University of Pittsburgh, who assisted in initial study design and power analysis; surgical colleagues in the Acute Care Surgery Division and General Surgery Section; Mike Inman, Atrium Health, who provided travel time analysis; and data safety monitoring board members who participated in ensuring study safety. No compensation from a funding source was received for these contributions.