Effect of In-Person vs Video Training and Access to All Functions vs a Limited Subset of Functions on Portal Use Among Inpatients: A Randomized Clinical Trial

Key Points

Question  How do training and technology functions of an inpatient portal affect portal use, patient satisfaction, and patient involvement in their care while hospitalized?

Findings  In this randomized clinical trial of 2892 participants, patients who had access to in-person training (compared with a training video) or more portal functions (compared with a limited set of functions) had significantly greater inpatient portal use.

Meaning  These findings suggest that providing either in-person training or advanced portal functions can be a powerful approach to increase patients’ engagement with portals.

Importance  Inpatient portals provide patients with clinical data and information about their care and have the potential to influence patient engagement and experience. Although significant resources have been devoted to implementing these portals, evaluation of their effects has been limited.

Objective  To assess the effects of patient training and portal functionality on use of an inpatient portal and on patient satisfaction and involvement with care.

Design, Setting, and Participants  This randomized clinical trial was conducted from December 15, 2016, to August 31, 2019, at 6 noncancer hospitals that were part of a single health care system. Patients who were at least 18 years of age, identified English as their preferred language, were not involuntarily confined or detained, and agreed to be provided a tablet to access the inpatient portal during their stay were eligible for participation. Data were analyzed from May 1, 2019, to March 15, 2021.

Interventions  A 2 × 2 factorial intervention design was used to compare 2 levels of a training intervention (touch intervention, consisting of in-person training vs built-in video tutorial) and 2 levels of portal function availability (tech intervention) within an inpatient portal (all functions operational vs a limited subset of functions).

Main Outcomes and Measures  The primary outcomes were inpatient portal use, measured by frequency and comprehensiveness of use, and patients’ satisfaction and involvement with their care.

Results  Of 2892 participants, 1641 were women (56.7%) with a median age of 47.0 (95% CI, 46.0-48.0) years. Most patients were White (2221 [76.8%]). The median Charlson Comorbidity Index was 1 (95% CI, 1-1) and the median length of stay was 6 (95% CI, 6-7) days. Notably, the in-person training intervention was found to significantly increase inpatient portal use (incidence rate ratio, 1.34 [95% CI, 1.25-1.44]) compared with the video tutorial. Patients who received in-person training had significantly higher odds of being comprehensive portal users than those who received the video tutorial (odds ratio, 20.75 [95% CI, 16.49-26.10]). Among patients who received the full-tech intervention, those who also received the in-person intervention used the portal more frequently (incidence rate ratio, 1.36 [95% CI, 1.25-1.48]) and more comprehensively (odds ratio, 22.52; [95% CI, 17.13-29.62]) than those who received the video tutorial. Patients who received in-person training had higher odds (OR, 2.01 [95% CI, 1.16-3.50]) of reporting being satisfied in the 6-month postdischarge survey. Similarly, patients who received the full-tech intervention had higher odds (OR, 2.06 [95%CI, 1.42-2.99]) of reporting being satisfied in the 6-month postdischarge survey.

Conclusions and Relevance  Providing in-person training or robust portal functionality increased inpatient engagement with the portal during the hospital stay. The effects of the training intervention suggest that providing personalized training to support use of this health information technology can be a powerful approach to increase patient engagement via portals.

Trial Registration  ClinicalTrials.gov Identifier: NCT02943109

Inpatient portals designed for the acute care setting can provide patients with both clinical data and information about their care, potentially ameliorating information gaps while empowering patients during hospitalization.^1-3 These portals have functions typical of outpatient portals such as allowing patients to view test results and send messages, which have shown benefits for patient engagement, satisfaction, and delivery of patient-centered care,^4,5 but inpatient portals also have unique functions such as allowing patients to view pictures of their health care team, review their schedule, and access discharge information.³

Evidence suggests that the introduction of patient-facing health information technology such as inpatient portals may fall short of its potential when they are not presented or supported in ways that address barriers to patients’ use, such as lack of knowledge about how to use a given tool and low perceived usefulness.^6-11 Failure to provide patients with the necessary support to facilitate engagement with portals may result in low levels of use of the tool or use of only a limited range of available functions.^12,13

Although hospitals have devoted significant resources to implement patient portals,^14,15 the evaluation of inpatient portal use thus far has been limited in scale and scope,^6,16-22 reducing the value of such findings in the context of clinical efforts to implement commercial products.^3,7 To address this gap, we conducted a pragmatic randomized clinical trial to evaluate the effect of training and the availability of portal functions on inpatient portal use and on patient satisfaction and involvement with their care.

The High Touch and High Tech Study²³ was an investigator-initiated, large-scale randomized clinical trial conducted across 6 hospitals that were part of a single health care system sharing a common electronic health record platform. The study was reviewed and approved by the institutional review board of The Ohio State University. All patients provided written informed consent, including a release of access to their electronic health records to retrieve patient portal use data. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline. The statistical analysis plan and final study protocol are available in Supplement 1.

All patients admitted to 1 of the 6 noncancer hospitals at The Ohio State University Wexner Medical Center (OSUWMC) who were aged at least 18 years, identified English as their preferred language, and were not involuntarily confined or detained were offered standardized tablet computers to use during their hospital stay. Patients could choose to decline the tablet. Through the tablet, patients had access to the inpatient portal MyChart Bedside (Epic Systems); patients may also have had access to the MyChart outpatient portal (Epic Systems). The trial was conducted from December 15, 2016, to August 31, 2019, after the implementation of MyChart Bedside across the 6 hospitals.

The eligible study population included 16 110 patients hospitalized across the 26 inpatient, progressive, or rehabilitation care units of the hospitals who had agreed to receive a tablet and consented to study participation.²⁴ Of the eligible study population, 2944 patients declined to participate, and 9384 were excluded for other reasons, including patients receiving a tablet but being discharged before completion of study consent or Health Insurance Portability and Accountability Act authorization or returning tablets before study introduction by research staff. A total of 3782 patients were enrolled to achieve 80% power to identify changes ranging from 2.2% to 3.8% across the panel of patient satisfaction scores from the Hospital Consumer Assessment of Healthcare Providers and Systems Survey as a baseline for outcome measures. Race and ethnicity data were obtained from the electronic health records of the institution. These data were deemed relevant to the study because they provided basic demographic information about the patient population.

The 2 × 2 factorial design of the trial was defined by both technology and training dimensions with patients randomized to 1 of 4 groups: (1) full technology and high level of training (full-tech, high-touch); (2) full technology and low level of training (full-tech, low-touch); (3) less technology and high level of training (lite-tech, high-touch); or (4) less technology and low level of training (lite-tech, low-touch). The tech intervention groups specified 2 versions of the inpatient portal application. In the full-tech version, the application made 10 functions available to the patient (Dining on Demand, Bedside Tutorial, To Learn, Happening Soon, Taking Care of Me, Messages, MyHealth, Notes, I Would Like, and MyChart [outpatient portal]). In the lite-tech version, 3 functions were available (Dining on Demand, Bedside Tutorial, and To Learn) (additional information is provided in the eMethods and eTable 1 in Supplement 2).

The touch intervention groups specified the training offered on how to use the inpatient portal. The training was delivered by a study team member identified as a technology navigator. For patients assigned to the high-touch groups, the in-person training involved reviewing the functions available and supervising patients as they navigated tasks in the portal, engaging in both audit and feedback about task success. This in-person training protocol was developed with input from patients, clinicians, research team members, and education specialists as described in the study protocol.²³ Patients in group 1 received access to the full-tech training video (11 minutes, 17 seconds) and a visit by a technology navigator that lasted a mean (SD) of 13.3 (5.5) minutes. Patients in group 2 received access to the full-tech training video and a brief visit from a study team member to introduce the study. Patients in group 3 received access to the lite-tech training video (4 minutes, 29 seconds) and a visit by a technology navigator that lasted a mean (SD) of 6.9 (3.6) minutes. Patients in group 4 received access to the lite-tech training video and a brief visit from a study team member to introduce the study.

Participants were randomized to 1 of the 4 study groups in a 2-step process. First, all patients were randomly assigned to 1 of the 2 technology groups at the time of tablet provisioning by a member of the nursing team. After provisioning, the technology navigator visited eligible patients to formally invite their voluntary participation in the study. Training assignment was then randomly assigned at the time of consent and patient enrollment in the study via the tablet. Study allocation was later rebalanced to ensure sufficient numbers of patients in the high-touch treatment groups owing to patient loss to follow-up in these groups caused by patient discharge before intervention delivery, patients not being present or available when technology navigators were rounding to deliver the intervention (eg, patient out of room for a procedure or test, care team members in room taking precedence over study participation), and tablets being returned before delivery of the intervention (eg, the patient changed his or her mind about using the tablet).

The primary outcomes were inpatient portal use, defined as frequency and comprehensiveness of portal use, and patient satisfaction and involvement with their care for patients receiving high-touch vs low-touch (pooled across tech randomization) and for patients receiving full-tech vs lite-tech (pooled across touch randomization). Portal use outcomes were measured based on audit log files.^25,26 Frequency of portal use was defined as the number of unique sessions using the portal during the patient’s hospital stay. Comprehensiveness of use was dependent on tech assignment and defined by the diversity of functions used during the stay, with a comprehensive user defined by use of all 3 lite-tech functions (for lite-tech participants) or 8 or more of the 10 full-tech functions (for full-tech participants) (per section 1.5.2 in the statistical analysis plan in Supplement 1).

Patient satisfaction and involvement with care were measured using survey questions including Likert scale and binary choice items administered at 3 different time points: admission, 15 days post discharge, and 6 months post discharge. Questions about satisfaction were derived from the Hospital Consumer Assessment of Healthcare Providers and Systems Survey²⁷; questions about involvement in care were derived from the Health Information National Trends Survey.²⁸

Data were analyzed from May 1, 2019, to March 15, 2021. The primary outcomes included data reported as counts as well as dichotomous and continuous variables. We therefore used 3 different analytic approaches appropriate to these data types. We selected a negative binomial model for the frequency analysis from an assessment of the data distribution, and more specifically, overdispersion identified in the use measures. These results are presented using an incidence rate ratio (IRR). We used logistic regression to test the effect of the intervention on comprehensiveness of use, patient satisfaction, and patient involvement in care. These results are presented as odds ratios (ORs).

Interaction effects between both interventions are reported as a ratio of the IRRs or ORs based on the estimated model. All analytic models were specified with the lite-tech, low-touch study group as the reference group and were tested with and without an interaction of tech and touch groups, when appropriate. Estimates of the patient satisfaction and involvement with care outcomes were adjusted for their corresponding baseline levels (ie, at admission) in our models.

An analysis was performed among patients in the full-tech subsample to assess the effect of the touch intervention on inpatient portal use. Binary outcomes were again modeled using logistic regression and reported as ORs. The counts of portal sessions were modeled using negative binomial regression and reported as IRRs. Additionally, the frequency of use of each MyChart Bedside function was calculated as a percentage relative to all other functions available to users of the full version of the tablet and was assessed via a fractional logistic regression model.

All available data were used for the analyses, and no imputation was used. Outcomes were reported as ORs or IRRs with corresponding 95% CIs. We report 95% CIs for all point estimates and did not make formal statistical adjustments to them for multiple hypothesis testing. All analyses included cluster-robust SEs on the basis of facility at study enrollment to account for heterogeneity in patient characteristics across the 6 hospitals. All analyses were conducted using Stata MP, version 14.2 (StataCorp LLC).

Of the 2892 patients randomized into the study, 1641 were women (56.7%) and 1251 were men (43.3%), with a median age of 47.0 (95% CI, 46.0-48.0) years. In terms of race and ethnicity, 550 patients were Black (19.0%), 2221 were White (76.8%), and 121 were other races or ethnicities (including African, American Indian or Alaska Native, Asian or Asian American, multiple races or ethnicities, and unknown race or ethnicity) (4.2%). The median Charlson Comorbidity Index was 1 (95% CI, 1-1) and the median length of stay was 6 (95% CI, 6-7) days (Table 1). A total of 3782 patients were enrolled; of these, 890 were excluded from further analyses because they did not use MyChart Bedside during their stay, requested to be switched to a tablet with full-tech capability (per OSUWMC policy), or had an admission duration of fewer than 3 days, which was defined in the study protocol as the length of time needed for patients to have sufficient opportunity to engage with the tablet (Figure 1).²³ Survey response rates were similar across the 4 groups, with responses from 2835 of 2892 patients (98.0%) for the admission survey, 459 of 2892 (15.9%) for the 15-day postdischarge survey, and 673 of 2892 (23.3%) for the 6-month postdischarge survey; however, the full-tech, high-touch group had a relatively higher response rate (154 of 624 [24.7%]) for the 15-day postdischarge survey (eTable 2 in Supplement 2).

The results of all primary outcome measures for the touch and tech interventions, along with IRRs or ORs and 95% CIs, are presented in Figure 2. A summary of our 2 × 2 factorial study design linked to study results is presented in Figure 3.

As shown in Table 2, patients who received the high-touch intervention had significantly higher frequency of inpatient portal use compared with patients who received the low-touch intervention (IRR, 1.34 [95% CI, 1.25-1.44]). Patients who received the high-touch intervention also had significantly higher odds of being classified as comprehensive inpatient portal users (ie, used all 3 lite-tech functions or ≥8 full-tech functions) compared with patients who received the low-touch intervention (OR, 20.75 [95% CI, 16.49-26.10]).

Overall, patients who received the full-tech intervention had significantly higher frequency of inpatient portal use than those who received the lite-tech intervention (IRR, 1.47 [95% CI, 1.25-1.72]). However, patients with the full-tech intervention had lower odds of being classified as comprehensive inpatient portal users (OR, 0.79 [95% CI, 0.72-0.87]).

The high-touch intervention significantly increased the odds of a patient reporting being satisfied in the 6-month postdischarge survey (OR, 2.01 [95% CI, 1.16-3.50]) (Figure 2) (for details see eTable 4 in Supplement 2). Patients who received in-person training also had higher odds of being satisfied with interactions with clinicians in both the 15-day (OR, 1.78 [95% CI, 1.09-2.90]) and 6-month (OR, 1.28 [95% CI, 1.08-1.53]) postdischarge surveys. Patients who received the high-touch intervention had higher odds of being involved in their care by having all their questions answered (OR, 1.36 [95% CI, 1.10-1.67]) (Figure 2), but not by other measures of involvement in care (eTable 5 in Supplement 2).

The full-tech intervention increased the odds of a patient reporting being fully satisfied in the 6-month postdischarge survey (OR, 2.06 [95% CI, 1.42-2.99]) (Figure 2). Patients who received the full-tech intervention also had higher odds of using the tablet to access MyChart Bedside (OR, 2.10 [95% CI, 1.10-4.01]), and lower odds of using the tablet to play games (OR, 0.74 [95% CI, 0.69-0.80]) (eTable 5 in Supplement 2).

In combination, these interventions demonstrated a substitution effect such that patients in either the full-tech or high-touch groups had higher satisfaction at 6 months post discharge (ratio of ORs, 0.50 [95% CI, 0.29-0.85]). Furthermore, patients who received the full-tech, high-touch interventions had lower odds of needing to ask a staff member questions related to their care compared with those in the lite-tech, low-touch group (ratio of ORs, 0.64 [95% CI, 0.45-0.92]) and higher odds of using the tablet to research heath issues (ratio of ORs, 1.77 [95% CI, 1.05-2.98]) (details are provided in eTables 4 and 5 in Supplement 2). There were no significant interaction effects between the 2 interventions for frequency of use (IRR, 1.06 [95% CI, 0.98-1.15]) or comprehensive use (OR, 1.34 [95% CI, 0.94-1.90]) (eTable 3 in Supplement 2).

The results of subgroup analyses are shown in eTable 6 in Supplement 2. Within the full-tech subsample, patients receiving the high-touch intervention had higher frequency of use (IRR, 1.36 [95% CI, 1.25-1.48]) and greater odds of being comprehensive users of the inpatient portal (OR, 22.52 [95% CI, 17.13-29.62]) than patients who received the low-touch intervention, with use by available function varying by intervention group.

This randomized clinical trial found that both technology and training interventions affected patients’ use of an inpatient portal, as well as mixed evidence that the technology and training interventions can positively affect patient satisfaction and involvement in care in both the short and long term after hospital discharge. First, we found that having more functions available within the inpatient portal was associated with greater patient use of this technology. This finding is consistent with previous reports that patients value inpatient portal technology for both information and care team access.^7,16,29-32 We also found that patients who received in-person training used the portal more than patients who were only directed toward tablet- and video-based training. Previous studies of outpatient portals have identified lack of training as a significant barrier to portal use, reporting that patients had difficulty both appreciating the utility of a portal and learning to use its functions.^33-36 In addressing this gap, our training intervention may also have signaled to patients that the care team and hospital valued the technology and its use.³⁷ Although delivering in-person training is not cost neutral, our findings may help inform decisions about whether a cost is justified if it contributes to higher levels of patient engagement. Furthermore, because recent research has shown that patients hospitalized for COVID-19 or heart failure who had active patient portal accounts had shorter hospital stays,³⁸ this training could have cost savings implications.

Of note, both the technology and training interventions showed positive effects on patient satisfaction, consistent with prior evidence linking satisfaction to portal use,^3,39,40 tablet availability,⁴¹ and perceptions about portals’ usefulness.⁴² For instance, patients receiving the in-person training may have learned to use functions that informed them about their health status and care plan,^31,32 which enabled them to ask more targeted questions, and this could have increased their satisfaction post discharge. More research is needed to understand the value that hospitals might place on patient satisfaction and how this value relates to the cost of implementing either the training or the technology.

It is clear from our study that patients used the patient portal more when they had formal training and when the portal had advanced functions available. Although delivering in-person training may be challenging given the ongoing COVID-19 pandemic, the value of training revealed by our study suggests that offering in-person training, even if provided virtually, should be considered.

This study has some limitations. Because this was a trial deployed in the context of a large-scale, operational academic medical center, factors outside the control of the study team had a substantial impact on the study design. Most notably, the choice was made by OSUWMC to expand tablet access across the institution, enabling our original study of 12 hospital units to extend to 26 units across 6 hospitals. We chose the approach of proceeding without cluster randomization given the complexity associated with revising our randomization strategy. Consequently, we used cluster-robust SEs as a statistical correction for this design limitation. Given that management of tablet provisioning was consistent across units, we are confident that our results are robust against bias from unstratified site randomization.

In addition, with the decision to distribute tablets to all eligible patients, we were unable to compare the experiences of patients who received a tablet with those of patients who received no tablet. We were therefore unable to assess whether factors such as patient satisfaction and involvement in care would have been worse without portal access. Because prior studies have found that enhanced patient self-management can have positive effects on control of chronic illness^43-46 and that access to their medical record, personalized health information, and reminders lead to increases in guideline adherence, patient satisfaction, and health status,^47-49 these associations should be examined in future work. Finally, our study was not designed to detect differences in the primary outcomes based on patients’ demographic characteristics, such as age and race or ethnicity; thus, exploration of these factors will be critical in new research.⁵⁰

In this randomized clinical trial, both training and the availability of advanced functions fostered greater use of an inpatient portal and showed some benefits regarding patient satisfaction and involvement in care. Investments in both human and technological resources may therefore be needed to ensure that these types of patient-facing health information technology tools can be leveraged to increase patient engagement in their care overall and ultimately have a positive effect on both care and experiential outcomes.

Accepted for Publication: July 22, 2022.

Published: September 13, 2022. doi:10.1001/jamanetworkopen.2022.31321

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 McAlearney AS et al. JAMA Network Open.

Corresponding Author: Ann Scheck McAlearney, ScD, MS, Center for the Advancement of Team Science, Analytics, and Systems Thinking, College of Medicine, The Ohio State University, 700 Ackerman Rd, Ste 4000, Columbus, OH 43202 (ann.mcalearney@osumc.edu).

Author Contributions: Drs McAlearney and Huerta had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: McAlearney, Walker, Sieck, Fareed, Hefner, Gaughan, Moffatt-Bruce, Rizer, Huerta.

Acquisition, analysis, or interpretation of data: McAlearney, Walker, Sieck, Fareed, MacEwan, Hefner, Di Tosto, Gaughan, Sova, Rush, Huerta.

Drafting of the manuscript: McAlearney, Walker, Sieck, Fareed, MacEwan, Hefner, Di Tosto, Gaughan, Rush, Moffatt-Bruce, Huerta.

Critical revision of the manuscript for important intellectual content: McAlearney, Walker, Sieck, Fareed, MacEwan, Hefner, Gaughan, Sova, Rush, Rizer, Huerta.

Statistical analysis: Walker, Fareed, Di Tosto, Sova, Huerta.

Obtained funding: McAlearney, Sieck, Huerta.

Administrative, technical, or material support: McAlearney, Sieck, Hefner, Gaughan, Sova, Rush, Rizer, Huerta.

Supervision: McAlearney, Fareed, Gaughan, Moffatt-Bruce, Huerta.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by grants R01HS024091, R21HS024767, and P30HS024379 from the Agency for Healthcare Research and Quality.

Role of the Funder/Sponsor: The sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We thank our research team members who assisted with this project, including Seth Scarborough, MAS (all from The Ohio State University). These individuals were not compensated financially beyond their salaries.

Additional Information: We thank all of the study participants for their involvment in the study.