[Skip to Content]
[Skip to Content Landing]
Figure 1.
CONSORT Flow Study Diagram
CONSORT Flow Study Diagram

t-HELP indicates Tailored Family-Involved Hospital Elder Life Program.

Figure 2.
Change in Physical and Cognitive Function Over Time From Baseline to 30 Days After Discharge
Change in Physical and Cognitive Function Over Time From Baseline to 30 Days After Discharge

A, The Barthel Index or activity of daily living (ADL) scores of patients in the usual care group were lower than those in the Tailored, Family-Involved Hospital Elder Life Program (t-HELP) group at discharge (T2) and 30 days after discharge (T3). B, The instrumental activity of daily living (IADL) scores of patients in the usual care group were lower than those in the t-HELP group at T2 and T3. C, The FRAIL (Fatigue, Resistance, Ambulation, Illnesses, and Loss of Weight) scale scores in the usual care group were higher than those in the t-HELP group at T2 and T3. D, The proportion of intact cognition (SPMSQ [Short Portable Mental Status Questionnaire] score) of patients in the t-HELP group was higher than that in the usual care group at T2 and T3. T1 indicates period before operation.

Table 1.  
Baseline Patient Characteristics
Baseline Patient Characteristics
Table 2.  
Primary Outcomes Between Groupsa
Primary Outcomes Between Groupsa
Table 3.  
Secondary Outcomes Between Groups, by Per-Protocol Analysis
Secondary Outcomes Between Groups, by Per-Protocol Analysis
1.
Beliveau  MM, Multach  M.  Perioperative care for the elderly patient.  Med Clin North Am. 2003;87(1):273-289. doi:10.1016/S0025-7125(02)00155-4PubMedGoogle ScholarCrossref
2.
Lu  WL, Kumar  DS, Zheng  SB.  Analysis of perioperative complications and mortality in hospitalized elderly surgical patients  [in Chinese].  Chin J Geriatr. 2013;32(12):1319-1321. doi:10.3760/cma.j.issn.0254-9026.2013.12.017Google Scholar
3.
Inouye  SK, Westendorp  RG, Saczynski  JS.  Delirium in elderly people.  Lancet. 2014;383(9920):911-922. doi:10.1016/S0140-6736(13)60688-1PubMedGoogle ScholarCrossref
4.
Pitkala  KH, Laurila  JV, Strandberg  TE, Tilvis  RS.  Prognostic significance of delirium in frail older people.  Dement Geriatr Cogn Disord. 2005;19(2-3):158-163. doi:10.1159/000082888PubMedGoogle ScholarCrossref
5.
Leslie  DL, Zhang  Y, Holford  TR, Bogardus  ST, Leo-Summers  LS, Inouye  SK.  Premature death associated with delirium at 1-year follow-up.  Arch Intern Med. 2005;165(14):1657-1662. doi:10.1001/archinte.165.14.1657PubMedGoogle ScholarCrossref
6.
Davis  DH, Muniz-Terrera  G, Keage  HA,  et al; Epidemiological Clinicopathological Studies in Europe (EClipSE) Collaborative Members.  Association of delirium with cognitive decline in late life: a neuropathologic study of 3 population-based cohort studies.  JAMA Psychiatry. 2017;74(3):244-251. doi:10.1001/jamapsychiatry.2016.3423PubMedGoogle ScholarCrossref
7.
Leslie  DL, Marcantonio  ER, Zhang  Y, Leo-Summers  L, Inouye  SK.  One-year health care costs associated with delirium in the elderly population.  Arch Intern Med. 2008;168(1):27-32. doi:10.1001/archinternmed.2007.4PubMedGoogle ScholarCrossref
8.
Gleason  LJ, Schmitt  EM, Kosar  CM,  et al.  Effect of delirium and other major complications on outcomes after elective surgery in older adults.  JAMA Surg. 2015;150(12):1134-1140. doi:10.1001/jamasurg.2015.2606PubMedGoogle ScholarCrossref
9.
Siddiqi  N, House  AO, Holmes  JD.  Occurrence and outcome of delirium in medical in-patients: a systematic literature review.  Age Ageing. 2006;35(4):350-364. doi:10.1093/ageing/afl005PubMedGoogle ScholarCrossref
10.
Fong  TG, Tulebaev  SR, Inouye  SK.  Delirium in elderly adults: diagnosis, prevention and treatment.  Nat Rev Neurol. 2009;5(4):210-220. doi:10.1038/nrneurol.2009.24PubMedGoogle ScholarCrossref
11.
Oh  ES, Fong  TG, Hshieh  TT, Inouye  SK.  Delirium in older persons: advances in diagnosis and treatment.  JAMA. 2017;318(12):1161-1174. doi:10.1001/jama.2017.12067PubMedGoogle ScholarCrossref
12.
Inouye  SK, Bogardus  ST  Jr, Charpentier  PA,  et al.  A multicomponent intervention to prevent delirium in hospitalized older patients.  N Engl J Med. 1999;340(9):669-676. doi:10.1056/NEJM199903043400901PubMedGoogle ScholarCrossref
13.
Inouye  SK.  Prevention of delirium in hospitalized older patients: risk factors and targeted intervention strategies.  Ann Med. 2000;32(4):257-263. doi:10.3109/07853890009011770PubMedGoogle ScholarCrossref
14.
Inouye  SK, Bogardus  ST  Jr, Baker  DI, Leo-Summers  L, Cooney  LM  Jr; Hospital Elder Life Program.  The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients.  J Am Geriatr Soc. 2000;48(12):1697-1706. doi:10.1111/j.1532-5415.2000.tb03885.xPubMedGoogle ScholarCrossref
15.
Yue  J, Tabloski  P, Dowal  SL, Puelle  MR, Nandan  R, Inouye  SK.  NICE to HELP: operationalizing National Institute for Health and Clinical Excellence guidelines to improve clinical practice.  J Am Geriatr Soc. 2014;62(4):754-761. doi:10.1111/jgs.12768PubMedGoogle ScholarCrossref
16.
McDonald  HP, Garg  AX, Haynes  RB.  Interventions to enhance patient adherence to medication prescriptions: scientific review  [published correction appears in JAMA. 2003;289(24):3242].  JAMA. 2002;288(22):2868-2879. doi:10.1001/jama.288.22.2868PubMedGoogle ScholarCrossref
17.
Inouye  SK, van Dyck  CH, Alessi  CA, Balkin  S, Siegal  AP, Horwitz  RI.  Clarifying confusion: the Confusion Assessment Method: a new method for detection of delirium.  Ann Intern Med. 1990;113(12):941-948. doi:10.7326/0003-4819-113-12-941PubMedGoogle ScholarCrossref
18.
Inouye  SK, Kosar  CM, Tommet  D,  et al.  The CAM-S: development and validation of a new scoring system for delirium severity in 2 cohorts.  Ann Intern Med. 2014;160(8):526-533. doi:10.7326/M13-1927PubMedGoogle ScholarCrossref
19.
Gao  LL, Feng  DM, Wang  RH, Jiang  YY, Zhang  M, Yue  JR.  Validity and reliability of the Chinese version of short form of confused assessment method for the detection of delirium in the elderly  [in Chinese].  Shiyong Laonian Yixue. 2019;33(2):133-136. doi:10.3969/j.issn.1003-9198.2019.02.007Google Scholar
20.
Wu  YJ, Shi  YZ, Wang  MJ, Cao  J, Zu  YB, Shen  Y.  Chinese version of the Memorial Delirium Assessment Scale (MDAS): clinical validity and reliability in old postoperative patients  [in Chinese].  Chin J Mental Health. 2014;28(12):937-941. doi:10.3969/j.issn.1000-6729.2014.12.011Google Scholar
21.
Guo  JW.  A preliminary study on simple determination of cognitive impairment in the elderly  [in Chinese].  Chin J Mental Health. 1998(5):268-269. http://www.cnki.com.cn/Article/CJFDTotal-ZXWS805.004.htmGoogle Scholar
22.
Formiga  F, Fort  I, Robles  MJ, Rodriguez  D, Regalado  P.  Lower Barthel Index scores predict less prescription of pharmacological therapy in elderly patients with Alzheimer disease.  Dement Geriatr Cogn Disord. 2010;29(3):198-203. doi:10.1159/000278348PubMedGoogle ScholarCrossref
23.
Park  B, Jun  JK, Park  J.  Cognitive impairment and depression in the early 60s: which is more problematic in terms of instrumental activities of daily living?  Geriatr Gerontol Int. 2014;14(1):62-70. doi:10.1111/ggi.12055PubMedGoogle ScholarCrossref
24.
Dong  L, Qiao  X, Tian  X,  et al.  Cross-cultural adaptation and validation of the FRAIL scale in Chinese community-dwelling older adults.  J Am Med Dir Assoc. 2018;19(1):12-17. doi:10.1016/j.jamda.2017.06.011PubMedGoogle ScholarCrossref
25.
Marcantonio  ER, Flacker  JM, Wright  RJ, Resnick  NM.  Reducing delirium after hip fracture: a randomized trial.  J Am Geriatr Soc. 2001;49(5):516-522. doi:10.1046/j.1532-5415.2001.49108.xPubMedGoogle ScholarCrossref
26.
Li  P, Redden  DT.  Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials.  BMC Med Res Methodol. 2015;15:38-49. doi:10.1186/s12874-015-0026-xPubMedGoogle ScholarCrossref
27.
Hshieh  TT, Yue  J, Oh  E,  et al.  Effectiveness of multicomponent nonpharmacological delirium interventions: a meta-analysis.  JAMA Intern Med. 2015;175(4):512-520. doi:10.1001/jamainternmed.2014.7779PubMedGoogle ScholarCrossref
28.
Stall  NM, Campbell  A, Reddy  M, Rochon  PA.  Words matter: the language of family caregiving  [published online May 23, 2019].  J Am Geriatr Soc. 2019. doi:10.1111/jgs.15988PubMedGoogle Scholar
29.
Mohanty  S, Rosenthal  RA, Russell  MM, Neuman  MD, Ko  CY, Esnaola  NF.  Optimal perioperative management of the geriatric patient: a best practices guideline from the American College of Surgeons NSQIP and the American Geriatrics Society.  J Am Coll Surg. 2016;222(5):930-947. doi:10.1016/j.jamcollsurg.2015.12.026PubMedGoogle ScholarCrossref
30.
Dong  B, Yue  J, Cao  L,  et al; Journal of the American Geriatrics Society.  Transformation of a geriatric department in China.  J Am Geriatr Soc. 2018;66(1):184-190. doi:10.1111/jgs.15217PubMedGoogle ScholarCrossref
31.
Hshieh  TT, Yang  T, Gartaganis  SL, Yue  J, Inouye  SK.  Hospital Elder Life Program: systematic review and meta-analysis of effectiveness.  Am J Geriatr Psychiatry. 2018;26(10):1015-1033. doi:10.1016/j.jagp.2018.06.007PubMedGoogle ScholarCrossref
32.
Chen  CC, Li  HC, Liang  JT,  et al.  Effect of a modified Hospital Elder Life Program on delirium and length of hospital stay in patients undergoing abdominal surgery: a cluster randomized clinical trial.  JAMA Surg. 2017;152(9):827-834. doi:10.1001/jamasurg.2017.1083PubMedGoogle ScholarCrossref
33.
Young  J, Cheater  F, Collinson  M,  et al.  Prevention of delirium (POD) for older people in hospital: study protocol for a randomised controlled feasibility trial.  Trials. 2015;16(1):340-351. doi:10.1186/s13063-015-0847-2PubMedGoogle ScholarCrossref
34.
Heim  N, van Stel  HF, Ettema  RG, van der Mast  RC, Inouye  SK, Schuurmans  MJ.  HELP! problems in executing a pragmatic, randomized, stepped wedge trial on the Hospital Elder Life Program to prevent delirium in older patients.  Trials. 2017;18(1):220-231. doi:10.1186/s13063-017-1933-4PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,206
    Citations 0
    Original Investigation
    October 21, 2019

    Effect of the Tailored, Family-Involved Hospital Elder Life Program on Postoperative Delirium and Function in Older Adults: A Randomized Clinical Trial

    Author Affiliations
    • 1Department of Geriatrics and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, Sichuan Province, China
    • 2School of Nursing, The University of Texas at Austin, Austin
    • 3School of Nursing, The Johns Hopkins University, Baltimore, Maryland
    • 4Hinda and Arthur Marcus Institute for Aging Research, Hebrew SeniorLife, Boston, ‎Massachusetts
    • 5Department of Breast Surgery, West China Hospital, Sichuan University, Chengdu, Sichuan Province, China
    • 6Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, ‎Massachusetts
    JAMA Intern Med. Published online October 21, 2019. doi:https://doi.org/10.1001/jamainternmed.2019.4446
    Key Points

    Question  Does the Tailored, Family-Involved Hospital Elder Life Program reduce the incidence of postoperative delirium and functional decline in older patients after a noncardiac surgical procedure?

    Findings  In this single-blind randomized clinical trial of 281 older adults in China, postoperative delirium occurred in fewer participants randomized to receive the Tailored, Family-Involved Hospital Elder Life Program intervention compared with those randomized to the usual care control group. Decline in physical and cognitive functions during hospitalization was significantly less in patients in the intervention group, and the effect was sustained 30 days after discharge.

    Meaning  The Tailored, Family-Involved Hospital Elder Life Program may be beneficial for older patients who undergo noncardiac surgical treatment as it appears to help in reducing postoperative delirium, maintaining physical and cognitive functions, and shortening the length of hospital stay.

    Abstract

    Importance  Postoperative delirium (POD) is a common condition for older adults, contributing to their functional decline.

    Objective  To investigate the effectiveness of the Tailored, Family-Involved Hospital Elder Life Program (t-HELP) for preventing POD and functional decline in older patients after a noncardiac surgical procedure.

    Design, Setting, and Participants  A 2-arm, parallel-group, single-blind, cluster randomized clinical trial was conducted from August 24, 2015, to February 28, 2016, on 6 surgical floors (gastric, colorectal, pancreatic, biliary, thoracic, and thyroid) of West China Hospital in Chengdu, China. Eligible participants (n = 281) admitted to each of the 6 surgical floors were randomized into a nursing unit providing t-HELP (intervention group) or a nursing unit providing usual care (control group). All randomized patients were included in the intention-to-treat analyses for the primary outcome of POD incidence. Statistical analysis was performed from April 3, 2016, to December 30, 2017.

    Interventions  In addition to receiving usual care, all participants in the intervention group received the t-HELP protocols, which addressed each patient’s risk factor profile. Besides nursing professionals, family members and paid caregivers were involved in the delivery of many of the program interventions.

    Main Outcomes and Measures  The primary outcome was the incidence of POD, evaluated with the Confusion Assessment Method. Secondary outcomes included the pattern of functional and cognitive changes (activities of daily living [ADLs], instrumental activities of daily living [IADLs], Short Portable Mental Status Questionnaire [SPMSQ]) from hospital admission to 30 days after discharge, and the length of hospital stay (LOS).

    Results  Of the 475 patients screened for eligibility, 281 (171 [60.9%] male, mean [SD] age 74.7 [5.2] years) were enrolled and randomized to receive t-HELP (n = 152) or usual care (n = 129). Postoperative delirium occurred in 4 participants (2.6%) in the intervention group and in 25 (19.4%) in the control group, with a relative risk of 0.14 (95% CI, 0.05-0.38). The number needed to treat to prevent 1 case of POD was 5.9 (95% CI, 4.2-11.1). Participants in the intervention group compared with the control group showed less decline in physical function (median [interquartile range] for ADLs: −5 [−10 to 0] vs −20 [−30 to −10]; P < .001; for IADLs: −2 [−2 to 0] vs −4 [−4 to −2]; P < .001) and cognitive function (for the SPMSQ level: 1 [0.8%] vs 8 [7.0%]; P = .009) at discharge, as well as shorter mean (SD) LOS (12.15 [3.78] days vs 16.41 [4.69] days; P < .001).

    Conclusions and Relevance  The findings suggest that t-HELP, with family involvement at its core, is effective in reducing POD for older patients, maintaining or improving their physical and cognitive functions, and shortening the LOS. The results of this t-HELP trial may improve generalizability and increase the implementation of this program.

    Trial Registration  Chinese Clinical Trial Registry Identifier: ChiCTR-POR-15006944

    Introduction

    In the era of longer life expectancy and advances in medical science and technology, the number of older adults receiving operations has increased. In the United States, about 50% of people older than 65 years undergo surgical procedures.1 A similar trend is seen in mainland China, where 35% of operations are performed on older patients.2 Postoperative delirium (POD) is one of the most common complications for older patients, with incidence rates ranging from 13% to 50%.3 As such, POD is associated with serious adverse outcomes, including increased length of hospital stay (LOS),4 mortality,5 risk of functional decline, and subsequent dementia.6 In the United States, annual health care costs for POD are estimated to be more than $182 billion.7,8 However, previous studies have demonstrated that 30% to 40% of these cases are preventable.9,10 Thus, effective prevention strategies for delirium can yield substantial benefits.

    Currently, multicomponent nonpharmacological interventions are considered the first-line approach for delirium prevention.11 Among these approaches, the Hospital Elder Life Program (HELP)12 is the most widely implemented evidence-based model that targets multiple risk factors for delirium.13,14 In 2013, HELP protocols were adapted to align with the National Institute for Health and Care Excellence (NICE) guidelines.15 New interventions based on the NICE guidelines include hypoxia and catheter-associated urinary tract infection as well as refinement of 1 other protocol (expanding dehydration protocol to include constipation). For the present study, we therefore implemented a total of 11 protocols, comprising the original 6 volunteer-based HELP protocols, 2 original nursing-based HELP protocols, and 3 NICE guidelines–adapted protocols. Furthermore, we tailored the protocols to address each patient’s risk factor profile so that each patient received a customized menu of interventions.

    Involvement of volunteer teams is an important part of HELP in many hospitals, but most hospitals in China do not use volunteers. To implement the program at our hospital, we adapted HELP to the local culture. In China, family members are typically more involved in the care of older hospitalized patients compared with families in the United States or Europe. This cultural background provided us with a unique, alternative resource to conduct many of the nonpharmacological interventions in HELP.

    The innovation in this present study was called t-HELP (tailored, family-involved HELP), which involved family members instead of volunteers and applied a tailored approach to assigning HELP protocols. The overarching rationale for the trial was to adapt HELP to local circumstances in China and demonstrate its effectiveness in a different health care system. We believed this example of t-HELP implementation and this rigorous trial would increase the generalizability and adoption of HELP.

    Methods
    Study Design

    From August 24, 2015, to February 28, 2016, we conducted a 2-arm, parallel-group, single-blind, cluster randomized clinical trial (Chinese Clinical Trial Registry [http://www.chictr.org.cn/index.aspx] identifier: ChiCTR-POR-15006944) across 6 surgical floors—gastric, colorectal, pancreatic, biliary, thoracic, and thyroid—of West China Hospital in Chengdu, China. Each surgical floor was equipped with 80 to 100 beds and staffed with 34 to 38 nurses and 16 to 22 surgeons. The trial protocol (Supplement 1) was approved by the institutional review board of West China Hospital. Written informed consent was obtained from all participants or their legal representatives before the trial.

    Randomization and Masking

    The incidence of POD varied with the type of surgical procedure received. Participants were therefore divided into 6 blocks according to the different surgical floors for randomization. For each surgical floor, we used a 2-step randomization process. First, with standard computerized randomization algorithms, the nursing units were randomized as receiving t-HELP (intervention) or usual care (control) (Figure 1). Second, after admission to the hospital, eligible participants were randomized to t-HELP or usual care units by opening sealed envelopes containing the random assignments.

    Randomization was performed by a member of the hospital staff who was not involved in the intervention and was not on the research team. The random assignment was concealed from all individuals until the initiation of the intervention.

    Given the nature of the intervention, it was not feasible to blind participants and t-HELP personnel. However, outcome assessors and statistical analysts were blinded. The hospital wards used similar decorations and facilities in both groups. Blinding was strictly maintained and was monitored by an inspector (D.-M.X.). The trial maintained separation of the research assessors and data management staff from the clinical team and staff who delivered the intervention. All investigators, interventionists, and study participants were kept strictly blinded to outcome measurements and trial results. Patients and families were asked to avoid discussing their interventions with assessors. The study’s integrity was carefully monitored by the institutional review board of West China Hospital.

    Participants

    Older patients admitted to 24 nursing units from the 6 surgical floors were screened for enrollment by 6 trained nurses from August 24, 2015, through February 28, 2016. Patients aged 70 years or older and scheduled for an elective surgical procedure with an anticipated LOS longer than 2 days were eligible for inclusion. Exclusion criteria included (1) delirium at baseline as assessed with the Confusion Assessment Method (CAM); (2) a terminal condition with life expectancy of less than 6 months (eg, metastatic cancer, pancreatic cancer, or receiving end-of-life care); (3) inability to perform cognitive tests because of severe dementia, legal blindness, or severe deafness; (4) a documented history of schizophrenia or psychosis; and (5) a documented history of alcohol abuse or withdrawal within the past 6 months and/or reporting consumption of more than 5 drinks per day for men (4 for women).

    Interventions

    The t-HELP and usual care units were in separate hospital wards, independent of each other. The hospital staff (residents, nurses, and caregivers) did not change or cross over between the 2 sets of units during the study period, thus minimizing contamination between the intervention and control groups.

    After participants entered a t-HELP unit, they received the tailored HELP protocol daily from postoperative day 1 to postoperative day 7 or discharge (if the LOS was fewer than 7 days). The t-HELP group comprised patients whose delirium-related risk factors were assessed within 24 hours of enrollment or admission; the individualized (tailored) intervention was subsequently provided daily. The t-HELP intervention consisted of 3 universal protocols and 8 targeted protocols. The universal protocols, including orientation, therapeutic activities, and early mobilization protocol, were given to all t-HELP participants. The targeted protocols (eTable 1 in Supplement 2) were tailored for each patient on the basis of delirium-related risk factors, which were assessed daily. The workflow diagram (eFigure in Supplement 2) demonstrates how we assigned the targeted protocols. All protocols were tracked daily, with adherence scored as high (80% to 100% of days with complete adherence) or low (less than 80% of days with complete adherence).16 To ensure family adherence with the intervention, we used the following procedures. First, family education was provided to help the family members understand the importance of their work. Second, nurses supervised the family intervention and recorded the actual completion of each protocol. Third, the reasons for nonadherence were recorded, and any questions and concerns were then discussed and addressed.

    Control participants who entered a usual care unit received usual treatment and care. A flowchart of the assessment and interventions received by the 2 groups appears in the eFigure in Supplement 2.

    Assessments, Outcomes, and Sample Size

    At baseline, all patients underwent preoperative assessment the day before the surgical procedure, including sociodemographic data, medical history and comorbidities, and cognitive and physical function. Subsequently, patients were evaluated daily until postoperative day 7 or discharge through a structured interview consisting of delirium assessment and adverse events monitoring. A follow-up phone interview was conducted 1 month after discharge. Baseline screening and the evaluation process are summarized in eTable 2 in Supplement 2.

    The primary outcome was the incidence of POD within 7 days after the surgical procedure, defined according to the CAM criteria. The CAM has been used widely for rating delirium,17 with a sensitivity of 94% to 100%, a specificity of 90% to 95%, and high interobserver reliability.18 The Chinese-language version of the CAM has been used in Mandarin-speaking populations, demonstrating comparable sensitivity and specificity.19

    Secondary and additional outcomes were as follows: (1) incidence of severe delirium during hospitalization, as measured by the Chinese version of MDAS (Memorial Delirium Assessment Scale; score range: 0 to 30, with higher scores indicating more severe delirium, and severe delirium is defined as a score of ≥18)20; (2) change of cognitive function before and after the surgical procedure as measured by the Chinese version of the SPMSQ (Short Portable Mental Status Questionnaire; score counts the number of errors, and all scores are adjusted by educational level: 0 to 2 errors indicate normal mental functioning; 3 to 4, mild cognitive impairment; 5 to 7, moderate cognitive impairment; and ≥8, severe cognitive impairment)21; (3) change of physical function before and after the surgical procedure using the Barthel Index to measure activities of daily living (ADLs)22 and instrumental activities of daily living (IADLs)23; (4) LOS; (5) adherence to intervention; (6) adverse events (Supplement 1); and (7) follow-up assessments of SPMSQ level, ADLs, IADLs, and FRAIL score (Fatigue, Resistance, Ambulation, Illnesses, and Loss of Weight) at 30 days after discharge. The Chinese version of the FRAIL scale was translated from the original and included 5 questions for fatigue, resistance, aerobic capacity, illnesses, and loss of weight. Responses were scored from 0 to 5 points, with a score of 3 to 5 points defining patients as frail, 1 to 2 points as prefrail, and 0 points as healthy.24

    To minimize error and maximize reliability, the trial director (Y.-Y.W.) (1) provided intensive training to the assessors, including a review of the t-HELP procedure outlined in a short booklet and video to ensure high interrater reliability (κ≥0.9); (2) monitored the assessors’ administration of questionnaires to older adults who were not part of the study; and (3) met with the assessors every week to review procedures and check the quality of the assessment for the primary outcome. PASS 11 software (NCSS Statistical Software) was used to calculate the sample size.

    Previously published results suggested that the incidence of POD could be decreased 20% by implementing a targeted multicomponent delirium intervention.12,25 Assuming a type I error of 5%, a type II error of 20%, and taking clustering into account, we estimated the sample size to be 192 (96 participants per group). Allowing for 20% attrition, we increased the sample size to 230 patients from 24 units (115 participants per group) at baseline.

    Statistical Analysis

    We used SPSS, version 19.0 (IBM) for all statistical analyses, with 2-tailed tests as appropriate, and 2-sided P < .05 considered to be statistically significant. Participants’ baseline characteristics were compared using an unpaired t test or, if the variables were not normally distributed, the Mann-Whitney test. We used the χ2 or Fisher exact test for categorical or ranked variables. Statistical analysis was performed from April 3, 2016, to December 30, 2017.

    To account for the hierarchical structure of the data (ie, study participants nested within nursing units), we used a multilevel binomial regression model26 to estimate the effect of treatment on primary outcome (POD incidence). A generalized estimating equation was built to examine the association between intervention and secondary outcomes (ADLs, IADLs, FRAIL score, and SPMSQ level before the operation, at discharge, and 30 days after discharge). The cluster or type of the surgical procedure was treated as a random-effects factor in the models. Within the multilevel binomial regression analysis, 2 nested models were fitted. First, an unadjusted model was fit without confounding variables while treating the nursing unit or surgical floor as a random effect. Second, a fully adjusted model (cluster or surgical level) was fit by including age and sex to study the independent effect of treatment on delirium incidence.

    Intention-to-treat analyses (all randomized patients) and per-protocol analyses (excluding all participants who withdrew or were lost to follow-up) were used for the primary outcome (POD incidence). The number needed to treat was calculated when P < .05. For secondary outcomes, only per-protocol analysis was performed. Adherence to the intervention was quantified as the proportion of days with complete adherence. All adverse events were evaluated by the principal investigator (J.-R.Y.) and an expert panel to determine whether the adverse events were related to the intervention.

    We conducted a sensitivity analysis to verify the robustness of the findings. We conservatively recoded missing values owing to study withdrawals as delirium positive for the t-HELP group or delirium negative for the usual care group.

    Results

    Of the 475 patients assessed for eligibility, 281 (59.2%) from 24 nursing units were eligible and randomized. Participants were divided into 2 groups to receive either t-HELP (n = 152) on 12 nursing units or usual care (n = 129) on 12 nursing units. The numbers and patterns of protocols received by patients are shown in eTable 3 in Supplement 2. Demographic and clinical characteristics were similar between the 2 groups (Table 1). The mean (SD) age of all patients was 74.7 (5.2) years, and 171 (60.9%) were male. The withdrawal rate did not differ significantly between groups, with an overall withdrawal rate of 12.1% (34 of 281) (Figure 1).

    Primary Outcome: POD Within 7 Days After the Procedure

    According to the intention-to-treat analysis (n = 281), delirium occurred in 4 (2.6%) of 152 cases in the intervention group and in 25 (19.4%) of 129 cases in the control group. The incidence of POD within postoperative day 7 was statistically significantly lower in the intervention group, with a relative risk (RR) of 0.14 (95% CI, 0.05-0.38). The number needed to treat to prevent 1 case of POD was 5.9 (95% CI, 4.2-11.1). After adjustments for age, sex, and type of surgical procedure, the effect remained significant (RR, 0.07; 95% CI, 0.02-0.26) (Table 2).

    To test the robustness of these findings, a sensitivity analysis was conducted by coding withdrawals after the surgical procedure as delirium positive in the intervention arm and delirium negative in the control arm. In this conservative scenario, the incidence of delirium remained significantly different between groups (RR, 0.41, 95% CI, 0.21-0.78), and the number needed to treat increased to 9.1 (95% CI, 5.0-33.3).

    Secondary Outcomes

    The t-HELP intervention was even more effective than the usual care in preventing severe delirium as measured by a score of 18 or higher in the Chinese version of the MDAS (2 [1.5%] patients vs 11 [9.6%] patients; P = .008). In addition, those in the t-HELP group compared with the usual care control group experienced less decline in physical (median [interquartile range (IQR)] change for ADLs: −5 [−10 to 0] vs −20 [−30 to −10]; P < .001; for IADLs: −2 [−2 to 0] vs −4 [−4 to −2]; P < .001) and cognitive (for SPMSQ level: 1 [0.8%] vs 8 [7.0%]; P = .009) functions at discharge, as well as shorter mean (SD) LOS (12.15 [3.78] days vs 16.41 [4.69] days; P < .001) (Table 3).

    Trends for improvement in ADLs, IADLs, and FRAIL scores between baseline and 30 days after discharge were statistically significant between groups (median [IQR] change for ADLs: 0 [0 to 0] vs −5 [−10 to −5]; P < .001; for IADLs: 0 [0 to 0] vs −2 [−3 to −2]; P < .001; for FRAIL score: 0 [−1 to 0] vs 0 [0 to 0]; P = .01), with t-HELP participants showing greater improvement over usual care participants (Figure 2A to C; eTable 4 in Supplement 2). The proportion with intact cognition increased over time in the intervention group but decreased in the control group (proportion change, 9.7% vs −9.2%; P < .001). The intervention had an independent effect on cognition after controlling for all of the covariates (Figure 2D; eTable 4 in Supplement 2).

    Adherence and Adverse Events

    Adherence rates differed across the 3 universal protocols. Among the targeted protocols, pain management and sleep enhancement were requested most (eTable 5 in Supplement 2). Adverse events occurring during the study included massive hemorrhage (n = 3), nosocomial infection (n = 5), fall (n = 3), and dyspnea (n = 1). No deaths occurred. None of these adverse events was determined to be associated with the trial. Only 1 study-related minor adverse event was reported: an episode of minor dyspnea during aromatherapy for sleep. The incidence of adverse events was similar between the 2 groups (6 vs 6).

    Discussion

    This randomized clinical trial provided evidence that t-HELP, involving the family, was effective for prevention of POD in patients who underwent noncardiac surgical procedures, with an even greater reduction in the incidence of severe delirium. Furthermore, t-HELP was effective in attenuating the decline in cognitive and physical functions as well as in shortening LOS. Given that HELP involved mobility and enhanced activity protocols, improved physical function was hypothesized as a potential secondary benefit of the program, which was demonstrated in the results.14,27 In addition, the findings showed that LOS was approximately 4 days shorter in the t-HELP group than in the usual care group. This reduced LOS was likely the result of the decreased incidence of POD and improved physical functioning.

    Strengths of this study include involving family members to assist in delivering the t-HELP intervention. This novel approach may facilitate the adoption of HELP in other settings; tailoring of the menu of interventions received; customization of the protocols to maximize their effectiveness; addition of new protocols (eg, hypoxia, catheter-associated urinary tract infection); addition of anesthesiologists as interdisciplinary members involved in pain management; and rigorous evaluation, using a single-blind randomized clinical trial and sensitivity analyses, of the robustness of the findings to study dropouts.

    Compared with the United States and European countries, China has a distinct medical system with different cultural influences. First, the nurse-to-patient ratio in China is lower, with fewer nurses to serve patients. Second, the hospital LOS in China is much longer, primarily because few post–acute care facilities exist, such as skilled nursing facilities or rehabilitation centers for hospitalized patients. Third, Chinese surgeons do not tend to operate on the most frail and aged in the population; thus, the study sample was comparatively younger and less impaired at baseline. Fourth, older adults in China have close relationships with their families, who are often involved in their care. In addition, low-cost, family-paid caregivers play a unique role in hospitals in China. Given the lack of a volunteer system, we used family members and family-paid caregivers because these individuals were considered consonant with and enhanced the family role rather than requiring a major change in hospital culture (use of volunteers) that might not have been acceptable.

    Involvement through t-HELP may be important for family members and may serve as a resource in many settings. Family involvement can increase the effectiveness of the intervention by providing psychological and emotional support to older patients undergoing treatment in hospitals.28 In practice guidelines, the American College of Surgeons and the American Geriatrics Society encourage family members to be at bedside, participate in care, and assist with functional improvement.29 The family-paid caregivers in the present trial were a unique group of health care practitioners who functioned similarly to nursing aides in the United States. Hired by patients and their families, the caregivers spent 24 hours per day in the hospital.30 In the control group, family members and paid caregivers helped with usual care, which typically included assisting with basic ADLs and monitoring medications. In the intervention group, the paid caregivers conducted some of the t-HELP protocols. Because the patients and family members covered the costs of these caregivers, no costs accrued to the hospital or health care system.

    The estimated additional cost in the t-HELP group was about $105 per case (in 2019 US dollars). However, this cost was more than offset by reducing the occurrence of delirium and shortening the LOS. Although a formal cost-effectiveness analysis was not conducted in this study, previously published studies have demonstrated that the intervention yields a net savings.31

    The population in this trial was relatively younger and healthier than similar surgical populations in the United States. However, delirium risk factors remained prevalent in the perioperative period in this sample. Moreover, the demonstration of intervention effectiveness in this lower-risk population offers strong support for the potency of the approach used. Greater effects might have been demonstrated in a higher-risk population.

    Another important issue noted in previous studies32-34 is that ensuring adherence to assigned interventions is a key factor for success, although it is not always achievable in real-world settings.33 We made many efforts to monitor and ensure adherence to all of the intervention protocols by hospital staff, t-HELP staff, and family members.

    Limitations

    This study has some limitations. First, despite all efforts to maintain blinding, blinding might have been compromised in some situations owing to the nature of the research (ie, we could not double-blind because t-HELP patients and staff were aware of the interventions) and to potential subjective contributions to some of the outcome ratings (eg, delirium, function). These circumstances may have led to the overestimation of the effect of t-HELP. Second, the 7-day duration of the intervention was shorter than the mean LOS. This duration was chosen to balance the peak days of delirium onset in the population (POD 3-5) against the practical constraints of the study resources. Third, we had a higher enrollment of men than of women, but no significant effect of sex on the incidence of POD was observed. Fourth, because of feasibility constraints, we were unable to assess the long-term effect of t-HELP on cognitive or physical functioning months after onset of delirium, which remains an important area for future research.

    Conclusions

    This study shows that t-HELP, which involved the family, was effective in reducing POD for older patients, maintaining or improving their physical and cognitive functions at discharge and 30 days after discharge, and shortening the LOS. This example of t-HELP implementation may provide ideas to hospitals interested in instituting HELP on how they can launch the interventions using existing staff and resources. The t-HELP trial may improve the generalizability and increase the implementation of HELP.

    Back to top
    Article Information

    Accepted for Publication: August 11, 2019.

    Corresponding Author: Ji-Rong Yue, MD, Department of Geriatrics and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, No. 37 Guoxue Xiang, Chengdu, 610041, China (yuejirong11@hotmail.com).

    Published Online: October 21, 2019. doi:10.1001/jamainternmed.2019.4446

    Author Contributions: Dr Yue had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Wang, Yue, Carter, Gartaganis, Inouye.

    Acquisition, analysis, or interpretation of data: Wang, Yue, Xie, Carter, Li, Chen, Inouye.

    Drafting of the manuscript: Wang, Yue, Carter, Li, Gartaganis.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Li.

    Obtained funding: Yue.

    Administrative, technical, or material support: Yue, Gartaganis, Chen, Inouye.

    Supervision: Yue, Xie, Inouye.

    Preparations work: organize training for the surgical staff: Wang.

    Conflict of Interest Disclosures: Dr Inouye was the creator of the Hospital Elder Life Program (HELP) but receives no income or royalties from the program. The American Geriatrics Society holds the exclusive license to HELP. No other disclosures were reported.

    Funding/Support: This study was funded by grant 2018YFC1312300 from the National Key Research and Development Program of the Ministry of Science and Technology of China; grant H1403014 from the Milstein Medical Asian American Partnership Foundation; grant 81800092 from the National Natural Science Foundation of China; grant Z2018B03 from the National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University; grant 2018SZ0252 from the Sichuan Science and Technology Program; and grant 2019-109, 2017-111 from the Health Research of Cadres in Sichuan province. Dr Inouye was supported in part by grants P01AG031720, K07AG041835, R24AG054259, and R01AG044518 from the National Institutes of Health and by the Milton and Shirley F. Levy Family Chair.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: Hospital Elder Life Program (HELP) is the original source of the intervention protocols (Supplement 1). The American Geriatrics Society (AGS) holds the exclusive license to the HELP intellectual property. All information about the t-HELP model must be obtained from the AGS. The Cain Center for Nursing Research and the Center for Transdisciplinary Collaborative Research in Self-management Science at The University of Texas at Austin School of Nursing provided editorial support during manuscript development. We are grateful to all of the participants who agreed to take part in the trial and to all of the research and surgical staff members who supported the trial.

    Data Sharing Statement: See Supplement 3.

    References
    1.
    Beliveau  MM, Multach  M.  Perioperative care for the elderly patient.  Med Clin North Am. 2003;87(1):273-289. doi:10.1016/S0025-7125(02)00155-4PubMedGoogle ScholarCrossref
    2.
    Lu  WL, Kumar  DS, Zheng  SB.  Analysis of perioperative complications and mortality in hospitalized elderly surgical patients  [in Chinese].  Chin J Geriatr. 2013;32(12):1319-1321. doi:10.3760/cma.j.issn.0254-9026.2013.12.017Google Scholar
    3.
    Inouye  SK, Westendorp  RG, Saczynski  JS.  Delirium in elderly people.  Lancet. 2014;383(9920):911-922. doi:10.1016/S0140-6736(13)60688-1PubMedGoogle ScholarCrossref
    4.
    Pitkala  KH, Laurila  JV, Strandberg  TE, Tilvis  RS.  Prognostic significance of delirium in frail older people.  Dement Geriatr Cogn Disord. 2005;19(2-3):158-163. doi:10.1159/000082888PubMedGoogle ScholarCrossref
    5.
    Leslie  DL, Zhang  Y, Holford  TR, Bogardus  ST, Leo-Summers  LS, Inouye  SK.  Premature death associated with delirium at 1-year follow-up.  Arch Intern Med. 2005;165(14):1657-1662. doi:10.1001/archinte.165.14.1657PubMedGoogle ScholarCrossref
    6.
    Davis  DH, Muniz-Terrera  G, Keage  HA,  et al; Epidemiological Clinicopathological Studies in Europe (EClipSE) Collaborative Members.  Association of delirium with cognitive decline in late life: a neuropathologic study of 3 population-based cohort studies.  JAMA Psychiatry. 2017;74(3):244-251. doi:10.1001/jamapsychiatry.2016.3423PubMedGoogle ScholarCrossref
    7.
    Leslie  DL, Marcantonio  ER, Zhang  Y, Leo-Summers  L, Inouye  SK.  One-year health care costs associated with delirium in the elderly population.  Arch Intern Med. 2008;168(1):27-32. doi:10.1001/archinternmed.2007.4PubMedGoogle ScholarCrossref
    8.
    Gleason  LJ, Schmitt  EM, Kosar  CM,  et al.  Effect of delirium and other major complications on outcomes after elective surgery in older adults.  JAMA Surg. 2015;150(12):1134-1140. doi:10.1001/jamasurg.2015.2606PubMedGoogle ScholarCrossref
    9.
    Siddiqi  N, House  AO, Holmes  JD.  Occurrence and outcome of delirium in medical in-patients: a systematic literature review.  Age Ageing. 2006;35(4):350-364. doi:10.1093/ageing/afl005PubMedGoogle ScholarCrossref
    10.
    Fong  TG, Tulebaev  SR, Inouye  SK.  Delirium in elderly adults: diagnosis, prevention and treatment.  Nat Rev Neurol. 2009;5(4):210-220. doi:10.1038/nrneurol.2009.24PubMedGoogle ScholarCrossref
    11.
    Oh  ES, Fong  TG, Hshieh  TT, Inouye  SK.  Delirium in older persons: advances in diagnosis and treatment.  JAMA. 2017;318(12):1161-1174. doi:10.1001/jama.2017.12067PubMedGoogle ScholarCrossref
    12.
    Inouye  SK, Bogardus  ST  Jr, Charpentier  PA,  et al.  A multicomponent intervention to prevent delirium in hospitalized older patients.  N Engl J Med. 1999;340(9):669-676. doi:10.1056/NEJM199903043400901PubMedGoogle ScholarCrossref
    13.
    Inouye  SK.  Prevention of delirium in hospitalized older patients: risk factors and targeted intervention strategies.  Ann Med. 2000;32(4):257-263. doi:10.3109/07853890009011770PubMedGoogle ScholarCrossref
    14.
    Inouye  SK, Bogardus  ST  Jr, Baker  DI, Leo-Summers  L, Cooney  LM  Jr; Hospital Elder Life Program.  The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients.  J Am Geriatr Soc. 2000;48(12):1697-1706. doi:10.1111/j.1532-5415.2000.tb03885.xPubMedGoogle ScholarCrossref
    15.
    Yue  J, Tabloski  P, Dowal  SL, Puelle  MR, Nandan  R, Inouye  SK.  NICE to HELP: operationalizing National Institute for Health and Clinical Excellence guidelines to improve clinical practice.  J Am Geriatr Soc. 2014;62(4):754-761. doi:10.1111/jgs.12768PubMedGoogle ScholarCrossref
    16.
    McDonald  HP, Garg  AX, Haynes  RB.  Interventions to enhance patient adherence to medication prescriptions: scientific review  [published correction appears in JAMA. 2003;289(24):3242].  JAMA. 2002;288(22):2868-2879. doi:10.1001/jama.288.22.2868PubMedGoogle ScholarCrossref
    17.
    Inouye  SK, van Dyck  CH, Alessi  CA, Balkin  S, Siegal  AP, Horwitz  RI.  Clarifying confusion: the Confusion Assessment Method: a new method for detection of delirium.  Ann Intern Med. 1990;113(12):941-948. doi:10.7326/0003-4819-113-12-941PubMedGoogle ScholarCrossref
    18.
    Inouye  SK, Kosar  CM, Tommet  D,  et al.  The CAM-S: development and validation of a new scoring system for delirium severity in 2 cohorts.  Ann Intern Med. 2014;160(8):526-533. doi:10.7326/M13-1927PubMedGoogle ScholarCrossref
    19.
    Gao  LL, Feng  DM, Wang  RH, Jiang  YY, Zhang  M, Yue  JR.  Validity and reliability of the Chinese version of short form of confused assessment method for the detection of delirium in the elderly  [in Chinese].  Shiyong Laonian Yixue. 2019;33(2):133-136. doi:10.3969/j.issn.1003-9198.2019.02.007Google Scholar
    20.
    Wu  YJ, Shi  YZ, Wang  MJ, Cao  J, Zu  YB, Shen  Y.  Chinese version of the Memorial Delirium Assessment Scale (MDAS): clinical validity and reliability in old postoperative patients  [in Chinese].  Chin J Mental Health. 2014;28(12):937-941. doi:10.3969/j.issn.1000-6729.2014.12.011Google Scholar
    21.
    Guo  JW.  A preliminary study on simple determination of cognitive impairment in the elderly  [in Chinese].  Chin J Mental Health. 1998(5):268-269. http://www.cnki.com.cn/Article/CJFDTotal-ZXWS805.004.htmGoogle Scholar
    22.
    Formiga  F, Fort  I, Robles  MJ, Rodriguez  D, Regalado  P.  Lower Barthel Index scores predict less prescription of pharmacological therapy in elderly patients with Alzheimer disease.  Dement Geriatr Cogn Disord. 2010;29(3):198-203. doi:10.1159/000278348PubMedGoogle ScholarCrossref
    23.
    Park  B, Jun  JK, Park  J.  Cognitive impairment and depression in the early 60s: which is more problematic in terms of instrumental activities of daily living?  Geriatr Gerontol Int. 2014;14(1):62-70. doi:10.1111/ggi.12055PubMedGoogle ScholarCrossref
    24.
    Dong  L, Qiao  X, Tian  X,  et al.  Cross-cultural adaptation and validation of the FRAIL scale in Chinese community-dwelling older adults.  J Am Med Dir Assoc. 2018;19(1):12-17. doi:10.1016/j.jamda.2017.06.011PubMedGoogle ScholarCrossref
    25.
    Marcantonio  ER, Flacker  JM, Wright  RJ, Resnick  NM.  Reducing delirium after hip fracture: a randomized trial.  J Am Geriatr Soc. 2001;49(5):516-522. doi:10.1046/j.1532-5415.2001.49108.xPubMedGoogle ScholarCrossref
    26.
    Li  P, Redden  DT.  Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials.  BMC Med Res Methodol. 2015;15:38-49. doi:10.1186/s12874-015-0026-xPubMedGoogle ScholarCrossref
    27.
    Hshieh  TT, Yue  J, Oh  E,  et al.  Effectiveness of multicomponent nonpharmacological delirium interventions: a meta-analysis.  JAMA Intern Med. 2015;175(4):512-520. doi:10.1001/jamainternmed.2014.7779PubMedGoogle ScholarCrossref
    28.
    Stall  NM, Campbell  A, Reddy  M, Rochon  PA.  Words matter: the language of family caregiving  [published online May 23, 2019].  J Am Geriatr Soc. 2019. doi:10.1111/jgs.15988PubMedGoogle Scholar
    29.
    Mohanty  S, Rosenthal  RA, Russell  MM, Neuman  MD, Ko  CY, Esnaola  NF.  Optimal perioperative management of the geriatric patient: a best practices guideline from the American College of Surgeons NSQIP and the American Geriatrics Society.  J Am Coll Surg. 2016;222(5):930-947. doi:10.1016/j.jamcollsurg.2015.12.026PubMedGoogle ScholarCrossref
    30.
    Dong  B, Yue  J, Cao  L,  et al; Journal of the American Geriatrics Society.  Transformation of a geriatric department in China.  J Am Geriatr Soc. 2018;66(1):184-190. doi:10.1111/jgs.15217PubMedGoogle ScholarCrossref
    31.
    Hshieh  TT, Yang  T, Gartaganis  SL, Yue  J, Inouye  SK.  Hospital Elder Life Program: systematic review and meta-analysis of effectiveness.  Am J Geriatr Psychiatry. 2018;26(10):1015-1033. doi:10.1016/j.jagp.2018.06.007PubMedGoogle ScholarCrossref
    32.
    Chen  CC, Li  HC, Liang  JT,  et al.  Effect of a modified Hospital Elder Life Program on delirium and length of hospital stay in patients undergoing abdominal surgery: a cluster randomized clinical trial.  JAMA Surg. 2017;152(9):827-834. doi:10.1001/jamasurg.2017.1083PubMedGoogle ScholarCrossref
    33.
    Young  J, Cheater  F, Collinson  M,  et al.  Prevention of delirium (POD) for older people in hospital: study protocol for a randomised controlled feasibility trial.  Trials. 2015;16(1):340-351. doi:10.1186/s13063-015-0847-2PubMedGoogle ScholarCrossref
    34.
    Heim  N, van Stel  HF, Ettema  RG, van der Mast  RC, Inouye  SK, Schuurmans  MJ.  HELP! problems in executing a pragmatic, randomized, stepped wedge trial on the Hospital Elder Life Program to prevent delirium in older patients.  Trials. 2017;18(1):220-231. doi:10.1186/s13063-017-1933-4PubMedGoogle ScholarCrossref
    ×