[Skip to Navigation]
Sign In
Figure 1.  Study Selection Process
Study Selection Process

RCTs indicates randomized clinical trials.

Figure 2.  Meta-analysis of the Effect of a Mobile Telephone Text Message Intervention on Medication Adherence
Meta-analysis of the Effect of a Mobile Telephone Text Message Intervention on Medication Adherence

aThe text message intervention significantly improved adherence (odds ratio, 2.11; 95% CI, 1.52-2.93; P < .001). The effect remained significant after excluding 2 studies with extreme outcomes (Hardy et al30 and Kohnsari et al31) (odds ratio, 1.78; 95% CI, 1.35-2.35; P < .001).

Table 1.  Characteristics of the Randomized Clinical Trials Using Text Messaging to Promote Medication Adherence
Characteristics of the Randomized Clinical Trials Using Text Messaging to Promote Medication Adherence
Table 2.  Features of the Text Messaging Design Used in the Randomized Clinical Trials
Features of the Text Messaging Design Used in the Randomized Clinical Trials
Table 3.  Effect of Text Messaging on Medication Adherence in Subgroups Stratified By TM and Study Characteristics
Effect of Text Messaging on Medication Adherence in Subgroups Stratified By TM and Study Characteristics
1.
Osterberg  L, Blaschke  T.  Adherence to medication.  N Engl J Med. 2005;353(5):487-497.PubMedGoogle ScholarCrossref
2.
Ho  PM, Bryson  CL, Rumsfeld  JS.  Medication adherence: its importance in cardiovascular outcomes.  Circulation. 2009;119(23):3028-3035.PubMedGoogle ScholarCrossref
3.
World Health Organization. Adherence to Long-term Therapies: Evidence for Action. Geneva, Switzerland: World Health Organization; 2003.
4.
Bowry  AD, Shrank  WH, Lee  JL, Stedman  M, Choudhry  NK.  A systematic review of adherence to cardiovascular medications in resource-limited settings.  J Gen Intern Med. 2011;26(12):1479-1491.PubMedGoogle ScholarCrossref
5.
Ho  PM, Magid  DJ, Shetterly  SM,  et al.  Medication nonadherence is associated with a broad range of adverse outcomes in patients with coronary artery disease.  Am Heart J. 2008;155(4):772-779.PubMedGoogle ScholarCrossref
6.
Chisholm-Burns  MA, Spivey  CA. The “cost” of medication nonadherence: consequences we cannot afford to accept. J Am Pharm Assoc (2003).2012;52(6):823-826. PubMed
7.
Haynes  RB, McDonald  H, Garg  AX, Montague  P.  Interventions for helping patients to follow prescriptions for medications.  Cochrane Database Syst Rev. 2002;(2):CD000011.PubMedGoogle Scholar
8.
Haynes  RB, Ackloo  E, Sahota  N, McDonald  HP, Yao  X.  Interventions for enhancing medication adherence.  Cochrane Database Syst Rev. 2008;(2):CD000011.PubMedGoogle Scholar
9.
Nieuwlaat  R, Wilczynski  N, Navarro  T,  et al.  Interventions for enhancing medication adherence.  Cochrane Database Syst Rev. 2014;11:CD000011.PubMedGoogle Scholar
10.
World Health Organization. mHealth: new horizons for health through mobile technologies. Global Observatory for eHealth Series Vol 3. http://www.who.int/goe/publications/goe_mhealth_web.pdf. Published 2011. Accessed September 13, 2015.
11.
US Food and Drug Administration. Mobile medical applications: guidance for industry and Food and Drug Administration staff. http://www.fda.gov/downloads/MedicalDevices/.../UCM263366.pdf. Issued February 9, 2015. Accessed December 9, 2015.
12.
International Telecommunication Union. Measuring the Information Society Report. http://www.itu.int/en/ITU-D/Statistics/Documents/publications/mis2014/MIS2014_without_Annex_4.pdf. Published 2014. Accessed September 13, 2015.
13.
Vodafone Group Plc. The role of mobile phones in increasing accessibility and efficiency in health care. The Vodafone Policy Paper Series 4. http://www.vodafone.com/content/dam/vodafone/about/public_policy/policy_papers/public_policy_series_4.pdf. Published March 2006. Accessed September 13, 2015.
14.
Koivusilta  LK, Lintonen  TP, Rimpelä  AH.  Orientations in adolescent use of information and communication technology: a digital divide by sociodemographic background, educational career, and health.  Scand J Public Health. 2007;35(1):95-103.PubMedGoogle ScholarCrossref
15.
Goggin  G.  Cell Phone Culture: Mobile Technology in Everyday Life. New York, NY: Routledge; 2006.
16.
Kannisto  KA, Koivunen  MH, Välimäki  MA.  Use of mobile phone text message reminders in health care services: a narrative literature review.  J Med Internet Res. 2014;16(10):e222. doi:10.2196/jmir.3442.PubMedGoogle ScholarCrossref
17.
Park  LG, Howie-Esquivel  J, Dracup  K.  A quantitative systematic review of the efficacy of mobile phone interventions to improve medication adherence.  J Adv Nurs. 2014;70(9):1932-1953.PubMedGoogle ScholarCrossref
18.
Tao  D, Xie  L, Wang  T, Wang  T.  A meta-analysis of the use of electronic reminders for patient adherence to medication in chronic disease care.  J Telemed Telecare. 2015;21(1):3-13.PubMedGoogle ScholarCrossref
19.
Cochrane Collaboration. Higgins  JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed December 7, 2015.
20.
Higgins  JPT, Deeks  JJ, Altman  DG. Chapter 16.5.4: How to include multiple groups from one study. In: Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed January 20, 2015.
21.
Lv  Y, Zhao  H, Liang  Z,  et al.  A mobile phone short message service improves perceived control of asthma: a randomized controlled trial.  Telemed J E Health. 2012;18(6):420-426.PubMedGoogle ScholarCrossref
22.
Borenstein  MHL, Higgins  J, Rothstein  H.  Comprehensive Meta-Analysis. Version 2. Englewood, NJ: Biostat; 2005.
23.
Jones  AP, Riley  RD, Williamson  PR, Whitehead  A.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials.  Clin Trials. 2009;6(1):16-27.PubMedGoogle ScholarCrossref
24.
Cohen  J.  Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1998.
25.
Deeks  JJ, Higgins  JPT, Altman  DG. Chapter 9.5: Heterogeneity. In: Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed January 20, 2015.
26.
Duval  S, Tweedie  R.  Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis.  Biometrics. 2000;56(2):455-463.PubMedGoogle ScholarCrossref
27.
Burnier  M, Wuerzner  G, Struijker-Boudier  H, Urquhart  J.  Measuring, analyzing, and managing drug adherence in resistant hypertension.  Hypertension. 2013;62(2):218-225.PubMedGoogle ScholarCrossref
28.
Turner  BJ, Hecht  FM.  Improving on a coin toss to predict patient adherence to medications.  Ann Intern Med. 2001;134(10):1004-1006.PubMedGoogle ScholarCrossref
29.
da Costa  TM, Barbosa  BJ, Gomes e Costa  DA,  et al.  Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages.  Int J Med Inform. 2012;81(4):257-269.PubMedGoogle ScholarCrossref
30.
Hardy  H, Kumar  V, Doros  G,  et al.  Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy.  AIDS Patient Care STDS. 2011;25(3):153-161.PubMedGoogle Scholar
31.
Khonsari  S, Subramanian  P, Chinna  K, Latif  LA, Ling  LW, Gholami  O.  Effect of a reminder system using an automated short message service on medication adherence following acute coronary syndrome.  Eur J Cardiovasc Nurs. 2015;14(2):170-179.PubMedGoogle ScholarCrossref
32.
Lester  RT, Ritvo  P, Mills  EJ,  et al.  Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial.  Lancet. 2010;376(9755):1838-1845.PubMedGoogle ScholarCrossref
33.
Lua  PL, Neni  WS.  A randomised controlled trial of an SMS-based mobile epilepsy education system.  J Telemed Telecare. 2013;19(1):23-28.PubMedGoogle ScholarCrossref
34.
Maduka  O, Tobin-West  CI.  Adherence counseling and reminder text messages improve uptake of antiretroviral therapy in a tertiary hospital in Nigeria.  Niger J Clin Pract. 2013;16(3):302-308.PubMedGoogle ScholarCrossref
35.
Mbuagbaw  L, Thabane  L, Ongolo-Zogo  P,  et al.  The Cameroon Mobile Phone SMS (CAMPS) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy.  PLoS One. 2012;7(12):e46909. doi:10.1371/journal.pone.0046909.PubMedGoogle ScholarCrossref
36.
Park  LG, Howie-Esquivel  J, Chung  ML, Dracup  K.  A text messaging intervention to promote medication adherence for patients with coronary heart disease: a randomized controlled trial.  Patient Educ Couns. 2014;94(2):261-268.PubMedGoogle ScholarCrossref
37.
Pop-Eleches  C, Thirumurthy  H, Habyarimana  JP,  et al.  Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders.  AIDS. 2011;25(6):825-834.PubMedGoogle ScholarCrossref
38.
Quilici  J, Fugon  L, Beguin  S,  et al.  Effect of motivational mobile phone short message service on aspirin adherence after coronary stenting for acute coronary syndrome.  Int J Cardiol. 2013;168(1):568-569.PubMedGoogle ScholarCrossref
39.
Strandbygaard  U, Thomsen  SF, Backer  V.  A daily SMS reminder increases adherence to asthma treatment: a three-month follow-up study.  Respir Med. 2010;104(2):166-171.PubMedGoogle ScholarCrossref
40.
Vervloet  M, van Dijk  L, Santen-Reestman  J,  et al.  SMS reminders improve adherence to oral medication in type 2 diabetes patients who are real time electronically monitored.  Int J Med Inform. 2012;81(9):594-604.PubMedGoogle ScholarCrossref
41.
Wald  DS, Bestwick  JP, Raiman  L, Brendell  R, Wald  NJ.  Randomised trial of text messaging on adherence to cardiovascular preventive treatment (INTERACT trial).  PLoS One. 2014;9(12):e114268. doi:10.1371/journal.pone.0114268.PubMedGoogle ScholarCrossref
42.
Wang  K, Wang  C, Xi  L,  et al.  A randomized controlled trial to assess adherence to allergic rhinitis treatment following a daily short message service (SMS) via the mobile phone.  Int Arch Allergy Immunol. 2014;163(1):51-58.PubMedGoogle ScholarCrossref
43.
Márquez Contreras  E, de la Figuera von Wichmann  M, Gil Guillén  V,  et al.  Effectiveness of an intervention to provide information to patients with hypertension as short text messages and reminders sent to their mobile phone (HTA-Alert) [in Spanish].  Aten Primaria. 2004;34(8):399-405.PubMedGoogle ScholarCrossref
44.
Cole-Lewis  H, Kershaw  T.  Text messaging as a tool for behavior change in disease prevention and management.  Epidemiol Rev. 2010;32(1):56-69.PubMedGoogle ScholarCrossref
45.
Chow  CK, Redfern  J, Hillis  GS,  et al.  Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial.  JAMA. 2015;314(12):1255-1263.PubMedGoogle ScholarCrossref
46.
Mbuagbaw  L, van der Kop  ML, Lester  RT,  et al.  Mobile phone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta-analysis of randomised trials.  BMJ Open. 2013;3(12):e003950. doi:10.1136/bmjopen-2013-003950.PubMedGoogle ScholarCrossref
47.
Kripalani  S, Yao  X, Haynes  RB.  Interventions to enhance medication adherence in chronic medical conditions: a systematic review.  Arch Intern Med. 2007;167(6):540-550.PubMedGoogle ScholarCrossref
48.
Finitsis  DJ, Pellowski  JA, Johnson  BT.  Text message intervention designs to promote adherence to antiretroviral therapy (ART): a meta-analysis of randomized controlled trials.  PLoS One. 2014;9(2):e88166. doi:10.1371/journal.pone.0088166.PubMedGoogle ScholarCrossref
49.
Orwin  RG.  A fail-safe N for effect size in meta-analysis.  J Educ Stat. 1983;8(2):8157-8159.Google ScholarCrossref
50.
Nunes  V, Neilson  J, O’Flynn  N,  et al.  Clinical Guidelines and Evidence Review for Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence. London, England: National Collaborating Centre for Primary Care and Royal College of General Practitioners; 2009.
Original Investigation
March 2016

Mobile Telephone Text Messaging for Medication Adherence in Chronic Disease: A Meta-analysis

Author Affiliations
  • 1Department of Cardiology, Westmead Hospital, Sydney, Australia
  • 2The George Institute for Global Health, The University of Sydney, Sydney, Australia
  • 3The University of Sydney, Sydney, Australia
  • 4The George Institute for Global Health, University of Oxford, Oxford, England
  • 5Department of Epidemiology, Johns Hopkins Blumberg School of Public Health, Baltimore, Maryland
JAMA Intern Med. 2016;176(3):340-349. doi:10.1001/jamainternmed.2015.7667
Abstract

Importance  Adherence to long-term therapies in chronic disease is poor. Traditional interventions to improve adherence are complex and not widely effective. Mobile telephone text messaging may be a scalable means to support medication adherence.

Objectives  To conduct a meta-analysis of randomized clinical trials to assess the effect of mobile telephone text messaging on medication adherence in chronic disease.

Data Sources  MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, PsycINFO, and CINAHL (from database inception to January 15, 2015), as well as reference lists of the articles identified. The data were analyzed in March 2015.

Study Selection  Randomized clinical trials evaluating a mobile telephone text message intervention to promote medication adherence in adults with chronic disease.

Data Extraction  Two authors independently extracted information on study characteristics, text message characteristics, and outcome measures as per the predefined protocol.

Main Outcomes and Measures  Odds ratios and pooled data were calculated using random-effects models. Risk of bias and study quality were assessed as per Cochrane guidelines. Disagreement was resolved by consensus.

Results  Sixteen randomized clinical trials were included, with 5 of 16 using personalization, 8 of 16 using 2-way communication, and 8 of 16 using a daily text message frequency. The median intervention duration was 12 weeks, and self-report was the most commonly used method to assess medication adherence. In the pooled analysis of 2742 patients (median age, 39 years and 50.3% [1380 of 2742] female), text messaging significantly improved medication adherence (odds ratio, 2.11; 95% CI, 1.52-2.93; P < .001). The effect was not sensitive to study characteristics (intervention duration or type of disease) or text message characteristics (personalization, 2-way communication, or daily text message frequency). In a sensitivity analysis, our findings remained robust to change in inclusion criteria based on study quality (odds ratio, 1.67; 95% CI, 1.21-2.29; P = .002). There was moderate heterogeneity (I2 = 62%) across clinical trials. After adjustment for publication bias, the point estimate was reduced but remained positive for an intervention effect (odds ratio, 1.68; 95% CI, 1.18-2.39).

Conclusions and Relevance  Mobile phone text messaging approximately doubles the odds of medication adherence. This increase translates into adherence rates improving from 50% (assuming this baseline rate in patients with chronic disease) to 67.8%, or an absolute increase of 17.8%. While promising, these results should be interpreted with caution given the short duration of trials and reliance on self-reported medication adherence measures. Future studies need to determine the features of text message interventions that improve success, as well as appropriate patient populations, sustained effects, and influences on clinical outcomes.

Introduction

Adherence is defined as the extent to which a patient correctly follows a prescribed therapy. Adherence is the medically preferred term because it reflects active involvement of the patient and a therapeutic alliance between the patient and his or her physician.1 This term is in contrast to compliance, which reflects more unidirectional connotations.1,2 Adherence to long-term therapies in developed countries is typically reported to be approximately 50% at 1 year after initiation of therapy, with worse rates in lower socioeconomic groups and in developing countries.3,4 Poor adherence has been linked to successive hospitalizations, increased need for medical interventions, morbidity, and mortality.5 In addition, medication nonadherence results in increased health care cost, with estimates from North America of approximately $100 billion being spent annually and $2000 spent per patient per year in excess physician visits.6

Interventions that improve adherence may have far greater effect on the health of a population than any improvement in specific medical treatment.7 A review article by Haynes et al8 concluded that almost all interventions that were effective for long-term care were complex and not widely effective. There is widespread need for convenient and feasible innovations to help patients remain adherent to medications.9 In recent years, mobile health (mHealth) has emerged as a strategy to improve the implementation of evidence-based medicine and support public health by using mobile digital devices.10 The usefulness of this medium has been explored to improve treatment adherence. Electronic reminders can be delivered in various forms. The use of mobile apps (software applications) requires specialized devices (smartphones, tablet computers, or personal digital assistants).11 Other media for reminders are pagers or dedicated devices for audiovisual reminders. However, their availability within usual health care is low and presents a challenge for translation into routine clinical practice.

Mobile telephone text messaging may be a more feasible platform to deliver electronic reminders in practice. The technology is old and therefore can be delivered to any existing mobile telephone. Subscription to mobile telephones is ever increasing. According to one estimate, there were approximately 7 billion mobile subscribers by the end of 2014, roughly corresponding to the global population.12 This technology is increasingly used by people from all socioeconomic classes,13,14 age groups,13 and continents.15

In recent times, text messages have been widely used as a reminder and support in various health programs. While previous reviews have shown favorable effects of text messaging, only narrative reviews of text messages without meta-analysis16,17 and a meta-analysis18 that included a diverse range of electronic interventions (text messages, audiovisual reminders, pagers, and beepers) have been published to date. The aim of this review was to estimate the effect of text messaging on medication adherence in adults with chronic medical disorders. Secondary aims were to describe and examine the effect of characteristics of text message interventions, including frequency of messaging, interactivity, and customization, and to describe perceptions and acceptability to participants.

Methods

This review was written and detailed in accord with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement and the Cochrane Collaboration reporting items for systematic reviews and meta-analyses.19

Literature Search

A computerized literature search of MEDLINE, EMBASE, CINAHL, PsycINFO, Cochrane Central Register of Controlled Trials, and trial registries (clinicaltrials.gov and ANZCTR [http://www.anzctr.org.au/]) was conducted using Medical Subject Headings and keywords. The keywords included (1) intervention (text messaging, text messages, short message service, mobile phone, and cellular phone) and (2) medication use (adherence, nonadherence, compliance, noncompliance, refuse, refusal, treatment refusal, and patient compliance). No restriction on publication date was applied. The electronic database was last searched on January 15, 2015.

Study Eligibility

We included trials based on the following criteria: (1) the trial studied adult patients (≥18 years) with chronic disease, (2) the patients received a mobile telephone text message intervention designed to promote medication adherence, (3) the design was a randomized clinical trial (RCT) with at least 4 weeks’ follow-up, and (4) the trial reported quantitative measures of the effect of text messaging on medication adherence. We excluded studies based on the following criteria: (1) the primary intervention under consideration was not limited to text messages, (2) the focus was solely disease management or education and did not report medication adherence or reported only surrogate outcomes (eg, CD4 cell count or glycated hemoglobin level), and (3) the study involved psychiatric, military, or institutionalized patients. The latter criterion was to avoid the potential influence of psychosocial or institutional controls on adherence.

Data Extraction

Two of us (J.T. and R.K.) independently screened all identified titles and abstracts from the literature search using a predefined protocol. We reviewed reference lists of relevant articles for additional publications. Full texts of screened articles were reviewed for inclusion criteria and study quality. Disagreements were resolved through discussion or in consultation with a third independent reviewer (C.K.C.). If more than 1 publication of an original trial was identified, such articles were assessed together to maximize data collection. We extracted data on study characteristics, text message characteristics, and outcomes.

We used the Cochrane Handbook for Systematic Reviews of Interventions20 guidelines for trials with multiple intervention arms. When reported, we used the overall intervention effect. Otherwise, we combined trial arms (ie, text message vs non–text message arms). We did not follow the alternative strategy of selecting a single pair of interventions because that can result in loss of information and introduce bias if the arm with positive results is preferentially included for analysis. For example, for the trial by Lv et al,21 we combined control and traditional arms and treated this group as a non–text message intervention arm. For trials reporting multiple follow-ups, the final follow-up corresponding to the study duration was used.

Assessment of Study Quality

Two of us (J.T. and R.K.) independently assessed the risk of bias of included studies, with any disagreements resolved by discussion and a third opinion (C.K.C.) to reach consensus. We assessed the methodological risk of bias of included studies in accord with the Cochrane Handbook for Systematic Reviews of Interventions,19 which recommends reporting of the following individual elements for RCTs: random sequence generation, randomization sequence concealment, masking, completeness of outcome data, selective outcome reporting, and other sources of bias. Each item is judged as being at high, low, or unclear risk of bias. Studies were deemed to be at highest risk of bias if they were scored as being at high or unclear risk of bias for the sequence generation or randomization concealment domains.

Statistical Analysis

We used a software program (Comprehensive Meta-analysis, version 2.2.064; Biostat) for statistical analyses.22 We used the mean effect size approach to pool estimates, which has been used by others.18 The effect size was weighted as per the study sample size. We used individual patient data when available. If only aggregate data were available, we used estimates of treatment difference and their variance.23 We calculated the odds ratio (OR) for each primary study and used random-effects models to pool estimates. We also calculated Cohen d as a magnitude of the effect size. Cohen d values of 0.2, 0.5, and 0.8 are generally considered as small, medium, and large effect sizes, respectively.24 We used the I2 statistic to assess heterogeneity. An I2 statistic exceeding 50% with P < .05 was interpreted as representing substantial heterogeneity.25 We assessed publication bias using funnel plot symmetry and Egger regression intercept. If publication bias exists, the funnel plot is asymmetric, with Egger test P < .05. We used the trim-and-fill method by Duval and Tweedie26 to impute the missing studies. We performed subgroup analysis based on study and text message characteristics.

There are no clear guidelines or criterion standard recommendations on how to assess medication adherence or define adherence outcomes.27,28 Our primary analysis was to examine the effects of text message interventions on medication adherence. We defined adherent patients as those individuals determined to be adherent as defined by the individual included trials. For trials that reported multiple measures of adherence, we selected the most objective measure of adherence according to a predefined hierarchy (ie, electronic monitoring over pill count over self-recall, as well as a continuous scale over a dichotomized scale.18

Results
Study Characteristics

We assessed 44 full-text articles for eligibility and identified 16 RCTs21,29-43 (Figure 1, Table 1, and eTable 1 in the Supplement) involving 2742 patients that met our inclusion criteria. The median sample size was 97 (range, 21-538). The median age of participants was 39 years (age range, 31-64 years), and 50.3% (1380 of 2742) were female. The studies evaluated various chronic diseases, including human immunodeficiency virus (HIV) infection,29,30,32,34,35,37 cardiovascular disease (CVD),31,36,38,41,43 asthma,21,39 allergic rhinitis,42 diabetes mellitus,40 and epilepsy.33 The median intervention duration was 12 weeks (range, 4-48 weeks).

Self-recall was the most commonly used method to assess adherence,21,31-34,36,38,41,42 followed by medication event monitoring system29,30,36,37 and pill count.29,30,43 Adherence outcome data were reported in 12 studies21,29,31-35,37,38,41-43 as proportion of patients adherent and in 4 studies30,36,39,40 as proportion of medication doses taken as prescribed. The adherence cutoff was defined at 95% in 6 HIV trials and 1 CVD trial, at 90% in 1 HIV trial, at 80% in 1 CVD trial, and at 80% in one hypertension trial. The control arm in 15 studies was standard therapy. Only 1 study30 compared the text message intervention with a control arm (using additional 1-way pager or beeper).

Text Message Characteristics

There was considerable variation in the text message intervention characteristics (Table 2). Fifteen studies sent text messages at a fixed predetermined frequency. One study40 used real-time medication monitoring in which patients were sent a text message reminder only if the participant failed to open the medication dispenser. Five studies30,31,36,38,41 incorporated personalization into their messages. For example, the trial by Khonsari et al31 used the following personalization: [Mr or Ms] [patient name], please take [medicine quantity] tablet of [medication name] at [time]. Eight studies used a 2-way communication strategy, which was mandatory in 4 studies30,32,36,41 and encouraged in 4 studies.21,33-35 The message content was predominantly medication reminders31,33-43 but also included medical educational information21,33,35-37,43 or nonmedical general topics29,30,32 (eg, jokes, Bible verses, humor, etc). With respect to frequency; the most common pattern was a daily text message in 8 studies,21,30,31,36-39,42 followed by a weekly text message in 3 studies.32,35,37 Only 1 study41 used a variable frequency pattern, with daily send for 2 weeks, followed by alternate days for 2 weeks, and then weekly for the remainder of the study duration. Four studies31,36,40,41 matched message send times with the time of patients’ medication doses. The sending of messages was managed by automation or computer programs in 10 studies.29-33,36,38,40,41,43

Meta-analysis of the Intervention Efficacy

In the pooled analysis of 2742 patients, text message interventions significantly improved medication adherence (OR, 2.11; 95% CI, 1.52-2.93; P < .001) (Figure 2). The weighted mean effect size (Cohen d) was 0.41 (95% CI, 0.23-0.59). Text message interventions were similarly effective when analyses were restricted to text message studies31,32,34,35,37,40-42 reporting outcomes by intent to treat (OR, 2.25; 95% CI, 1.51-3.37; P < .001; Cohen d, 0.45) (eFigure 1 in the Supplement) or by per-protocol analysis (OR, 1.65; 95% CI, 1.07-2.53; P = .02; Cohen d, 0.28) (eFigure 2 in the Supplement). We did not find significant effects of text messages on adherence in subgroup analysis based on text message characteristics and study variables (Table 3). There was moderate heterogeneity (I2 = 62%) across clinical trials. Publication bias was detected by funnel plot asymmetry (eFigure 3 in the Supplement) and Egger regression coefficient (1-tailed P = .02). Using trim-and-fill imputation for missing studies, the point estimate was reduced but remained positive, with an OR of 1.68 (95% CI, 1.18-2.39; P < .05).

Risk of Bias and Sensitivity Analysis

We performed a sensitivity analysis based on the quality of studies. Fourteen studies described a randomization sequence generation technique that was at low risk of bias. Ten studies used a low-risk method for randomization concealment. Masking of study participants was not possible because of the nature of the intervention. Masking of outcome assessment was not clearly described in most studies. Almost half of the studies performed their primary analysis according to the principles of intent to treat. Overall, 10 studies were thought to be of high quality (eFigure 4 and eTable 2 in the Supplement). Text messaging improved adherence even when we included only high-quality studies29,32-37,40-42 (OR, 1.67; 95% CI, 1.21-2.29; P = .002).

Text Message Acceptability

Participant feedback on text message acceptability was reported in 11 studies (Table 2). Most reported moderate to high levels of satisfaction with the program. They acknowledged text message support as a valuable reminder and expressed desire for continuation of the program. One study21 that used twice-daily text message reminders had a small fraction (6%) of participants reporting messages as being intrusive and inconvenient. Participants in 2 studies39,42 reported that morning hours (7 am in the study by Wang et al42 and 10 am in the study by Strandbygaard et al39) were not suitable because they tend to disrupt routines.

Discussion

We identified 16 RCTs that investigated the effect of text messaging on medication adherence in patients with chronic disease. We found that text message interventions increased medication adherence, with an approximate doubling of the odds of patients’ achieving adherence to their medication regimens. This increase translates into adherence rates improving from 50% (assuming this percentage as the baseline rate in patients with chronic disease from the literature in developing countries3,7) to 67.8%, or an absolute increase of 17.8%. Given the simplicity of the intervention and potential scalability, this finding suggests that text message–based interventions could have substantial potential to improve medication adherence in patients with chronic disease. Our findings are consistent with previous observations that text messaging can be a useful tool for behavioral change in disease prevention44 and monitoring and management.45,46 Our analyses indicated the presence of publication bias. However, even after taking this bias into account, the effect estimate was still significantly positive. There was also substantial heterogeneity noted, which is likely because of several reasons, including clinical heterogeneity (true variation of effect arising from variation in the characteristics of the text message interventions and variation in the patient populations). Nevertheless, there is also likely methodological heterogeneity, most likely arising because of variability in the ways outcomes were defined and measured in each study.

Improving medication adherence is a challenge. Various interventions targeting medication adherence have been reported, including patient education and counseling, allied health support (pharmacist-based or nurse-led interventions), use of reminders (beepers, pagers, smartphone apps, and automated telephone calls), packaged medications, and frequent clinic visits.1 Successful strategies usually involve multimodal combinations. However, implementing such complex combination methods is resource intensive and may not be feasible in routine clinical practice.2 A systematic review by Kripalani et al47 reported adherence outcomes of interventions broadly categorized as informational, behavioral, and combined interventions. Most effective informational interventions showed small to medium effect sizes (Cohen d range, 0.35-0.68) only when intensive counseling was offered over multiple sessions. Even behavioral interventions showed mixed success. Other techniques, such as specialized packaging, direct observed therapy, and cognitive behavior therapy, did not significantly affect adherence or clinical outcomes. The review by Haynes et al8 concluded that interventions effective for long-term care are complex and require a combination of multiple approaches. Our meta-analysis showed a comparable effect size (Cohen d, 0.41; 95% CI, 0.23-0.59) of a mobile telephone text message intervention in enhancing adherence. The distinct advantages of text messaging over other interventions are simplicity and ease of administration, often in an automated fashion using a computerized program.

Another area of interest is the effect of text message characteristics. The characteristics of interventions in this meta-analysis varied substantially. While certain characteristics of text message–based programs such as increased message frequency and 2-way communication have been suggested to improve outcomes, we found no significant heterogeneity of effects across subgroups within this review. However, the results of a comparative analysis between these subgroups should be interpreted with caution and regarded as inconclusive because of sparse data available for analysis. However, some aspects are noteworthy. Interventions that delivered personalized messages showed a moderate effect size (Cohen d, 0.63; 95% CI, 0.29-0.96; P < .01), which suggests that sending a text message with one’s preferred name may increase acceptance and participant engagement. There was no significant difference between daily text messages (Cohen d, 0.48; 95% CI, 0.17-0.79) compared with less frequent messaging (Cohen d, 0.35; 95% CI, 0.13-0.57), which is contrary to concerns raised in other studies37,48 that daily reminders may lead to habituation and response fatigue. However, one possible explanation is that response fatigue may be a feature of longer-duration interventions, while the median length of studies in this review was 3 months.

Overall, the text message intervention had high acceptance rates. The patients in our meta-analysis were middle-aged (age range, 31-64 years), and 50.3% (1380 of 2742) were female. Most studies did not report the educational or socioeconomic status of the participants. Hence, there were insufficient data to determine whether these individual characteristics may be associated with variation in response.

While there has been mounting evidence largely in favor of texting interventions, many questions remain unanswered. There is a need for future high-quality studies to address more comprehensively what features of programs make them more effective. For example, is there greater benefit with more text message sophistication such as fixed-frequency or real-time medication monitoring, matched short message service with medication times, varied message content, personalization, and 2-way communication? Also, do text message–based interventions work better in some groups of patients compared with others? For example, do they vary based on participant characteristics such as nonadherence, cultural background, type of disease, educational level, and socioeconomic class? This information will help inform how best to formulate text message–based interventions for different patient cohorts.

One unique feature of text message interventions is the ability to offer confidential and unobtrusive support, which is an advantage of text messaging over other electronic reminders such as pagers or beepers. In the study by Hardy et al,30 one-third of the control subjects objected to the use of beepers, which provoked curiosity from people around them and violated confidentiality. This concern was not reported in association with the text message intervention arm in any study, perhaps because of widespread use and recognition of conventional text message alert tones, which are less likely to arouse curiosity among peers. There is still a potential for problems in the case of unattended mobile telephones without password lock or smartphones with a message preview function, but these concerns can be managed with appropriate participant training.

There are several limitations of our meta-analysis. First, this study is subject to publication bias, although the effect of publication bias is likely small based on the fail-safe N test by Orwin.49 This test provides an estimate of the number of missing studies. At a generally acceptable threshold of 0.1 for the effect size, this value was 34, which means that we would need to locate an additional 34 studies with a mean standard difference in means of 0 to bring the combined standard difference in means to below 0.1. Second, the adherence levels that defined patients as adherent varied among the studies. For example, these thresholds in HIV studies were 90% in the study by Pop-Eleches et al37 vs 95% in the study by Lester et al;32 and in CVD studies they were 80% in the study by Wald et al41 vs 95% in the study by Quilici et al38. If the threshold is lowered, it can overestimate the effect of text messaging on adherence and vice versa. Third, many studies used self-reporting to determine the outcomes, which carries the possibility of recall bias and social desirability bias. Self-recall is commonly used to measure adherence owing to the convenience, the lack of a criterion standard, and the challenges of recording behaviors objectively. However, it tends to overestimate adherence, and there is no consensus on optimal recall periods (3 days, 7 days, or 1 month).50 Social desirability bias may also be an important consideration in the studies included given their cultural diversity. Fourth, the RCTs identified in our meta-analysis had short intervention duration and follow-up (median, 12 weeks), and none of the studies reported data on adherence behavior beyond the end of the trial or completion of the intervention. The short duration of the trials suggests uncertainty about the duration of the effect, the time-effect relationship, and the continuation or decay of the effect after the intervention is withdrawn. While our meta-analysis identified a positive effect of text messaging on medication adherence in the short term, it is uncertain if this influence will translate into longer-term effects on adherence behavior or on clinical outcomes.

Conclusions

We found that mobile telephone text messaging increased adherence to taking medications among middle-aged patients with chronic disease. The ease of use, instantaneous relay of information, and boundless reach make it an attractive tool for public health. While our analyses indicate some heterogeneity across clinical trials, this finding is likely because of variation in the characteristics of the interventions studied and in the definitions of outcomes among the studies. These results should be interpreted with caution given that most trials were of short duration and that most used self-reported outcome measures. Hence, uncertainty remains about the effect size of text messages over longer periods and on objective measures of outcome. Future research on the benefit of different features of text message interventions, the longevity of the effect, and the influence on objective clinical measures of outcomes are needed to help better identify the role of text message interventions to improve medication adherence in chronic disease care.

Back to top
Article Information

Accepted for Publication: November 16, 2015.

Corresponding Author: Jay Thakkar, FRACP, The George Institute for Global Health, The University of Sydney, Missenden Road, PO Box M 201, Level 10, King George V Bldg, Camperdown, New South Wales, Australia 2050 (jthakkar@georgeinstitute.org.au).

Published Online: February 1, 2016. doi:10.1001/jamainternmed.2015.7667.

Author Contributions: Dr Thakkar had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Thakkar, Redfern, Chow.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Thakkar, Kurup, Thiagalingam, Redfern, Chow.

Critical revision of the manuscript for important intellectual content: Thakkar, Laba, Santo, Thiagalingam, Rogers, Woodward, Redfern, Chow.

Statistical analysis: Thakkar, Santo, Woodward.

Administrative, technical, or material support: Thakkar, Kurup, Thiagalingam, Redfern.

Study supervision: Thiagalingam, Rogers, Woodward, Redfern, Chow.

Conflict of Interest Disclosures: Dr Thakkar reported being a PhD student at The University of Sydney and reported having an Australian postgraduate award scholarship. Dr Rodgers reported having a National Health and Medical Research Council (NHMRC) principal research fellowship. Dr Woodward reported having an NHMRC principal research fellowship and reported receiving funding from Roche, Amgen, and Novartis. Dr Redfern reported having a career development and future leader fellowship cofunded by the NHMRC and by grant APP1061793 from the National Heart Foundation. Dr Chow reported having a career development and future leader fellowship cofunded by the NHMRC and by grant 1033478 from the National Heart Foundation and Sydney Medical Foundation Chapman Fellowship. No other disclosures were reported.

Additional Contributions: Jeremy Culis, BAppSci (faculty liaison librarian and assistant manager, Medical Sciences Library, The University of Sydney) designed the search strategy and facilitated the identification of studies included in the meta-analysis, for which he did not receive compensation outside of his usual salary.

References
1.
Osterberg  L, Blaschke  T.  Adherence to medication.  N Engl J Med. 2005;353(5):487-497.PubMedGoogle ScholarCrossref
2.
Ho  PM, Bryson  CL, Rumsfeld  JS.  Medication adherence: its importance in cardiovascular outcomes.  Circulation. 2009;119(23):3028-3035.PubMedGoogle ScholarCrossref
3.
World Health Organization. Adherence to Long-term Therapies: Evidence for Action. Geneva, Switzerland: World Health Organization; 2003.
4.
Bowry  AD, Shrank  WH, Lee  JL, Stedman  M, Choudhry  NK.  A systematic review of adherence to cardiovascular medications in resource-limited settings.  J Gen Intern Med. 2011;26(12):1479-1491.PubMedGoogle ScholarCrossref
5.
Ho  PM, Magid  DJ, Shetterly  SM,  et al.  Medication nonadherence is associated with a broad range of adverse outcomes in patients with coronary artery disease.  Am Heart J. 2008;155(4):772-779.PubMedGoogle ScholarCrossref
6.
Chisholm-Burns  MA, Spivey  CA. The “cost” of medication nonadherence: consequences we cannot afford to accept. J Am Pharm Assoc (2003).2012;52(6):823-826. PubMed
7.
Haynes  RB, McDonald  H, Garg  AX, Montague  P.  Interventions for helping patients to follow prescriptions for medications.  Cochrane Database Syst Rev. 2002;(2):CD000011.PubMedGoogle Scholar
8.
Haynes  RB, Ackloo  E, Sahota  N, McDonald  HP, Yao  X.  Interventions for enhancing medication adherence.  Cochrane Database Syst Rev. 2008;(2):CD000011.PubMedGoogle Scholar
9.
Nieuwlaat  R, Wilczynski  N, Navarro  T,  et al.  Interventions for enhancing medication adherence.  Cochrane Database Syst Rev. 2014;11:CD000011.PubMedGoogle Scholar
10.
World Health Organization. mHealth: new horizons for health through mobile technologies. Global Observatory for eHealth Series Vol 3. http://www.who.int/goe/publications/goe_mhealth_web.pdf. Published 2011. Accessed September 13, 2015.
11.
US Food and Drug Administration. Mobile medical applications: guidance for industry and Food and Drug Administration staff. http://www.fda.gov/downloads/MedicalDevices/.../UCM263366.pdf. Issued February 9, 2015. Accessed December 9, 2015.
12.
International Telecommunication Union. Measuring the Information Society Report. http://www.itu.int/en/ITU-D/Statistics/Documents/publications/mis2014/MIS2014_without_Annex_4.pdf. Published 2014. Accessed September 13, 2015.
13.
Vodafone Group Plc. The role of mobile phones in increasing accessibility and efficiency in health care. The Vodafone Policy Paper Series 4. http://www.vodafone.com/content/dam/vodafone/about/public_policy/policy_papers/public_policy_series_4.pdf. Published March 2006. Accessed September 13, 2015.
14.
Koivusilta  LK, Lintonen  TP, Rimpelä  AH.  Orientations in adolescent use of information and communication technology: a digital divide by sociodemographic background, educational career, and health.  Scand J Public Health. 2007;35(1):95-103.PubMedGoogle ScholarCrossref
15.
Goggin  G.  Cell Phone Culture: Mobile Technology in Everyday Life. New York, NY: Routledge; 2006.
16.
Kannisto  KA, Koivunen  MH, Välimäki  MA.  Use of mobile phone text message reminders in health care services: a narrative literature review.  J Med Internet Res. 2014;16(10):e222. doi:10.2196/jmir.3442.PubMedGoogle ScholarCrossref
17.
Park  LG, Howie-Esquivel  J, Dracup  K.  A quantitative systematic review of the efficacy of mobile phone interventions to improve medication adherence.  J Adv Nurs. 2014;70(9):1932-1953.PubMedGoogle ScholarCrossref
18.
Tao  D, Xie  L, Wang  T, Wang  T.  A meta-analysis of the use of electronic reminders for patient adherence to medication in chronic disease care.  J Telemed Telecare. 2015;21(1):3-13.PubMedGoogle ScholarCrossref
19.
Cochrane Collaboration. Higgins  JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed December 7, 2015.
20.
Higgins  JPT, Deeks  JJ, Altman  DG. Chapter 16.5.4: How to include multiple groups from one study. In: Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed January 20, 2015.
21.
Lv  Y, Zhao  H, Liang  Z,  et al.  A mobile phone short message service improves perceived control of asthma: a randomized controlled trial.  Telemed J E Health. 2012;18(6):420-426.PubMedGoogle ScholarCrossref
22.
Borenstein  MHL, Higgins  J, Rothstein  H.  Comprehensive Meta-Analysis. Version 2. Englewood, NJ: Biostat; 2005.
23.
Jones  AP, Riley  RD, Williamson  PR, Whitehead  A.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials.  Clin Trials. 2009;6(1):16-27.PubMedGoogle ScholarCrossref
24.
Cohen  J.  Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1998.
25.
Deeks  JJ, Higgins  JPT, Altman  DG. Chapter 9.5: Heterogeneity. In: Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://www.cochrane-handbook.org. Updated March 2011. Accessed January 20, 2015.
26.
Duval  S, Tweedie  R.  Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis.  Biometrics. 2000;56(2):455-463.PubMedGoogle ScholarCrossref
27.
Burnier  M, Wuerzner  G, Struijker-Boudier  H, Urquhart  J.  Measuring, analyzing, and managing drug adherence in resistant hypertension.  Hypertension. 2013;62(2):218-225.PubMedGoogle ScholarCrossref
28.
Turner  BJ, Hecht  FM.  Improving on a coin toss to predict patient adherence to medications.  Ann Intern Med. 2001;134(10):1004-1006.PubMedGoogle ScholarCrossref
29.
da Costa  TM, Barbosa  BJ, Gomes e Costa  DA,  et al.  Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages.  Int J Med Inform. 2012;81(4):257-269.PubMedGoogle ScholarCrossref
30.
Hardy  H, Kumar  V, Doros  G,  et al.  Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy.  AIDS Patient Care STDS. 2011;25(3):153-161.PubMedGoogle Scholar
31.
Khonsari  S, Subramanian  P, Chinna  K, Latif  LA, Ling  LW, Gholami  O.  Effect of a reminder system using an automated short message service on medication adherence following acute coronary syndrome.  Eur J Cardiovasc Nurs. 2015;14(2):170-179.PubMedGoogle ScholarCrossref
32.
Lester  RT, Ritvo  P, Mills  EJ,  et al.  Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial.  Lancet. 2010;376(9755):1838-1845.PubMedGoogle ScholarCrossref
33.
Lua  PL, Neni  WS.  A randomised controlled trial of an SMS-based mobile epilepsy education system.  J Telemed Telecare. 2013;19(1):23-28.PubMedGoogle ScholarCrossref
34.
Maduka  O, Tobin-West  CI.  Adherence counseling and reminder text messages improve uptake of antiretroviral therapy in a tertiary hospital in Nigeria.  Niger J Clin Pract. 2013;16(3):302-308.PubMedGoogle ScholarCrossref
35.
Mbuagbaw  L, Thabane  L, Ongolo-Zogo  P,  et al.  The Cameroon Mobile Phone SMS (CAMPS) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy.  PLoS One. 2012;7(12):e46909. doi:10.1371/journal.pone.0046909.PubMedGoogle ScholarCrossref
36.
Park  LG, Howie-Esquivel  J, Chung  ML, Dracup  K.  A text messaging intervention to promote medication adherence for patients with coronary heart disease: a randomized controlled trial.  Patient Educ Couns. 2014;94(2):261-268.PubMedGoogle ScholarCrossref
37.
Pop-Eleches  C, Thirumurthy  H, Habyarimana  JP,  et al.  Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders.  AIDS. 2011;25(6):825-834.PubMedGoogle ScholarCrossref
38.
Quilici  J, Fugon  L, Beguin  S,  et al.  Effect of motivational mobile phone short message service on aspirin adherence after coronary stenting for acute coronary syndrome.  Int J Cardiol. 2013;168(1):568-569.PubMedGoogle ScholarCrossref
39.
Strandbygaard  U, Thomsen  SF, Backer  V.  A daily SMS reminder increases adherence to asthma treatment: a three-month follow-up study.  Respir Med. 2010;104(2):166-171.PubMedGoogle ScholarCrossref
40.
Vervloet  M, van Dijk  L, Santen-Reestman  J,  et al.  SMS reminders improve adherence to oral medication in type 2 diabetes patients who are real time electronically monitored.  Int J Med Inform. 2012;81(9):594-604.PubMedGoogle ScholarCrossref
41.
Wald  DS, Bestwick  JP, Raiman  L, Brendell  R, Wald  NJ.  Randomised trial of text messaging on adherence to cardiovascular preventive treatment (INTERACT trial).  PLoS One. 2014;9(12):e114268. doi:10.1371/journal.pone.0114268.PubMedGoogle ScholarCrossref
42.
Wang  K, Wang  C, Xi  L,  et al.  A randomized controlled trial to assess adherence to allergic rhinitis treatment following a daily short message service (SMS) via the mobile phone.  Int Arch Allergy Immunol. 2014;163(1):51-58.PubMedGoogle ScholarCrossref
43.
Márquez Contreras  E, de la Figuera von Wichmann  M, Gil Guillén  V,  et al.  Effectiveness of an intervention to provide information to patients with hypertension as short text messages and reminders sent to their mobile phone (HTA-Alert) [in Spanish].  Aten Primaria. 2004;34(8):399-405.PubMedGoogle ScholarCrossref
44.
Cole-Lewis  H, Kershaw  T.  Text messaging as a tool for behavior change in disease prevention and management.  Epidemiol Rev. 2010;32(1):56-69.PubMedGoogle ScholarCrossref
45.
Chow  CK, Redfern  J, Hillis  GS,  et al.  Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial.  JAMA. 2015;314(12):1255-1263.PubMedGoogle ScholarCrossref
46.
Mbuagbaw  L, van der Kop  ML, Lester  RT,  et al.  Mobile phone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta-analysis of randomised trials.  BMJ Open. 2013;3(12):e003950. doi:10.1136/bmjopen-2013-003950.PubMedGoogle ScholarCrossref
47.
Kripalani  S, Yao  X, Haynes  RB.  Interventions to enhance medication adherence in chronic medical conditions: a systematic review.  Arch Intern Med. 2007;167(6):540-550.PubMedGoogle ScholarCrossref
48.
Finitsis  DJ, Pellowski  JA, Johnson  BT.  Text message intervention designs to promote adherence to antiretroviral therapy (ART): a meta-analysis of randomized controlled trials.  PLoS One. 2014;9(2):e88166. doi:10.1371/journal.pone.0088166.PubMedGoogle ScholarCrossref
49.
Orwin  RG.  A fail-safe N for effect size in meta-analysis.  J Educ Stat. 1983;8(2):8157-8159.Google ScholarCrossref
50.
Nunes  V, Neilson  J, O’Flynn  N,  et al.  Clinical Guidelines and Evidence Review for Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence. London, England: National Collaborating Centre for Primary Care and Royal College of General Practitioners; 2009.
×