[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.163.129.96. Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Download PDF
Figure 1.
Article Flowchart
Article Flowchart

IPA indicates International Pharmaceutical Abstracts; TEP, technical expert panel.

Figure 2.
Effect of Medication Therapy Management (MTM) on Mean Number of Outpatient Visits
Effect of Medication Therapy Management (MTM) on Mean Number of Outpatient Visits

Three low or medium risk-of-bias randomized clinical trials of MTM versus control groups without MTM using the basic arm of the Touchette et al25 study. Our meta-analysis found no significant difference in the number of outpatient visits between patients receiving MTM interventions and those receiving usual care (standardized mean difference, 0.05; 95% CI, −0.03 to 0.13; P = .25). IMPROVE indicates Impact of Managed Pharmaceutical care on Resource utilization and Outcomes in Veterans Affairs Medical Center.

Figure 3.
Effect of Medication Therapy Management (MTM) on Mean Number of Hospitalizations
Effect of Medication Therapy Management (MTM) on Mean Number of Hospitalizations

Three low or medium risk-of-bias randomized clinical trials of MTM versus control groups without MTM, using the basic arm of the Touchette et al25 study. Our meta-analysis found no significant difference in the number of hospitalizations between patients receiving MTM and those receiving usual care (WMD, 0.04; 95% CI, −0.01 to 0.08; P = .09). IMPROVE indicates Impact of Managed Pharmaceutical care on Resource utilization and Outcomes in Veterans Affairs Medical Center; WMD, weighted mean difference.

Table 1.  
Summary of Findings and Strength of Evidence for MTM Interventions on Drug-Related Problems and Biomarkers of Morbidity
Summary of Findings and Strength of Evidence for MTM Interventions on Drug-Related Problems and Biomarkers of Morbidity
Table 2.  
Summary of Findings and Strength of Evidence for Patient-Centered Outcomes of MTM Interventions
Summary of Findings and Strength of Evidence for Patient-Centered Outcomes of MTM Interventions
Table 3.  
Summary of Findings and Strength of Evidence for Health Care Use Outcomes of MTM Interventions
Summary of Findings and Strength of Evidence for Health Care Use Outcomes of MTM Interventions
1.
Adams  K, Corrigan  E. Priority Areas for Quality Improvement. Washington, DC: Institute of Medicine of the National Academies, National Academies Press; 2003.
2.
Kohn  LT, Corrigan  J, Donaldson  MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000.
3.
McGlynn  EA, Asch  SM, Adams  J,  et al.  The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635-2645.
PubMedArticle
4.
Sarkar  U, López  A, Maselli  JH, Gonzales  R.  Adverse drug events in US adult ambulatory medical care. Health Serv Res. 2011;46(5):1517-1533.
PubMedArticle
5.
Warholak  TL, Nau  DP. Quality and Safety in Pharmacy Practice. New York, NY: McGraw-Hill Medical; 2010.
6.
McGivney  MS, Meyer  SM, Duncan-Hewitt  W, Hall  DL, Goode  JV, Smith  RB.  Medication therapy management: its relationship to patient counseling, disease management, and pharmaceutical care. J Am Pharm Assoc (2003).2007;47(5):620-628.
PubMedArticle
7.
Pellegrino  AN, Martin  MT, Tilton  JJ, Touchette  DR.  Medication therapy management services: definitions and outcomes. Drugs. 2009;69(4):393-406.
PubMedArticle
8.
Medicare Prescription Drug, Improvement, and Modernization Act of 2003. 1st ed. 108th Cong (2004).
9.
Bluml  BM.  Definition of medication therapy management: development of professionwide consensus. J Am Pharm Assoc (2003). 2005;45(5):566-572.Article
10.
American Pharmacists Association; National Association of Chain Drug Stores Foundation.  Medication therapy management in pharmacy practice: core elements of an MTM service model (version 2.0). J Am Pharm Assoc (2003). 2008;48(3):341-353.Article
11.
Isetts  BJ, Buffington  DE; Pharmacist Services Technical Advisory Coalition.  CPT code-change proposal: national data on pharmacists’ medication therapy management services. Consult Pharm. 2007;22(8):684-689.
PubMedArticle
12.
American Medical Association. CPT Changes 2006: An Insider's View. Chicago, IL: American Medical Association; 2005:309-312.
13.
Pharmacist Services Technical Advisory Coalition. Medication therapy management service codes.http://www.pstac.org/services/mtms-codes.html. Accessed March 5, 2013.
14.
Viswanathan  M, Kahwati  LC, Golin  CE,  et al. Medication therapy management interventions in outpatient settings; comparative effectiveness review No. 138. Publication No. 14(15)-EHC037-EF. Prepared by the RTI International–University of North Carolina at Chapel Hill Evidence-Based Practice Center (contract No. 290-2012-00008-I). Rockville, MD: Agency for Healthcare Research and Quality; November 17, 2014. http://www.effectivehealthcare.ahrq.gov/reports/final.cfm. Accessed November 17, 2014.
15.
Agency for Healthcare Research and Quality. Evidence-Based Practice Center Systematic Review Protocol Project Title: Medication Therapy Management.http://effectivehealthcare.ahrq.gov/ehc/products/516/1601/medication-therapy-management-protocol-130724.pdf. Accessed October 2, 2014.
16.
Viswanathan  M, Ansari  MT, Berkman  ND,  et al. Methods Guide for Comparative Effectiveness Reviews: Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Rockville, MD: Agency for Healthcare Research and Quality; March 2012. Publication No. 12-EHC047-EF.
17.
Higgins  JP, Altman  DG, Gøtzsche  PC,  et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
PubMedArticle
18.
Viswanathan  M, Berkman  ND.  Development of the RTI item bank on risk of bias and precision of observational studies. J Clin Epidemiol. 2012;65(2):163-178.
PubMedArticle
19.
West  SL, Gartlehner  G, Mansfield  AJ,  et al. Comparative Effectiveness Review Methods: Clinical Heterogeneity: Methods Research Paper. Rockville, MD: Agency for Healthcare Research and Quality; September 2010. Publication No. 10-EHC070-EF.
20.
Higgins  JP, Thompson  SG, Deeks  JJ, Altman  DG.  Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560.
PubMedArticle
21.
Berkman  ND, Lohr  KN, Ansari  M,  et al. Methods Guide for Comparative Effectiveness Reviews: Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Rockville, MD: Agency for Healthcare Research and Quality; November 2013. Publication No. 13(14)-EHC130-EF (prepared by the RTI-UNC Evidence-Based Practice Center under contract No. 290-2007-10056-I).
22.
Guyatt  GH, Oxman  AD, Vist  G,  et al.  GRADE guidelines, IV: rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407-415.
PubMedArticle
23.
Atkins  D, Best  D, Briss  PA,  et al; GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.
PubMedArticle
24.
Atkins  D, Chang  SM, Gartlehner  G,  et al.  Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011;64(11):1198-1207.
PubMedArticle
25.
Touchette  DR, Masica  AL, Dolor  RJ,  et al.  Safety-focused medication therapy management: a randomized controlled trial. J Am Pharm Assoc (2003). 2012;52(5):603-612.
PubMedArticle
26.
Malone  DC, Carter  BL, Billups  SJ,  et al.  Can clinical pharmacists affect SF-36 scores in veterans at high risk for medication-related problems? Med Care. 2001;39(2):113-122.
PubMedArticle
27.
Sellors  J, Kaczorowski  J, Sellors  C,  et al.  A randomized controlled trial of a pharmacist consultation program for family physicians and their elderly patients. CMAJ. 2003;169(1):17-22.
PubMed
28.
Malone  DC, Carter  BL, Billups  SJ,  et al.  An economic analysis of a randomized, controlled, multicenter study of clinical pharmacist interventions for high-risk veterans: the IMPROVE study. Pharmacotherapy. 2000;20(10):1149-1158.
PubMedArticle
29.
Ellis  SL, Billups  SJ, Malone  DC,  et al.  Types of interventions made by clinical pharmacists in the IMPROVE study. Pharmacotherapy. 2000;20(4):429-435.
PubMedArticle
30.
Ellis  SL, Carter  BL, Malone  DC,  et al.  Clinical and economic impact of ambulatory care clinical pharmacists in management of dyslipidemia in older adults: the IMPROVE study. Pharmacotherapy. 2000;20(12):1508-1516.
PubMedArticle
31.
Taylor  CT, Byrd  DC, Krueger  K.  Improving primary care in rural Alabama with a pharmacy initiative. Am J Health Syst Pharm. 2003;60(11):1123-1129.
PubMed
32.
Hanlon  JT, Weinberger  M, Samsa  GP,  et al.  A randomized, controlled trial of a clinical pharmacist intervention to improve inappropriate prescribing in elderly outpatients with polypharmacy. Am J Med. 1996;100(4):428-437.
PubMedArticle
33.
Cowper  PA, Weinberger  M, Hanlon  JT,  et al.  The cost-effectiveness of a clinical pharmacist intervention among elderly outpatients. Pharmacotherapy. 1998;18(2):327-332.
PubMed
34.
Park  JJ, Kelly  P, Carter  BL, Burgess  PP.  Comprehensive pharmaceutical care in the chain setting. J Am Pharm Assoc (Wash). 1996;NS36(7):443-451.
PubMed
35.
Pai  AB, Boyd  A, Depczynski  J, Chavez  IM, Khan  N, Manley  H.  Reduced drug use and hospitalization rates in patients undergoing hemodialysis who received pharmaceutical care: a 2-year, randomized, controlled study. Pharmacotherapy. 2009;29(12):1433-1440..
PubMedArticle
36.
Pai  AB, Boyd  A, Chavez  A, Manley  HJ.  Health-related quality of life is maintained in hemodialysis patients receiving pharmaceutical care: a 2-year randomized, controlled study. Hemodial Int. 2009;13(1):72-79..
PubMedArticle
37.
Welch  EK, Delate  T, Chester  EA, Stubbings  T.  Assessment of the impact of medication therapy management delivered to home-based Medicare beneficiaries. Ann Pharmacother. 2009;43(4):603-610.
PubMedArticle
38.
Moczygemba  LR, Barner  JC, Gabrillo  ER.  Outcomes of a Medicare Part D telephone medication therapy management program. J Am Pharm Assoc (2003). 2012;52(6):e144-e152. doi:10.1331/JAPhA.2012.11258.
PubMedArticle
39.
Moczygemba  LR, Barner  JC, Lawson  KA,  et al.  Impact of telephone medication therapy management on medication and health-related problems, medication adherence, and Medicare Part D drug costs: a 6-month follow up. Am J Geriatr Pharmacother. 2011;9(5):328-338.
PubMedArticle
40.
Moczygemba  LR, Barner  JC, Gabrillo  ER, Godley  PJ.  Development and implementation of a telephone medication therapy management program for Medicare beneficiaries. Am J Health Syst Pharm. 2008;65(17):1655-1660.
PubMedArticle
41.
Perlroth  D, Marrufo  G, Montesinos  A,  et al. Medication Therapy Management in Chronically Ill Populations: Final Report. Baltimore, MD: Center for Medicare & Medicaid Innovation, Centers for Medicare & Medicaid Services; 2013. Contract HHSM-500-2011-00012I/TOT0001.
42.
Brummel  AR, Soliman  AM, Carlson  AM, de Oliveira  DR.  Optimal diabetes care outcomes following face-to-face medication therapy management services. Popul Health Manag. 2013;16(1):28-34.
PubMedArticle
43.
Moore  JM, Shartle  D, Faudskar  L, Matlin  OS, Brennan  TA.  Impact of a patient-centered pharmacy program and intervention in a high-risk group. J Manag Care Pharm. 2013;19(3):228-236.
PubMed
44.
Volume  CI, Farris  KB, Kassam  R, Cox  CE, Cave  A.  Pharmaceutical care research and education project: patient outcomes. J Am Pharm Assoc (Wash). 2001;41(3):411-420.
PubMed
45.
Sidel  VW, Beizer  JL, Lisi-Fazio  D,  et al.  Controlled study of the impact of educational home visits by pharmacists to high-risk older patients. J Community Health. 1990;15(3):163-174.
PubMedArticle
46.
McDonough  RP, Doucette  WR, Kumbera  P, Klepser  DG.  An evaluation of managing and educating patients on the risk of glucocorticoid-induced osteoporosis. Value Health. 2005;8(1):24-31.
PubMedArticle
47.
Gattis  WA, Hasselblad  V, Whellan  DJ, O’Connor  CM.  Reduction in heart failure events by the addition of a clinical pharmacist to the heart failure management team: results of the Pharmacist in Heart Failure Assessment Recommendation and Monitoring (PHARM) Study. Arch Intern Med. 1999;159(16):1939-1945.
PubMedArticle
48.
Jameson  J, VanNoord  G, Vanderwoud  K.  The impact of a pharmacotherapy consultation on the cost and outcome of medical therapy. J Fam Pract. 1995;41(5):469-472.
PubMed
49.
Clifford  RM, Batty  KT, Davis  TME,  et al.  A randomised controlled trial of a pharmaceutical care programme in high-risk diabetic patients in an outpatient clinic. Int J Pharm Pract. 2002;10(2):85-89.Article
50.
Soliman  AM, Carlson  AM, MacLehose  RF, Brummel  AR, Schommer  JC.  Patient characteristics predicting the frequency of medication therapy management visits for patients with diabetes. Clin Ther. 2013;35(4):534-540.
PubMedArticle
51.
Ramalho de Oliveira  D, Brummel  AR, Miller  DB.  Medication therapy management: 10 years of experience in a large integrated health care system. J Manag Care Pharm. 2010;16(3):185-195.
PubMed
52.
Jeong  E, Hui  R, Gallagher  M,  et al. Clinical, Economical and Humanistic Outcomes From a Pharmacist-Managed Medication Therapy Management Program (MTMP) in Medicare Patients From an Integrated Healthcare Delivery System. Oakland, CA: Kaiser Permanante; 2009:1-31.
53.
Hanlon  JT, Schmader  KE.  The medication appropriateness index at 20: where it started, where it has been, and where it may be going. Drugs Aging. 2013;30(11):893-900.
PubMedArticle
54.
Williams  ME, Pulliam  CC, Hunter  R,  et al.  The short-term effect of interdisciplinary medication review on function and cost in ambulatory elderly people. J Am Geriatr Soc. 2004;52(1):93-98.
PubMedArticle
55.
Marques  LA, Galduróz  JC, Fernandes  MR, Oliveira  CC, Beijo  LA, Noto  AR.  Assessment of the effectiveness of pharmacotherapy follow-up in patients treated for depression. J Manag Care Pharm. 2013;19(3):218-227.
PubMed
56.
Yamada  B. Comparison of Outcomes in the Medicare Medication Therapy Management Program Within Kaiser Permanente California. Oakland, CA: Kaiser Permanente; 2012.
57.
Pindolia  VK, Stebelsky  L, Romain  TM, Luoma  L, Nowak  SN, Gillanders  F.  Mitigation of medication mishaps via medication therapy management. Ann Pharmacother. 2009;43(4):611-620.
PubMedArticle
58.
Kassam  R, Farris  KB, Burback  L, Volume  CI, Cox  CE, Cave  A.  Pharmaceutical Care Research and Education Project: pharmacists’ interventions. J Am Pharm Assoc (Wash). 2001;41(3):401-410.
PubMed
59.
Sellors  C, Dalby  DM, Howard  M, Kaczorowski  J, Sellors  J.  A pharmacist consultation service in community-based family practices: a randomized, controlled trial in seniors. J Pharm Technol. 2001;17:264-269.
60.
Roughead  EE, Barratt  JD, Ramsay  E,  et al.  The effectiveness of collaborative medicine reviews in delaying time to next hospitalization for patients with heart failure in the practice setting: results of a cohort study. Circ Heart Fail. 2009;2(5):424-428.
PubMedArticle
61.
Shimp  LA, Kucukarslan  SN, Elder  J,  et al.  Employer-based patient-centered medication therapy management program: evidence and recommendations for future programs. J Am Pharm Assoc (2003). 2012;52(6):768-776.
PubMedArticle
62.
Christensen  DB, Roth  M, Trygstad  T, Byrd  J.  Evaluation of a pilot medication therapy management project within the North Carolina state health plan. J Am Pharm Assoc (2003). 2007;47(4):471-483.
PubMedArticle
63.
Wittayanukorn  S, Westrick  SC, Hansen  RA,  et al.  Evaluation of medication therapy management services for patients with cardiovascular disease in a self-insured employer health plan. J Manag Care Pharm. 2013;19(5):385-395.
PubMed
64.
Krska  J, Cromarty  JA, Arris  F,  et al.  Pharmacist-led medication review in patients over 65: a randomized, controlled trial in primary care. Age Ageing. 2001;30(3):205-211.
PubMedArticle
65.
Fischer  LR, Defor  TA, Cooper  S,  et al.  Pharmaceutical care and health care utilization in an HMO. Eff Clin Pract. 2002;5(2):49-57.
PubMed
66.
Carter  BL, Barnette  DJ, Chrischilles  E, Mazzotti  GJ, Asali  ZJ.  Evaluation of hypertensive patients after care provided by community pharmacists in a rural setting. Pharmacotherapy. 1997;17(6):1274-1285.
PubMed
67.
Barnette  DJ, Murphy  CM, Carter  BL.  Clinical skill development for community pharmacists. J Am Pharm Assoc (Wash). 1996;NS36(9):573-580.
PubMed
68.
Grymonpre  RE, Williamson  DA, Montgomery  PR.  Impact of a pharmaceutical care model for non-institutionalised elderly: results of a randomised, controlled trial. Int J Pharm Pract. 2001;9(4):235-241.Article
69.
Chrischilles  EA, Carter  BL, Lund  BC,  et al.  Evaluation of the Iowa Medicaid pharmaceutical case management program. J Am Pharm Assoc (2003). 2004;44(3):337-349.
PubMedArticle
70.
Winston  S, Lin  YS.  Impact on drug cost and use of Medicare Part D of medication therapy management services delivered in 2007. J Am Pharm Assoc (2003). 2009;49(6):813-820.
PubMedArticle
71.
Witry  MJ, Doucette  WR, Gainer  KL.  Evaluation of the pharmaceutical case management program implemented in a private sector health plan. J Am Pharm Assoc (2003). 2011;51(5):631-635.
PubMedArticle
72.
de Bruin  M, Viechtbauer  W, Schaalma  HP, Kok  G, Abraham  C, Hospers  HJ.  Standard care impact on effects of highly active antiretroviral therapy adherence interventions: a meta-analysis of randomized controlled trials. Arch Intern Med. 2010;170(3):240-250.
PubMedArticle
73.
Boutron  I, Moher  D, Altman  DG, Schulz  KF, Ravaud  P; CONSORT Group.  Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295-309.
PubMedArticle
74.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; Iniciativa STROBE.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies [in Spanish]. Rev Esp Salud Publica. 2008;82(3):251-259.
PubMedArticle
75.
Des Jarlais  DC, Lyles  C, Crepaz  N; TREND Group.  Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004;94(3):361-366.
PubMedArticle
76.
Ogrinc  G, Mooney  SE, Estrada  C,  et al.  The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17(suppl 1):i13-i32.
PubMedArticle
77.
Albrecht  L, Archibald  M, Arseneau  D, Scott  SD.  Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implement Sci. 2013;8:52.
PubMedArticle
78.
Bradley  EH, Curry  LA, Ramanadhan  S, Rowe  L, Nembhard  IM, Krumholz  HM.  Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009;4:25.
PubMedArticle
79.
Brown  CA, Lilford  RJ.  The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.
PubMedArticle
Original Investigation
January 2015

Medication Therapy Management Interventions in Outpatient SettingsA Systematic Review and Meta-analysis

Author Affiliations
  • 1RTI International, Research Triangle Park, North Carolina
  • 2Cecil G. Sheps Center for Health Services Research, University of North Carolina, Chapel Hill
  • 3Eshelman School of Pharmacy, University of North Carolina, Chapel Hill
JAMA Intern Med. 2015;175(1):76-87. doi:10.1001/jamainternmed.2014.5841
Abstract

Importance  Medication therapy management (MTM) services (also called clinical pharmacy services) aim to reduce medication-related problems and their downstream outcomes.

Objective  To assess the effect of MTM interventions among outpatients with chronic illnesses.

Data Sources  MEDLINE, Cochrane Library, and International Pharmaceutical Abstracts through January 9, 2014.

Study Selection  Two reviewers selected studies with comparators and eligible outcomes of ambulatory adults.

Data Extraction and Synthesis  Dual review of titles, abstracts, full-text, extractions, risk of bias, and strength of evidence grading. We conducted meta-analyses using random-effects models.

Main Outcomes and Measures  Medication-related problems, morbidity, mortality, quality of life, health care use, costs, and harms.

Results  Forty-four studies met the inclusion criteria. The evidence was insufficient to determine the effect of MTM interventions on most evaluated outcomes (eg, drug therapy problems, adverse drug events, disease-specific morbidity, disease-specific or all-cause mortality, and harms). The interventions improved a few measures of medication-related problems and health care use and costs (low strength of evidence) when compared with usual care. Specifically, MTM interventions improved medication appropriateness (4.9 vs 0.9 points on the medication appropriateness index, P < .001), adherence (approximately 4.6%), and percentage of patients achieving a threshold adherence level (odds ratios [ORs] ranged from 0.99 to 5.98) and reduced medication dosing (mean difference, −2.2 doses; 95% CI, −3.738 to −0.662). Medication therapy management interventions reduced health plan expenditures on medication costs, although the studies reported wide CIs. For patients with diabetes mellitus or heart failure, MTM interventions lowered the odds of hospitalization (diabetes: OR, 0.91 to 0.93 based on type of insurance; adjusted hazard rate for heart failure: 0.55; 95% CI, 0.39 to 0.77) and hospitalization costs (mean differences ranged from −$363.45 to −$398.98). The interventions conferred no benefit for patient satisfaction and most measures of health-related quality of life (low strength).

Conclusions and Relevance  We graded the evidence as insufficient for most outcomes because of inconsistency and imprecision that stem in part from underlying heterogeneity in populations and interventions. Medication therapy management interventions may reduce the frequency of some medication-related problems, including nonadherence, and lower some health care use and costs, but the evidence is insufficient with respect to improvement in health outcomes.

Introduction

Used appropriately, medications can alleviate distressing symptoms, prevent many acute and chronic illnesses, and improve patients’ health. However, medications often are used inappropriately.13 In the United States, adverse drug events led to an estimated 4.5 million ambulatory care visits per year during 2005 to 2007.4 In addition to problems involving adverse drug events, many patients experience drug therapy problems related to appropriateness (eg, unwarranted polypharmacy, suboptimal regimens), effectiveness (eg, subtherapeutic doses, therapeutic response not achieved at an adequate dose), safety (eg, adverse drug effects, drug-drug interactions, and supratherapeutic doses), and adherence.5

Medication therapy management (MTM) is a strategy for delivering a variety of nondispensing clinical pharmacy services to patients and their clinicians; it is a structure for providing what pharmacists referred to in the early 1990s as pharmaceutical care.6 Medication therapy management services aim to optimize therapeutic outcomes by identifying and resolving drug therapy problems.7 The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (Public Law 108-173)8 expanded access to MTM services for patients with certain chronic conditions through Medicare Part D prescription drug benefits. In 2005, 11 national pharmacy organizations established a consensus definition for MTM9; in 2008, a subset of these organizations established 5 core elements for an MTM service model.10 These elements include a medication therapy review, a personal medication record, a medication action plan, intervention and/or referral for drug therapy problems, and documentation and follow-up.7 Also by 2008, Current Procedural Terminology for MTM services (codes 99605, 99606, and 99607) became available and further defined MTM service-level expectations to include an assessment of drug-related needs, identification of drug therapy problems, and care planning and follow-up.1113

Medication therapy management can be viewed as a professionally delivered service with common core features and goals; specific intervention components may vary based on the scope and setting of each MTM program. Recent widespread implementation of MTM services, stemming from perceived clinical need and access to billing codes, drives an urgent need for actionable, evidence-based information on its effectiveness and harms. Our study assessed the effectiveness and harms of outpatient-based MTM compared with usual care; we focus on the effect of MTM on drug-therapy problems and their sequelae, including biomarkers of morbidity, health outcomes, mortality, patient-centered functioning, quality of life, satisfaction, and health care use and costs. This article is based on a systematic evidence report commissioned by the US Agency for Healthcare Research and Quality (AHRQ) to determine the effectiveness of outpatient MTM.14

Methods

We developed a protocol for this review.15 A trained librarian, in consultation with investigators, used numerous terms to identify MTM-related studies in MEDLINE, the Cochrane Library, and the International Pharmaceutical Abstracts database (inception through January 9, 2014) (eTable 1 in the Supplement). We limited searches to English-language and human-only studies. We manually searched the reference lists of landmark and background articles for additional relevant citations. We also searched numerous sources for unpublished literature described in eTable 1 in the Supplement. Finally, the AHRQ placed a request for unpublished studies in the Federal Register.

Study Selection

We prespecified required intervention characteristics to reduce underlying heterogeneity. We required MTM interventions to include a comprehensive (rather than condition-specific) medication review, patient-directed education, care coordination, and opportunity for follow-up. When articles included insufficient detail for us to decide on inclusion or exclusion, we contacted the authors for additional information. We limited our review to ambulatory settings. We anticipated modest randomized clinical trial (RCT) evidence and therefore also included nonrandomized clinical trials (NRCTs), cohort studies, and case-control studies. We excluded studies of MTM services provided within inpatient settings or shortly after hospital discharge; the goals of therapy and severity of illness are likely to differ markedly between outpatients and patients experiencing or recovering from an acute hospital stay.

Two trained members of the research team (M.V., L.C.K., C.E.G., S.J.B, E.C.-S., and R.P. performed all functions discussed) independently reviewed each of the titles, abstracts, and articles against the inclusion/exclusion criteria (eTable 2 in the Supplement). Studies marked for possible inclusion by either reviewer underwent dual, independent full-text review.

Data Extraction and Risk of Bias Assessment

A trained reviewer abstracted information from eligible studies into structured evidence tables; a second team member reviewed all data abstractions for completeness and accuracy. Reviewers rated the risk of bias independently against prespecified criteria.16 For RCTs, we relied on the risk-of-bias tool developed by the Cochrane Collaboration.17 We assessed the risk of bias of NRCTs using an item bank developed by RTI International.18

We evaluated RCTs on the adequacy of randomization, allocation concealment, similarity of groups at baseline, blinding of outcome assessment, attrition, use of intention-to-treat analysis, method of handling dropouts and missing data, validity and reliability of outcome measures, and treatment fidelity. For NRCTs, we assessed for confounding rather than adequacy of randomization and allocation concealment. We also evaluated RCTs for confounding attributable to randomization failure through biased selection or attrition. We rated the studies as low, medium, high, or unclear risk of bias.16 Two independent reviewers assessed the risk of bias for each study and resolved disagreements by consensus or by consulting a third member of the team.

Data Synthesis and Analysis

We excluded studies that we deemed at high risk of bias from our main data analyses, but we included them in sensitivity analyses. When we had no other evidence, we included high risk-of-bias studies in the main analyses.

We conducted meta-analysis when appropriate (≥3 studies with similar populations, interventions, comparators, outcomes, and design) using Comprehensive Meta-Analysis software (Biostat Inc). For all pooled analyses, we used random-effects models. To determine whether quantitative analyses were appropriate, we assessed the clinical and methodologic heterogeneity of the studies under consideration.19 For all quantitative syntheses, we assessed statistical heterogeneity in effects between studies by calculating the χ2 statistic and the I2 statistic (the percentage of total variation across studies attributable to heterogeneity rather than chance).20 When relevant, we examined potential sources of heterogeneity from the risk of bias using sensitivity analysis.

When quantitative analyses were inappropriate, we synthesized the data qualitatively, considering issues of consistency, precision, directness, and risk of bias. Whenever possible, we computed 95% CIs for outcomes reported in single studies, but numerous articles did not provide sufficient information for such computations.

Two reviewers assessed strength of evidence domains for each outcome and resolved differences by consensus or referral to a third senior member of the team. We based our grades on low or medium risk-of-bias RCTs or observational studies (unless none was available) based on guidance established by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group.2123 We considered one of the following grades for each outcome: (1) high (high confidence that the evidence reflects the true effect), (2) moderate (moderate confidence that the evidence reflects the true effect; future research may change the estimate), (3) low (low confidence that the evidence reflects the true effect; further research is likely to change the estimate), or (4) insufficient evidence.24

Results

We identified 61 articles (representing 44 studies) that met the inclusion criteria (Figure 1). The evidence base consisted of 21 RCTs, 4 NRCTs, and 19 cohort studies. Most studies compared an MTM intervention with usual care (no MTM intervention).

Methodologic problems in numerous studies led us to rate them as having a medium or high risk of bias. Of the 44 studies, we rated 16 as high risk of bias overall; in these studies, concerns about randomization failure, confounding, or overall attrition increased the risk of bias for all outcomes. In a few instances, we rated studies that were otherwise of low or medium risk of bias as high risk for individual outcomes chiefly because of measurement or detection biases related to the specific outcome. Because of the underlying heterogeneity of the populations, settings, and outcomes and the limited number of studies on any single outcome, we could use only 4 low or medium risk-of-bias studies in the meta-analysis2530 (Figure 2 and Figure 3 and eFigures 1-8 in the Supplement)2533 and 2 additional high risk-of-bias studies for sensitivity analysis (eTable 5).3436

Characteristics of included studies are provided in eTable 3 and eTable 4 in the Supplement. The numbers of participants analyzed in each study ranged from 10 to 200 722. All studies used pharmacists to deliver MTM services, specifically medication review, patient-directed education, care coordination, and opportunity for follow-up. Despite these 4 common features, MTM interventions differed considerably across the studies. Of the 44 included studies, 34 were broadly focused (ie, included a patient population not defined by any specific clinical condition), and 10 were relatively narrowly focused on specific patient populations (eg, chronic heart failure, hypertension, diabetes mellitus, human immunodeficiency virus, glucocorticoid-induced osteoporosis, or hemodialysis) who then received comprehensive medication review. Services were provided face to face in half of the included studies. Included studies provided interventions in a variety of settings including community pharmacies, centralized pharmacies or pharmacy call centers, outpatient medical clinics, and patients’ homes.

Drug-Related Problems and Biomarkers of Morbidity

Thirty-two studies reported on 1 or more intermediate outcomes (Table 1). We were unable to pool the results because too few studies within each design stratum (ie, RCT or cohort) were of low or medium risk of bias.

Medication therapy management interventions significantly improved medication appropriateness (measured with a reliable and valid 10-item index of medication appropriateness measuring improved prescribing quality53; 4.9 vs 0.9 points, P < .001).32 For some medications, MTM improved adherence as the percentage of prescribed doses taken (mean improvement, approximately 4.6%)41 or percentage of patients achieving a threshold adherence level (odds ratio [ORs], 0.99 to 5.98)41 and reduced medication dosing (mean difference, −2.2 doses; 95% CI, −3.738 to −0.662)48 (Table 1). We had enough consistency and precision across the body of evidence to assign a low strength-of-evidence grade for these outcomes. Evidence was insufficient to draw any conclusions about effectiveness for the number of drug therapy problems identified and resolved.

Evidence was insufficient to draw any conclusions about effectiveness for other intermediate outcomes, including the effect on anticoagulation, blood pressure, hemoglobin A1c, and low-density lipoprotein cholesterol. We identified no studies that addressed the effect of MTM on meeting the goals of therapy.

Morbidity, Mortality, and Other Patient-Centered Outcomes

Twenty-one studies reported 1 or more patient-centered outcomes; we were able to synthesize some outcomes quantitatively. Specifically, MTM interventions did not improve measures of health-related quality of life26,31,32 (Table 2 and eTable 5 and eFigures 1-8 in the Supplement), yielding low strength of evidence for no benefit. For general health-related quality of life outcomes, meta-analyses of 7 of 8 domains and the 2 component scores (physical health and mental health) showed no significant benefit from MTM interventions. However, the vitality domain of the SF-36 showed benefit in meta-analysis estimates that did not adjust for multiple comparisons. We graded this outcome as imprecise after correcting for multiple comparisons and the evidence on vitality as insufficient (eFigure 5 in the Supplement). The various patient satisfaction outcomes used by studies26,2830,32,44,58 also showed no effect from MTM programs (low strength of evidence for no benefit), although we were unable to pool results because of the heterogeneity of measures. Evidence was insufficient to draw conclusions about the effect of MTM on adverse drug events, gastrointestinal bleeding, mortality, and cognitive and affective function. No studies evaluated the effect of MTM on activities of daily living, work or school absenteeism, or patient and caregiver participation in medical care and decision making.

Health Care Use Outcomes

Thirty-three studies reported 1 or more outcomes related to the use of health care, such as the number of emergency department visits, number of hospitalizations, costs of laboratory tests, and total expenditures on medications. In addition to qualitative syntheses on all outcomes, we were able to pool the results on the number of outpatient and hospital visits.

Medication therapy management improved health plan expenditures on medications, but studies had wide CIs.27,48,59 The MTM interventions also reduced the risk of hospitalization (2 studies): the adjusted OR for diabetes ranged from 0.91 (95% CI, 0.87 to 0.95) to 0.93 (95% CI, 0.88 to 0.98),41 and the adjusted hazard rate for heart failure was 0.55 (95% CI, 0.39 to 0.77).60 Medication therapy management also reduced the costs of diabetes-specific hospitalization (1 study41) (mean difference, −$363.45 [95% CI, −$562.00 to −$164.91]) (Table 3). These results are imprecise or come from cohort studies with inherent limitations, leading to a judgment of low strength of evidence.

Three RCTs did not reduce the number of hospitalizations,2530 although a single cohort study43 found a lower mean number of inpatient visits for patients accepting MTM (compared with patients refusing MTM). Overall, we judged the strength of evidence for mean number of hospitalizations to be insufficient because of the lack of consistency across studies.

Evidence was insufficient on the effect of MTM interventions on the use of generic medications; measures of medication costs other than health plan expenditures; number (Figure 2) and costs of outpatient visits; number and costs of laboratory tests; number and costs of emergency department visits; and the number (Figure 3), percentage, and costs of hospitalization for unspecified conditions, congestive heart failure, or chronic obstructive pulmonary disease. Frequently, the results were inconsistent for the same outcome; that is, MTM may have raised the use of health care services in one study and lowered it in another. We could not interpret these results as either benefits or harms because studies uniformly evaluated overall change in health care use rather than whether the increase or decrease in use met the goal of MTM for individual patients.

Harms of MTM Interventions

Only 1 cohort study (high risk of bias) reported harms, specifically on inconvenience resulting from monthly appointments.66,67 The evidence was insufficient to judge the effect of MTM on inconvenience. We found no evidence on other prespecified harms including care fragmentation, patient decisional conflict, patient anxiety, increased adverse drug events, prescriber confusion, and prescriber dissatisfaction.

Effectiveness of MTM by Intervention Features

We found information from only 1 study for each of 5 intervention features: (1) access of pharmacists to patient records,25 (2) intensity of care coordination and follow-up after comprehensive medication review,68 (3) community pharmacy vs call center,69 (4) level of intensity of intervention,70 and (5) type of payer (private vs Medicaid) (eTable 6 in the Supplement).71 Evidence was insufficient for most outcomes for the first 2 intervention features, with 2 exceptions. First, MTM delivered by community pharmacists when compared with call center pharmacists increased the proportion of generics dispensed (low strength of evidence). Second, enhanced MTM with pharmacists’ access to patient records reduced the mean number of adverse drug events; this finding suggested benefit when compared with basic MTM (low strength of evidence). Evidence was insufficient for all outcomes for the intensity of intervention and type of payer.

Discussion

Evidence was insufficient to draw conclusions about the effect of MTM on most of the outcomes that we evaluated. We assigned grades of low strength of evidence for either a benefit (some measures of drug-therapy problems and health care use) or a lack of benefit (patient satisfaction and health-related quality of life) to few evaluation outcomes. The degree to which identifying and resolving drug therapy problems translate into consistent, detectable improvements in biomarkers of morbidity, health, patient experience, use, and costs cannot be determined from this body of evidence.

Although we tried to capture a range of contexts and settings and fill gaps by including observational studies, we did not find credible, consistent evidence that MTM improved outcomes globally regardless of study design. We included a very large retrospective cohort study of Medicare Part D MTM programs operating in 2010.41 This evaluation focused on beneficiaries with congestive heart failure, chronic obstructive pulmonary disease, and diabetes. The authors reported that, although uptake of and adherence to evidence-based medication improved for patients with congestive heart failure or diabetes, these improvements did not consistently translate to fewer condition-specific hospitalizations and emergency department visits.41

Overall, we rated the MTM evidence base as insufficient, even though some studies evaluated important outcomes and had low or moderate risk of bias. Our assessment was often based on inconsistency in the magnitude or direction of effect. Inconsistent effects of the intervention, as well as poor measures, could explain inconsistent results.

For example, clinically effective MTM can either increase or decrease health care use and expenditures based on the needs of the patient. The goal of MTM overall is to optimize pharmacotherapy and reduce unnecessary health care use, but for patients who need more care, MTM could optimize pharmacotherapy while increasing necessary health care use.

Future studies will benefit from better outcome measures. In addition, analyses should take into account whether increases in health care use are appropriate and whether identified inappropriate use can be attributed to overuse or underuse. Studies often used nonstandardized measures for outcomes, such as adverse events, adherence, and expenditures or costs; this tendency limited our ability to meta-analyze results.

Included MTM studies were largely practice based; they varied substantially in usual-care comparators, specific intervention elements, and patient populations. Standard medical care usually involves varying degrees of MTM-like services from the patient’s prescribing providers and health care teams. This reality is problematic for systematic reviews because MTM effectiveness in relationship to usual care can be adequately characterized only by controlling for the variation in the active intervention components that might also be present in the usual-care group.72

For MTM, this variation in practice likely reflects the evolution of the professional practice of pharmacy. Most studies were not designed to capitalize on variants in MTM program elements for a rigorous, prospective evaluation of outcomes by those variants. In addition, most studies did not report patient characteristics beyond age and sex, thus limiting our ability to address the underlying heterogeneity in our review. Finally, most studies did not measure fidelity to intended MTM intervention elements; thus, whether studies demonstrating no effect of MTM were actually failures of implementation is difficult to determine.

Our findings emphasize several important needs for future efforts to review MTM programs systematically. The first is for researchers and program evaluators to specify and design MTM interventions based on existing definitions, taxonomies, and service models. The second is to use the appropriate consensus guidelines for study reporting based on design,7376 giving particular attention to reporting intervention features, usual care practices, and fidelity of intervention delivery.77 Progress on these steps would enable systematic reviews of MTM services to better differentiate between various types of services, different levels of intervention fidelity, and heterogeneous comparison groups.

Medication therapy management is already in widespread practice, which presents both challenges and opportunities for researchers and policy makers. The MTM programs of the future may contribute to coordinated and improved care through delivery within accountable-care organizations or patient-centered medical homes. However, as MTM becomes more integrated into routine health care, the more difficult it is to attribute change to MTM alone. Furthermore, secular trends in related quality-improvement initiatives (eg, medication adherence interventions; practice-based, medication-related patient safety initiatives or requirements; and meaningful use requirements for electronic health records) might obscure the effects of MTM efforts. Positive deviance analyses78 with rigorous measurement of implementation features or stepped wedge trial designs79 may be useful, since they provide rigorous approaches to evaluating real-world implementation. Finally, the population effect of MTM may depend on higher rates of patient participation; future studies and evaluations should consider including measures of reach and examine alternative ways of enrolling patients and keeping them engaged.

Study Limitations

Because MTM services vary substantially, any constraints applied by a systematic review to establish scope necessarily limit the applicability of the review findings. Our review did not address MTM interventions conducted in inpatient settings or single-episode types of interventions (eg, medication reconciliation, which some view as a specific type of MTM service). Although we tried to distinguish MTM from disease or case management interventions, making this distinction was challenging. We allowed MTM interventions that targeted patients with a single condition, such as diabetes or hypertension, as long as the MTM services included a comprehensive review of all medications rather than just medications for the single condition. Despite these limitations, the range of included study designs enhanced the applicability of findings for real-world settings when evidence was sufficient.24

We included interventions labeled pharmaceutical care or medicinesmanagement to ensure that the evidence base included studies before the Medicare Part D MTM era and non-US studies. Although our approach made results more challenging to interpret because of the resultant heterogeneity, it ensured that we captured interventions that had MTM components but lacked the descriptor phrase medication therapy management. We attempted to stratify findings by whether the study was a Part D program, but we did not have enough studies that used the same outcomes to be able to draw conclusions.

Future Research

New research should be based on national priorities. Studies designed to identify causal relationships between MTM interventions and their outcomes (including cost-effectiveness analyses) must control for confounding, but they may offer limited information on the elements that explain program success or failure. Studies designed to explore the reasons for program success or failure using qualitative or single-arm designs may offer hypotheses-generating rather than hypotheses-confirming insights on MTM effectiveness. New research, regardless of specific focus, will likely continue to find inconsistent results until studies account for underlying sources of variability in populations, interventions, and outcome measures.

Conclusions

We found a low strength of evidence of benefit for a limited number of intermediate and health care use and cost outcomes. Evidence was insufficient for most outcomes because of inconsistency in direction and magnitude and also because of imprecision. Wide variations in populations and MTM interventions likely explain these inconsistencies.

Back to top
Article Information

Accepted for Publication: August 27, 2014.

Corresponding Author: Meera Viswanathan, PhD, RTI International, 3040 E Cornwallis Rd, PO Box 12194, Research Triangle Park, NC 27709 (viswanathan@rti.org).

Published Online: November 17, 2014. doi:10.1001/jamainternmed.2014.5841.

Author Contributions: Drs Viswanathan and Kahwati had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Viswanathan, Kahwati, Golin, Blalock, Coker-Schwimmer.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Viswanathan, Kahwati, Golin, Blalock, Coker-Schwimmer, Lohr.

Statistical analysis: Viswanathan, Kahwati, Blalock, Coker-Schwimmer.

Obtained funding: Viswanathan, Lohr.

Administrative, technical, or material support: All authors.

Study supervision: Viswanathan.

Conflict of Interest Disclosures: None reported.

Funding/Support: This project was funded under contract No. HHSA290201200008I from the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services, to RTI International.

Role of the Funder/Sponsor: The AHRQ staff participated in development of the scope of work and reviewed drafts of the manuscript. Approval from the AHRQ was required before the manuscript could be submitted for publication, but the authors are solely responsible for its content and the decision to submit it for publication. The AHRQ staff did not participate in the conduct of the review, data collection, data management, data analysis, interpretation of the data, or preparation of the manuscript. Statements in the report should not be construed as endorsement by the AHRQ or the US Department of Health and Human Services.

References
1.
Adams  K, Corrigan  E. Priority Areas for Quality Improvement. Washington, DC: Institute of Medicine of the National Academies, National Academies Press; 2003.
2.
Kohn  LT, Corrigan  J, Donaldson  MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000.
3.
McGlynn  EA, Asch  SM, Adams  J,  et al.  The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635-2645.
PubMedArticle
4.
Sarkar  U, López  A, Maselli  JH, Gonzales  R.  Adverse drug events in US adult ambulatory medical care. Health Serv Res. 2011;46(5):1517-1533.
PubMedArticle
5.
Warholak  TL, Nau  DP. Quality and Safety in Pharmacy Practice. New York, NY: McGraw-Hill Medical; 2010.
6.
McGivney  MS, Meyer  SM, Duncan-Hewitt  W, Hall  DL, Goode  JV, Smith  RB.  Medication therapy management: its relationship to patient counseling, disease management, and pharmaceutical care. J Am Pharm Assoc (2003).2007;47(5):620-628.
PubMedArticle
7.
Pellegrino  AN, Martin  MT, Tilton  JJ, Touchette  DR.  Medication therapy management services: definitions and outcomes. Drugs. 2009;69(4):393-406.
PubMedArticle
8.
Medicare Prescription Drug, Improvement, and Modernization Act of 2003. 1st ed. 108th Cong (2004).
9.
Bluml  BM.  Definition of medication therapy management: development of professionwide consensus. J Am Pharm Assoc (2003). 2005;45(5):566-572.Article
10.
American Pharmacists Association; National Association of Chain Drug Stores Foundation.  Medication therapy management in pharmacy practice: core elements of an MTM service model (version 2.0). J Am Pharm Assoc (2003). 2008;48(3):341-353.Article
11.
Isetts  BJ, Buffington  DE; Pharmacist Services Technical Advisory Coalition.  CPT code-change proposal: national data on pharmacists’ medication therapy management services. Consult Pharm. 2007;22(8):684-689.
PubMedArticle
12.
American Medical Association. CPT Changes 2006: An Insider's View. Chicago, IL: American Medical Association; 2005:309-312.
13.
Pharmacist Services Technical Advisory Coalition. Medication therapy management service codes.http://www.pstac.org/services/mtms-codes.html. Accessed March 5, 2013.
14.
Viswanathan  M, Kahwati  LC, Golin  CE,  et al. Medication therapy management interventions in outpatient settings; comparative effectiveness review No. 138. Publication No. 14(15)-EHC037-EF. Prepared by the RTI International–University of North Carolina at Chapel Hill Evidence-Based Practice Center (contract No. 290-2012-00008-I). Rockville, MD: Agency for Healthcare Research and Quality; November 17, 2014. http://www.effectivehealthcare.ahrq.gov/reports/final.cfm. Accessed November 17, 2014.
15.
Agency for Healthcare Research and Quality. Evidence-Based Practice Center Systematic Review Protocol Project Title: Medication Therapy Management.http://effectivehealthcare.ahrq.gov/ehc/products/516/1601/medication-therapy-management-protocol-130724.pdf. Accessed October 2, 2014.
16.
Viswanathan  M, Ansari  MT, Berkman  ND,  et al. Methods Guide for Comparative Effectiveness Reviews: Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Rockville, MD: Agency for Healthcare Research and Quality; March 2012. Publication No. 12-EHC047-EF.
17.
Higgins  JP, Altman  DG, Gøtzsche  PC,  et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
PubMedArticle
18.
Viswanathan  M, Berkman  ND.  Development of the RTI item bank on risk of bias and precision of observational studies. J Clin Epidemiol. 2012;65(2):163-178.
PubMedArticle
19.
West  SL, Gartlehner  G, Mansfield  AJ,  et al. Comparative Effectiveness Review Methods: Clinical Heterogeneity: Methods Research Paper. Rockville, MD: Agency for Healthcare Research and Quality; September 2010. Publication No. 10-EHC070-EF.
20.
Higgins  JP, Thompson  SG, Deeks  JJ, Altman  DG.  Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560.
PubMedArticle
21.
Berkman  ND, Lohr  KN, Ansari  M,  et al. Methods Guide for Comparative Effectiveness Reviews: Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Rockville, MD: Agency for Healthcare Research and Quality; November 2013. Publication No. 13(14)-EHC130-EF (prepared by the RTI-UNC Evidence-Based Practice Center under contract No. 290-2007-10056-I).
22.
Guyatt  GH, Oxman  AD, Vist  G,  et al.  GRADE guidelines, IV: rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407-415.
PubMedArticle
23.
Atkins  D, Best  D, Briss  PA,  et al; GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.
PubMedArticle
24.
Atkins  D, Chang  SM, Gartlehner  G,  et al.  Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011;64(11):1198-1207.
PubMedArticle
25.
Touchette  DR, Masica  AL, Dolor  RJ,  et al.  Safety-focused medication therapy management: a randomized controlled trial. J Am Pharm Assoc (2003). 2012;52(5):603-612.
PubMedArticle
26.
Malone  DC, Carter  BL, Billups  SJ,  et al.  Can clinical pharmacists affect SF-36 scores in veterans at high risk for medication-related problems? Med Care. 2001;39(2):113-122.
PubMedArticle
27.
Sellors  J, Kaczorowski  J, Sellors  C,  et al.  A randomized controlled trial of a pharmacist consultation program for family physicians and their elderly patients. CMAJ. 2003;169(1):17-22.
PubMed
28.
Malone  DC, Carter  BL, Billups  SJ,  et al.  An economic analysis of a randomized, controlled, multicenter study of clinical pharmacist interventions for high-risk veterans: the IMPROVE study. Pharmacotherapy. 2000;20(10):1149-1158.
PubMedArticle
29.
Ellis  SL, Billups  SJ, Malone  DC,  et al.  Types of interventions made by clinical pharmacists in the IMPROVE study. Pharmacotherapy. 2000;20(4):429-435.
PubMedArticle
30.
Ellis  SL, Carter  BL, Malone  DC,  et al.  Clinical and economic impact of ambulatory care clinical pharmacists in management of dyslipidemia in older adults: the IMPROVE study. Pharmacotherapy. 2000;20(12):1508-1516.
PubMedArticle
31.
Taylor  CT, Byrd  DC, Krueger  K.  Improving primary care in rural Alabama with a pharmacy initiative. Am J Health Syst Pharm. 2003;60(11):1123-1129.
PubMed
32.
Hanlon  JT, Weinberger  M, Samsa  GP,  et al.  A randomized, controlled trial of a clinical pharmacist intervention to improve inappropriate prescribing in elderly outpatients with polypharmacy. Am J Med. 1996;100(4):428-437.
PubMedArticle
33.
Cowper  PA, Weinberger  M, Hanlon  JT,  et al.  The cost-effectiveness of a clinical pharmacist intervention among elderly outpatients. Pharmacotherapy. 1998;18(2):327-332.
PubMed
34.
Park  JJ, Kelly  P, Carter  BL, Burgess  PP.  Comprehensive pharmaceutical care in the chain setting. J Am Pharm Assoc (Wash). 1996;NS36(7):443-451.
PubMed
35.
Pai  AB, Boyd  A, Depczynski  J, Chavez  IM, Khan  N, Manley  H.  Reduced drug use and hospitalization rates in patients undergoing hemodialysis who received pharmaceutical care: a 2-year, randomized, controlled study. Pharmacotherapy. 2009;29(12):1433-1440..
PubMedArticle
36.
Pai  AB, Boyd  A, Chavez  A, Manley  HJ.  Health-related quality of life is maintained in hemodialysis patients receiving pharmaceutical care: a 2-year randomized, controlled study. Hemodial Int. 2009;13(1):72-79..
PubMedArticle
37.
Welch  EK, Delate  T, Chester  EA, Stubbings  T.  Assessment of the impact of medication therapy management delivered to home-based Medicare beneficiaries. Ann Pharmacother. 2009;43(4):603-610.
PubMedArticle
38.
Moczygemba  LR, Barner  JC, Gabrillo  ER.  Outcomes of a Medicare Part D telephone medication therapy management program. J Am Pharm Assoc (2003). 2012;52(6):e144-e152. doi:10.1331/JAPhA.2012.11258.
PubMedArticle
39.
Moczygemba  LR, Barner  JC, Lawson  KA,  et al.  Impact of telephone medication therapy management on medication and health-related problems, medication adherence, and Medicare Part D drug costs: a 6-month follow up. Am J Geriatr Pharmacother. 2011;9(5):328-338.
PubMedArticle
40.
Moczygemba  LR, Barner  JC, Gabrillo  ER, Godley  PJ.  Development and implementation of a telephone medication therapy management program for Medicare beneficiaries. Am J Health Syst Pharm. 2008;65(17):1655-1660.
PubMedArticle
41.
Perlroth  D, Marrufo  G, Montesinos  A,  et al. Medication Therapy Management in Chronically Ill Populations: Final Report. Baltimore, MD: Center for Medicare & Medicaid Innovation, Centers for Medicare & Medicaid Services; 2013. Contract HHSM-500-2011-00012I/TOT0001.
42.
Brummel  AR, Soliman  AM, Carlson  AM, de Oliveira  DR.  Optimal diabetes care outcomes following face-to-face medication therapy management services. Popul Health Manag. 2013;16(1):28-34.
PubMedArticle
43.
Moore  JM, Shartle  D, Faudskar  L, Matlin  OS, Brennan  TA.  Impact of a patient-centered pharmacy program and intervention in a high-risk group. J Manag Care Pharm. 2013;19(3):228-236.
PubMed
44.
Volume  CI, Farris  KB, Kassam  R, Cox  CE, Cave  A.  Pharmaceutical care research and education project: patient outcomes. J Am Pharm Assoc (Wash). 2001;41(3):411-420.
PubMed
45.
Sidel  VW, Beizer  JL, Lisi-Fazio  D,  et al.  Controlled study of the impact of educational home visits by pharmacists to high-risk older patients. J Community Health. 1990;15(3):163-174.
PubMedArticle
46.
McDonough  RP, Doucette  WR, Kumbera  P, Klepser  DG.  An evaluation of managing and educating patients on the risk of glucocorticoid-induced osteoporosis. Value Health. 2005;8(1):24-31.
PubMedArticle
47.
Gattis  WA, Hasselblad  V, Whellan  DJ, O’Connor  CM.  Reduction in heart failure events by the addition of a clinical pharmacist to the heart failure management team: results of the Pharmacist in Heart Failure Assessment Recommendation and Monitoring (PHARM) Study. Arch Intern Med. 1999;159(16):1939-1945.
PubMedArticle
48.
Jameson  J, VanNoord  G, Vanderwoud  K.  The impact of a pharmacotherapy consultation on the cost and outcome of medical therapy. J Fam Pract. 1995;41(5):469-472.
PubMed
49.
Clifford  RM, Batty  KT, Davis  TME,  et al.  A randomised controlled trial of a pharmaceutical care programme in high-risk diabetic patients in an outpatient clinic. Int J Pharm Pract. 2002;10(2):85-89.Article
50.
Soliman  AM, Carlson  AM, MacLehose  RF, Brummel  AR, Schommer  JC.  Patient characteristics predicting the frequency of medication therapy management visits for patients with diabetes. Clin Ther. 2013;35(4):534-540.
PubMedArticle
51.
Ramalho de Oliveira  D, Brummel  AR, Miller  DB.  Medication therapy management: 10 years of experience in a large integrated health care system. J Manag Care Pharm. 2010;16(3):185-195.
PubMed
52.
Jeong  E, Hui  R, Gallagher  M,  et al. Clinical, Economical and Humanistic Outcomes From a Pharmacist-Managed Medication Therapy Management Program (MTMP) in Medicare Patients From an Integrated Healthcare Delivery System. Oakland, CA: Kaiser Permanante; 2009:1-31.
53.
Hanlon  JT, Schmader  KE.  The medication appropriateness index at 20: where it started, where it has been, and where it may be going. Drugs Aging. 2013;30(11):893-900.
PubMedArticle
54.
Williams  ME, Pulliam  CC, Hunter  R,  et al.  The short-term effect of interdisciplinary medication review on function and cost in ambulatory elderly people. J Am Geriatr Soc. 2004;52(1):93-98.
PubMedArticle
55.
Marques  LA, Galduróz  JC, Fernandes  MR, Oliveira  CC, Beijo  LA, Noto  AR.  Assessment of the effectiveness of pharmacotherapy follow-up in patients treated for depression. J Manag Care Pharm. 2013;19(3):218-227.
PubMed
56.
Yamada  B. Comparison of Outcomes in the Medicare Medication Therapy Management Program Within Kaiser Permanente California. Oakland, CA: Kaiser Permanente; 2012.
57.
Pindolia  VK, Stebelsky  L, Romain  TM, Luoma  L, Nowak  SN, Gillanders  F.  Mitigation of medication mishaps via medication therapy management. Ann Pharmacother. 2009;43(4):611-620.
PubMedArticle
58.
Kassam  R, Farris  KB, Burback  L, Volume  CI, Cox  CE, Cave  A.  Pharmaceutical Care Research and Education Project: pharmacists’ interventions. J Am Pharm Assoc (Wash). 2001;41(3):401-410.
PubMed
59.
Sellors  C, Dalby  DM, Howard  M, Kaczorowski  J, Sellors  J.  A pharmacist consultation service in community-based family practices: a randomized, controlled trial in seniors. J Pharm Technol. 2001;17:264-269.
60.
Roughead  EE, Barratt  JD, Ramsay  E,  et al.  The effectiveness of collaborative medicine reviews in delaying time to next hospitalization for patients with heart failure in the practice setting: results of a cohort study. Circ Heart Fail. 2009;2(5):424-428.
PubMedArticle
61.
Shimp  LA, Kucukarslan  SN, Elder  J,  et al.  Employer-based patient-centered medication therapy management program: evidence and recommendations for future programs. J Am Pharm Assoc (2003). 2012;52(6):768-776.
PubMedArticle
62.
Christensen  DB, Roth  M, Trygstad  T, Byrd  J.  Evaluation of a pilot medication therapy management project within the North Carolina state health plan. J Am Pharm Assoc (2003). 2007;47(4):471-483.
PubMedArticle
63.
Wittayanukorn  S, Westrick  SC, Hansen  RA,  et al.  Evaluation of medication therapy management services for patients with cardiovascular disease in a self-insured employer health plan. J Manag Care Pharm. 2013;19(5):385-395.
PubMed
64.
Krska  J, Cromarty  JA, Arris  F,  et al.  Pharmacist-led medication review in patients over 65: a randomized, controlled trial in primary care. Age Ageing. 2001;30(3):205-211.
PubMedArticle
65.
Fischer  LR, Defor  TA, Cooper  S,  et al.  Pharmaceutical care and health care utilization in an HMO. Eff Clin Pract. 2002;5(2):49-57.
PubMed
66.
Carter  BL, Barnette  DJ, Chrischilles  E, Mazzotti  GJ, Asali  ZJ.  Evaluation of hypertensive patients after care provided by community pharmacists in a rural setting. Pharmacotherapy. 1997;17(6):1274-1285.
PubMed
67.
Barnette  DJ, Murphy  CM, Carter  BL.  Clinical skill development for community pharmacists. J Am Pharm Assoc (Wash). 1996;NS36(9):573-580.
PubMed
68.
Grymonpre  RE, Williamson  DA, Montgomery  PR.  Impact of a pharmaceutical care model for non-institutionalised elderly: results of a randomised, controlled trial. Int J Pharm Pract. 2001;9(4):235-241.Article
69.
Chrischilles  EA, Carter  BL, Lund  BC,  et al.  Evaluation of the Iowa Medicaid pharmaceutical case management program. J Am Pharm Assoc (2003). 2004;44(3):337-349.
PubMedArticle
70.
Winston  S, Lin  YS.  Impact on drug cost and use of Medicare Part D of medication therapy management services delivered in 2007. J Am Pharm Assoc (2003). 2009;49(6):813-820.
PubMedArticle
71.
Witry  MJ, Doucette  WR, Gainer  KL.  Evaluation of the pharmaceutical case management program implemented in a private sector health plan. J Am Pharm Assoc (2003). 2011;51(5):631-635.
PubMedArticle
72.
de Bruin  M, Viechtbauer  W, Schaalma  HP, Kok  G, Abraham  C, Hospers  HJ.  Standard care impact on effects of highly active antiretroviral therapy adherence interventions: a meta-analysis of randomized controlled trials. Arch Intern Med. 2010;170(3):240-250.
PubMedArticle
73.
Boutron  I, Moher  D, Altman  DG, Schulz  KF, Ravaud  P; CONSORT Group.  Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295-309.
PubMedArticle
74.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; Iniciativa STROBE.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies [in Spanish]. Rev Esp Salud Publica. 2008;82(3):251-259.
PubMedArticle
75.
Des Jarlais  DC, Lyles  C, Crepaz  N; TREND Group.  Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004;94(3):361-366.
PubMedArticle
76.
Ogrinc  G, Mooney  SE, Estrada  C,  et al.  The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17(suppl 1):i13-i32.
PubMedArticle
77.
Albrecht  L, Archibald  M, Arseneau  D, Scott  SD.  Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implement Sci. 2013;8:52.
PubMedArticle
78.
Bradley  EH, Curry  LA, Ramanadhan  S, Rowe  L, Nembhard  IM, Krumholz  HM.  Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009;4:25.
PubMedArticle
79.
Brown  CA, Lilford  RJ.  The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.
PubMedArticle
×