Background Most elderly patients do not receive recommended preventive care, acute care, and care for chronic conditions.
Methods We conducted a controlled trial to assess the effectiveness of electronic medical record (EMR) reminders, with or without panel management, on health care proxy designation, osteoporosis screening, and influenza and pneumococcal vaccinations in patients older than 65 years. Physicians were assigned to 1 of the following 3 arms: EMR reminder, EMR reminder plus panel manager, or control. We assessed completion of recommended practices during a 1-year period.
Results Among patients who had not already received the recommended care, health care proxy was designated in 6.5% of patients in the control arm, 8.8% of the EMR reminder arm, and 19.7% of the EMR reminder plus panel manager arm (P = .002). Bone density screening was completed in 17.7% of patients in the control arm, 19.7% of the EMR reminder arm, and 30.5% of the EMR reminder plus panel manager arm (P = .02). Pneumococcal vaccine was given to 13.1% of patients in the control arm, 19.5% of the EMR reminder arm, and 25.6% of the EMR reminder plus panel manager arm (P = .02). Influenza vaccine was given to 46.8% of patients in the control arm, 56.5% of the EMR reminder arm, and 59.7% of the EMR reminder plus panel manager arm (P = .002). Results were similar when adjusted for individual physician performance in the preceding year, patient age, patient sex, years cared for by the practice, and number of visits.
Conclusions Electronic medical record reminders alone facilitated improvement in vaccination rates and, when augmented by panel management, facilitated further improvement in vaccination rates and boosted the rates of health care proxy designation and bone density screening.
Trial Registration clinicaltrials.gov Identifier: NCT01313169
Primary care physicians (PCPs) provide most of the care to elderly patients in the United States. Evaluations of quality of care in the United States suggest that performance is suboptimal. In an assessment1 of clinicians' performance on 439 indicators of quality of care, only 55% of adult patients received recommended preventive care, acute care, and care for chronic conditions. Evaluations of care targeted to the elderly suggest potentially greater underperformance when looking at advance directive designation, osteoporosis screening, and vaccination rates. Studies2 find that only 18% to 36% of Americans have completed an advance directive, with less than half of terminally ill patients having an advance directive available. Studies3 also show that less than one-third of at-risk women receive bone mineral density testing. Finally, a study4 found that one-third to one-half of adults older than 65 years did not receive influenza and pneumococcal vaccines as recommended.
These shortfalls in performance reflect failure of the current model for the delivery of primary health care services in the United States. It has been estimated that it would take a PCP 18 hours a day to provide all the recommended preventive services and care for chronic conditions to a typical patient panel.5 For most providers and health care systems, adding time to each encounter is not a viable option.6
Applying health information technology offers one approach to help PCPs. Electronic medical record (EMR)–based reminders deliver prompts for a recommended action at the point of care. They have been studied and found to have a potential beneficial effect on practice behaviors, including cancer screening, drug monitoring, advance directive completion, and vaccine administration.7 Although EMR reminders are typically used by individual health care providers at the point of care, they also offer the opportunity for further action through panel management. By providing a synopsis of all active reminders for a health care provider's panel, PCPs can also proactively direct care for their patients using support staff to conduct outreach.8 We developed a new set of EMR reminders targeting the improvement of care provided to elderly patients and conducted a controlled trial to assess the effectiveness of the EMR reminders, with or without the added support of a panel manager, on the following 4 practice behaviors: designation of a health care proxy, osteoporosis screening, and administration of influenza and pneumococcal vaccinations. We hypothesized that EMR reminders would improve adherence to practice guidelines and that benefits would be enhanced with the support of a panel manager.
The Beth Israel Deaconess General Medicine and Primary Care Division has 55 faculty PCPs practicing in 2 separate office locations within an urban academic medical center. Both offices use the same EMR. All practicing faculty physicians were invited and all consented to participate. The Committee on Clinical Investigations approved this study.
Eligible patients were required to meet the following criteria: 65 years or older at the start of the study, having a designated faculty PCP at the start of the study, and completion of at least 1 visit to the practice in the 18 months before the study start. Faculty PCPs assigned to receive the EMR reminders received a written overview with screenshots followed by a live demonstration of the new features. Finally, faculty were invited to meet with one of us (T.S.L.) for further instruction. Instruction about use of the reminders did not include education about the clinical guidelines.
This study included 3 arms: EMR reminder, EMR reminder plus panel manager, and usual care (control). About two-thirds of the faculty practice (n = 34) were located in office A and were provided with the new EMR reminders. Half of this group (n = 17) was then randomly assigned to also have a panel manager to assist them with the 4 targeted practice behaviors. The 20 remaining faculty practicing in office B were assigned to the control arm; to prevent cross-contamination, they all continued to use the current EMR without the benefit of EMR reminders or a panel manager.
The intervention took place from May 1, 2009, to May 1, 2010. The EMR reminders were selected from published recommendations from the Assessing Care of Vulnerable Elders project, Agency for Healthcare Research and Quality, the Mayo Clinic, National Institute on Aging, and the United States Preventive Services Task Force and included advance directives, aspirin chemoprevention, Beers medication alerts, fall screening, osteoporosis screening, and vaccinations9-13 (eTable 1). This set of reminders was displayed in each patient's EMR available at the point of care (Figure 1) but also in summary form in the provider's panel list for patients older than 65 years to facilitate panel management (eFigure). The new EMR reminders were activated for both intervention arms (EMR reminder and EMR reminder plus panel manager) at the study start, whereas the control arm continued to use the existing EMR without the new reminders. Although the reminders address varied areas of care, we focused our study on the effect of the EMR reminders on the following 4 areas: completing a health care proxy designation, screening for osteoporosis, and providing influenza and pneumococcal vaccinations.
Most practicing faculty physicians were already familiar with EMR reminder functionality because EMR reminders for screening, preventive, and diabetes care were already in use at this center before this study. An active geriatric reminder was displayed by a “geriatrics alerts” link on the patient's profile, the first screen visualized when opening a record. Clicking on this link would bring the user to a new screen, a geriatrics sheet displaying all the active geriatric EMR reminders and the last status for each reminder (Figure 1). Each reminder displayed was an active link, allowing the user to respond to the reminder by proceeding or declining the suggested action. By design, viewing or responding to a reminder was voluntary and completely at the discretion of the user.
For the patients whose PCP was randomized to the EMR plus panel manager arm, the panel manager assisted patients and physicians in completing the 4 targeted practice behaviors. Our panel manager was an administrative assistant without any specific clinical training who spent half of his time working on this study. He was located off-site and communicated with physicians primarily through e-mail. The panel manager began by reviewing the EMR's geriatrics patient list for each assigned provider; the list displays which patients had no health care proxy designated, were due for osteoporosis screening, or were due for pneumococcal or influenza vaccination. This work list was then forwarded to the PCP to inform him or her of the items due and to obtain approval to contact the patient and facilitate completion of the items. If approved, up to 3 attempts were made to contact the patient by telephone and, if not reached, a letter with the same content that would have been provided by telephone was sent. Two cycles of contact were attempted, one during the first 6-month period and then another during the last 6-month period. One dedicated round of contact was made for influenza vaccination during the fall of 2009 because of the time-sensitive nature of this task.
On reaching the patient, the panel manager verified that the patient had not already completed the item due. If completed, he updated the information in the EMR and made a note that the action had already been completed before the intervention; the panel manager then asked the patient to send or bring the relevant record to their PCP for review. The panel manager described the care due and stated explicitly that their PCP recommended the care. The panel manager facilitated completion by mailing health care proxy information and forms to the patient, placing an order for a bone density scan and scheduling it, or placing an immunization order and scheduling it. If the patient declined action on any proposed item, a letter summarizing the recommended action was sent to the patient and forwarded electronically to the PCP to inform him or her of contact and to suggest that items due be addressed at the next office visit.
The primary outcome measures were (1) designation of a health care proxy, (2) initial bone density scan, (3) administration of annual influenza vaccine, and (4) administration of pneumococcal vaccine. A designated health care proxy was defined as a verified proxy documented in the EMR's health care proxy field. The initial bone density scan measure was defined as having a bone density scan ordered or completed. Administration of influenza and pneumococcal vaccine was defined as vaccination documented as given at any location.
We compared physician and patient characteristics using Kruskal-Wallis tests for continuous characteristics and Pearson χ2 tests for categorical variables.
The study assigned interventions at the physician level, but the outcomes were measured at the patient level. To account for within-physician correlation, we used generalized estimating equation methods to fit logistic regression models when comparing arms. For each outcome (as well as baseline performance on each of the measures), we initially compared the 3 arms using a Wald test with 2 df. If the 3-way comparison was significant, pairwise comparisons were performed without explicit adjustment for multiple comparisons.
In addition to unadjusted models, we also fit models adjusted for baseline differences among the arms. To adjust for baseline differences in physicians' practice patterns, we measured and adjusted for the physician's baseline tendency to provide the specific types of care being studied. For influenza vaccination, we used the physician's 2008-2009 influenza season vaccination rate. For each of the other outcomes, we used the physician's performance during the year before the study, defined as the percentage of patients among those eligible who received the recommended care in that year. We also adjusted for patient characteristics (age, sex, years cared for by the practice, and number of visits during the study period). We report adjusted odds ratios (ORs) and their 95% confidence intervals (CIs). We included all patients eligible for each of the health care recommendations. Because influenza vaccination is recommended annually, all patients were included. Bone density screening was limited to women who had not undergone previous testing. Pneumococcal vaccine administration and health care proxy designation included all patients except those who had the intervention before the start of the study.
Based on patient panel data from the 18 months before our study start and assuming within-physician correlation of 0.02, we estimated that our study had 97% power to detect improvement from 75% to 85% for influenza vaccine and had 92% power to detect improvement from 50% to 65% for pneumococcal vaccination. All P values were 2-sided.
All practicing faculty physicians agreed to participate and completed the study, except one assigned to the EMR reminder arm who departed the practice in the eighth month (Figure 2). Table 1 shows the characteristics of the physicians included in the study. There were differences between the control and intervention arms in physician sex, years since completion of training, and percentage of patients older than 65 years, none of which were statistically significant.
Four thousand six hundred sixty patients met the eligibility criteria for inclusion (Figure 2). At baseline, there were differences in patient characteristics among the 3 arms (Table 1). However, there were no significant differences across the arms in performance on the 4 quality measures in the year before the study (eTable 2) or in the percentage of patients who already had completed the recommended care before the start of the study (Table 2), indicating no imbalance across the 3 groups in PCP and patient probability of completing recommended care.
Performance on each quality measure before and after the intervention and the differences in performance are shown in Table 2. To more accurately gauge the effect of the intervention, we focused our statistical analyses on those who had not received the recommended care at the start of the study (Table 3). Influenza vaccination was an exception, because vaccination is recommended annually and all patients were therefore eligible. Among patients without a health care proxy designated at the start of the study, a health care proxy was designated during the study in 6.5% of patients in the control arm, in 8.8% in the EMR reminder arm, and in 19.7% in the EMR reminder plus panel manager arm (Table 3). Compared with the control arm, differences in performance were statistically significant for the EMR reminder plus panel manager arm (unadjusted OR, 3.40; 95% CI, 2.33-4.97; P < .001) but not for the EMR reminder arm (P = .08) (Table 4). The EMR reminder plus panel manager achieved statistically higher rates of performance than the EMR reminder alone (unadjusted OR, 2.34; 95% CI, 1.64-3.33; P < .001) (eTable 3).
Among patients who had not undergone bone density screening before the start of the study, a test was completed during the study in 17.7% of patients in the control arm, in 19.7% in the EMR reminder arm, and in 30.5% in the EMR reminder plus panel manager arm (Table 3). Similar to health care proxy designation, differences in performance were statistically significant for the EMR reminder plus panel manager arm compared with the control arm (unadjusted OR, 1.97; 95% CI, 1.28-3.04; P = .002) but not for the EMR reminder alone (P = .68) (Table 4). The EMR reminder plus panel manager achieved statistically higher rates of bone density screening than the EMR reminder alone (unadjusted OR, 1.80; 95% CI, 1.18-2.73; P = .006) (eTable 3).
Influenza vaccine was given during the study to 46.8% of patients in the control arm, in 56.5% in the EMR reminder arm, and in 59.7% in the EMR reminder plus panel manager arm (Table 3). Compared with the control arm, differences in performance were statistically significant for the EMR reminder (unadjusted OR, 1.44; 95% CI, 1.18-1.76; P < .001) and EMR reminder plus panel manager (unadjusted OR, 1.68; 95% CI, 1.34-2.10; P < .001) (Table 4). The difference in performance between the EMR reminder plus panel manager and the EMR reminder alone was not statistically significant (unadjusted OR, 1.16; 95% CI, 0.93-1.45; P = .19) (eTable 3).
Among patients who had not yet received pneumococcal vaccine before the start of the study, pneumococcal vaccine was given during the study in 13.1% of patients in the control arm, in 19.5% in the EMR reminder arm, and in 25.6% in the EMR reminder plus panel manager arm (Table 3). Compared with the control arm, differences in performance were statistically significant for the EMR reminder (unadjusted OR, 1.47; 95% CI, 1.02-2.11; P = .04) and the EMR reminder plus panel manager (unadjusted OR, 2.05; 95% CI, 1.31-3.23; P = .002) (Table 4). The difference in performance between the EMR reminder plus panel manager and EMR reminder alone arms was not statistically significant (unadjusted OR, 1.40; 95% CI, 0.88-2.21; P = .15) (eTable 3).
Results were similar in analyses adjusted for individual physician performance in the preceding year, patient age, sex, years cared for by the practice, and number of visits during the study period (Table 4 and eTable 3).
We found EMR reminders effective in helping physicians successfully achieve higher rates of health care proxy designation, osteoporosis screening, and influenza and pneumococcal vaccinations. Electronic medical record reminders alone facilitated significant improvement in influenza and pneumococcal vaccination rates, whereas EMR reminders with panel management facilitated significant improvement in health care proxy designation and osteoporosis screening rates.
The effectiveness of EMR reminders varies by the practice behavior targeted.14-17 A meta-analysis found the greatest increases in vaccination rates and the least in mammogram and Papanicolaou smear rates.16 Electronic medical record reminders targeting advance directives have shown inconsistent results ranging from modest to no benefit.18,19 One study targeting osteoporosis screening found only a modest increase in rate.20 Our EMR reminders yielded similar results with significant increases in vaccination rates but only nonsignificant increases in health care proxy designation and osteoporosis screening. This differential effect might be due in part to the greater time and effort required to address the latter 2 tasks, deterring providers from addressing them. Recent efforts augmenting computer-generated work lists with ancillary staff support have also shown increases in recommended care provided.21,22 Although these interventions were largely implemented around an office encounter, our panel manager contacted patients at home by telephone and mail. In our study, panel management facilitated substantial increases in health care proxy designation and osteoporosis screening rates. Panel management also augmented the improvements in vaccination rate achieved with the EMR reminder alone, but the additional benefits were not statistically significant. The effects of panel management on health care proxy designation and osteoporosis screening are not well studied, but the increases we observed are comparable in magnitude to the greater effects reported with panel management.21 The substantial effect we observed might be explained by the transfer of a time- and effort-intensive task from an overburdened PCP visit to a contact performed by ancillary staff at a time other than that of the visit. The smaller effect on vaccinations is consistent with other studies of panel management and might be because patients are responsive to a direct recommendation by their PCP during a face-to-face encounter and are able to immediately receive vaccinations at the point of care. Although we observed EMR reminders—alone or with panel management—to have beneficial effects similar to those observed in previous reports, this study might have underestimated their maximal effect because our analysis focused only on those who had not received the recommended care at the study start. Those patients constitute a population that may have already been predisposed to decline the recommended care because of preference, severity of medical illness, cognitive impairment, or a language barrier.
Our study has several limitations. We detected differences in the characteristics between the intervention and control arm physicians, likely because physicians were assigned by office location rather than at random. However, results were similar in analyses that adjusted for physicians' baseline performance on the target practice behaviors and for patient characteristics (Table 4). This study was performed at a single site at a hospital-based academic practice, which may not be representative of a typical outpatient primary care practice. The EMR software was developed at Beth Israel Deaconess Medical Center. Implementation and presentation of EMR reminders might differ from those of other EMR software. Our EMR reminders were designed to be used at the discretion of the provider; viewing or responding was not forced as it is with other EMR software. This minimized the burden placed on physicians but may have reduced the use of reminders. Our study period was 12 months, which may have been too short to detect reminder fatigue, limiting our ability to estimate the long-term effects of our reminders. The results from the EMR reminder plus panel manager arm are based on the efforts of a single panel manager, but his work followed a detailed written protocol and telephone and mail contact followed a written script, making the processes reproducible. Finally, influenza vaccination rates were adversely affected during our study by the unexpected appearance of the H1N1 subtype of influenza virus A and shortage of influenza vaccine, potentially blunting the effect of our interventions, but these factors should have had a similar effect across the 3 study arms.
Our results suggest that EMR reminders and panel management are effective when they assist physicians without adding significant burden to their workload. An e-mail survey of our faculty conducted midway through the study found one of the most frequently cited barriers to using EMR reminders was that they would add additional time to the clinic visit. Extending the reach of our physicians with a panel manager proved very effective at addressing time- and effort-intensive tasks that might otherwise have been neglected. This supports the concept that PCPs working with a team of health care professionals equipped with electronically generated work lists, as proposed in the patient-centered medical home, might narrow gaps in the performance of care provided by PCPs.22
Electronic medical record reminders proved effective in helping physicians successfully achieve higher rates of health care proxy designation, osteoporosis screening, and influenza and pneumococcal vaccination. Electronic medical record reminders alone facilitated significant improvement in vaccination rates. Panel management improved vaccination rates even further and significantly boosted the rates of health care proxy designation and bone density screening. These results suggest a 2-tiered approach with point-of-care EMR reminders for tasks completed with nominal effort and panel management for tasks that are more time and effort intensive. Our results support the concept that team-based care using electronic registries and work lists is more likely to successfully deliver recommended care than individual PCPs working on their own.
Correspondence: Timothy S. Loo, MD, Department of General Medicine, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Boston, MA 02215 (tloo@bidmc.harvard.edu).
Accepted for Publication: May 25, 2011.
Author Contributions: Dr Loo had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Loo, Lipsitz, Bates, Agarwal, Markson, and Hamel. Acquisition of data: Loo, Irish, and Markson. Analysis and interpretation of data: Loo, Davis, Lipsitz, Irish, Markson, and Hamel. Drafting of the manuscript: Loo, Lipsitz, and Hamel. Critical revision of the manuscript for important intellectual content: Loo, Davis, Irish, Bates, Agarwal, Markson, and Hamel. Statistical analysis: Davis and Irish. Obtained funding: Lipsitz, Bates, and Agarwal. Administrative, technical, and material support: Loo, Lipsitz, Irish, and Markson. Study supervision: Loo, Lipsitz, and Hamel.
Financial Disclosure: None reported.
Funding/Support: This study was supported by the Donald W. Reynolds Foundation.
Additional Contributions: Russell Phillips, MD, provided guidance and support, and Carolyn Conti, BCS, provided expertise in data management.
1. McGlynn EA, Asch SM, Adams J,
et al. The quality of health care delivered to adults in the United States.
N Engl J Med. 2003;348(26):2635-264512826639
PubMedGoogle ScholarCrossref 3.Lafata JE, Kolk D, Peterson EL,
et al. Improving osteoporosis screening: results from a randomized cluster trial.
J Gen Intern Med. 2007;22(3):346-35117356966
PubMedGoogle ScholarCrossref 5.Margolius D, Bodenheimer T. Transforming primary care: from past practice to the practice of the future.
Health Aff (Millwood). 2010;29(5):779-78420439861
PubMedGoogle ScholarCrossref 7.Weiner M, Callahan CM, Tierney WM,
et al. Using information technology to improve the health care of older adults.
Ann Intern Med. 2003;139(5, pt 2):430-43612965971
PubMedGoogle Scholar 8.Neuwirth EB, Schmittdiel JA, Tallman K, Bellows J. Understanding panel management: a comparative study of an emerging approach to population care.
Perm J. 2007;11(3):12-2021461107
PubMedGoogle Scholar 9.Shekelle PG, MacLean CH, Morton SC, Wenger NS. ACOVE quality indicators.
Ann Intern Med. 2001;135(8, pt 2):(suppl, pt 2)
653-66711601948
PubMedGoogle Scholar 10.US Department of Health and Human Services. The Pocket Guide to Staying Healthy 50+. Rockville, MD: Agency for Healthcare Research and Quality, US Dept of Health and Human Services; November 2003. AHRQ publication 04-IP001-A
11.Takahashi PY, Okhravi HR, Lim LS, Kasten MJ. Preventive health care in the elderly population: a guide for practicing physicians.
Mayo Clin Proc. 2004;79(3):416-42715008616
PubMedGoogle ScholarCrossref 12.National Institute on Aging. Bound for Your Good Health: A Collection of Age Pages. Bethesda, MD: National Institutes of Health; 2005
13.US Preventive Services Task Force. The Guide to Clinical Preventive Services 2007. Rockville, MD: Agency for Healthcare Research and Quality; September 2007. AHRQ publication 07-0510
14. McDonald CJ, Hui SL, Smith DM,
et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial.
Ann Intern Med. 1984;100(1):130-1386691639
PubMedGoogle Scholar 15.Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research.
Ann Intern Med. 1994;120(2):135-1428256973
PubMedGoogle Scholar 16.Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians.
Arch Intern Med. 2000;160(3):301-30810668831
PubMedGoogle ScholarCrossref 17.Demakis JG, Beauchamp C, Cull WL,
et al. Improving resident's compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders.
JAMA. 2000;284(11):1411-141610989404
PubMedGoogle ScholarCrossref 18.Dexter PR, Wolinsky FD, Gramelspacher GP,
et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms: a randomized, controlled trial.
Ann Intern Med. 1998;128(2):102-1109441569
PubMedGoogle Scholar 19.Heiman H, Bates DW, Fairchild D, Shaykevich S, Lehmann LS. Improving completion of advance directives in the primary care setting: a randomized controlled trial.
Am J Med. 2004;117(5):318-32415336581
PubMedGoogle ScholarCrossref 20.Kanter M, Martinez O, Lindsay G, Andrews K, Denver C. Proactive office encounter: a systematic approach to preventive and chronic care at every patient encounter.
Perm J. 2010;14(3):38-4320844703
PubMedGoogle Scholar 21.Feldstein AC, Perrin NA, Unitan R,
et al. Effect of a patient panel-support tool on care delivery.
Am J Manag Care. 2010;16(10):e256-e26620964477
PubMedGoogle Scholar 22.American College of Physicians. The Advanced Medical Home: A Patient-Centered, Physician-Guided Model of Health Care. Philadelphia, PA: American College of Physicians; 2005. Position paper