[Skip to Content]
[Skip to Content Landing]
Invited Commentary
June 2014

Measuring the Impact of “Meaningful Use” on Quality of Care

Author Affiliations
  • 1Department of Healthcare Policy and Research, Weill Cornell Medical College, New York, New York
  • 2Department of Medicine, Weill Cornell Medical College, New York, New York
  • 3Health Information Technology Evaluation Collaborative, New York, New York
  • 4Center for Healthcare Informatics and Policy, Weill Cornell Medical College, New York, New York
  • 5Department of Pediatrics, Weill Cornell Medical College, New York, New York
  • 6New York-Presbyterian Hospital, New York, New York
JAMA Intern Med. 2014;174(6):998-999. doi:10.1001/jamainternmed.2013.14092

Through the Health Information Technology Economic and Clinical Health Act (HITECH) of 2009, the federal government is investing nearly $30 billion in incentives for hospitals and health care providers to adopt and meaningfully use electronic health records (EHRs). The federal government’s goal is to be transformative—to enable new and improved ways of managing patients that cannot easily be accomplished via paper—rather than merely doing electronically what was previously done on paper.

The federal government has defined “meaningful use” (MU) in detail. Eligible hospitals and health care providers who meet these definitions can receive the financial incentive payments, which began in 2011. For example, in the first stage of the MU program, health care providers can earn financial incentives for meeting 15 “core” objectives, 5 of 10 “menu” objectives, and a total of 6 clinical quality measures. Examples of MU objectives include implementing computerized physician order entry, maintaining an up-to-date problem list, and generating and transmitting prescriptions electronically.

In this issue of JAMA Internal Medicine, Samal et al1 report on a cross-sectional study of 858 physicians at the Brigham and Women’s Hospital and affiliated ambulatory practices. They collected data on the physicians, all of whom were using EHRs, for a 3-month period in 2012, and then compared the quality of care provided by the 540 physicians who met MU criteria with the quality of care provided by the 318 who did not. The authors found essentially no association between MU and quality for 7 clinical quality measures that are included in the MU program. Meaningful users provided slightly higher quality of care for 2 measures, worse quality for 2 measures, and similar quality for 3 measures, compared with non–meaningful users.

This study raises important questions about how to measure the effects of MU and whether MU improves quality. The effects of MU on quality are not yet well understood. Previous studies have suggested that EHRs are associated with higher quality of care, but it is not known whether achieving MU per se will result in greater quality gains than adoption of EHRs without achieving MU.2,3 Samal et al1 have begun to address this issue, but additional studies are needed.

One major issue to consider when interpreting studies that explore the relationship between MU and quality is the study setting. The Brigham and Women’s Hospital is a national leader in the use of EHRs to improve quality and safety, and their physicians have extensive experience using an advanced EHR product. It is notable that this study was conducted at a time when the Brigham had a home-grown EHR that had been iteratively refined for decades, but that since then the Brigham has been transitioning to a commercially available EHR.4 Other studies are needed in settings that are typical of the rest of the country, including community-based settings with commercially available EHRs, in order to maximize generalizability.

Another issue is the duration of EHR use. This study took place at the end of 2012, which was the second year of the MU program. It is not clear how long these physicians had been using EHRs prior to the study period. Many meaningful users will be implementing EHRs for the first time because of this policy. Studies on applications for e-prescribing have shown that, even 2 years after implementation, physicians are still on a learning curve, improving their care.5 Thus, studies conducted too soon after implementation may not find an effect, even if one exists.

A third issue is the reliability of electronic reporting of quality measures. Previous studies have shown that automated electronic algorithms for extracting quality data from EHRs are not always accurate.6 Automated reporting can underestimate or overestimate rates of recommended care because it tends to capture only those elements that are structured fields (eg, drop-down menus, check boxes, or other similar fields) and not those elements that are documented as free text. These challenges can be addressed, with both more nuanced specifications for automated reporting and more structured documentation of the care provided.

A fourth issue is the importance of understanding how physicians actually use EHRs to achieve MU and how that usage affects medical decision-making.7 The study by Samal et al1 and other similar studies could be strengthened by measuring not only achievement of MU as the predictor variable but actual usage of EHRs as well. Without measurement of actual usage, any apparent association between achievement of MU and quality could be confounded by unmeasured variables.

Finally, MU is being rolled out in stages, and each stage is designed to become more complex and more difficult to achieve than the previous one. Stage 1 MU, which the study by Samal et al1 examined, began in 2011 and involves attesting to the use of EHRs to capture clinical data electronically. For example, health care providers attest to measures such as recording vital signs in the EHR for at least 50% of patients. Stage 2 begins in 2014 and raises the thresholds for many measures (eg, record vital signs for at least 80% of patients). Stage 2 also promotes electronic health information exchange and carries the option of reporting performance on quality measures electronically, among other goals. Stage 3 is expected to begin in 2017 and to reward providers for not only reporting the level of quality provided but for improving on that level as well. Thus, the full effects of MU on quality may not be measurable until stages 2 or 3.

In conclusion, the MU program is unprecedented, both in terms of the magnitude of the financial incentives and in the degree to which it is shaping day-to-day clinical care. It is not clear whether MU will improve quality, but it is also not clear that quality would be improved without adoption and use of EHRs. Electronic health records are powerful tools with which to manage populations of patients, and there are few ways to manage populations of patients with paper records. Quality is also only 1 relevant outcome; measuring the value of health care, which incorporates both quality and cost, is extremely important. The need to adopt and use EHRs is also leading to many secondary effects in health care delivery, including the merging of small practices that do not have enough resources to adopt EHRs on their own. Ongoing evaluation is critical to understanding the effects of the transformative MU program, not only on patients but also on the health care system.

Back to top
Article Information

Corresponding Author: Lisa M. Kern, MD, MPH, Weill Cornell Medical College, 425 E 61st St, New York, NY 10065 (lmk2003@med.cornell.edu).

Published Online: April 14, 2014. doi:10.1001/jamainternmed.2013.14092.

Conflict of Interest Disclosures: None reported.

Samal  L, Wright  A, Healy  MJ, Linder  JA, Bates  DW.  Meaningful use and quality of care [published online April 14, 2014].  JAMA Intern Med. doi:10.1001/jamainternmed.2014.662.Google Scholar
Buntin  MB, Burke  MF, Hoaglin  MC, Blumenthal  D.  The benefits of health information technology: a review of the recent literature shows predominantly positive results.  Health Aff (Millwood). 2011;30(3):464-471.PubMedGoogle ScholarCrossref
Kern  LM, Barrón  Y, Dhopeshwarkar  RV, Edwards  A, Kaushal  R; HITEC Investigators.  Electronic health records and ambulatory quality of care.  J Gen Intern Med. 2013;28(4):496-503.PubMedGoogle ScholarCrossref
Classen  DC, Bates  DW.  Finding the meaning in meaningful use.  N Engl J Med. 2011;365(9):855-858.PubMedGoogle ScholarCrossref
Abramson  EL, Malhotra  S, Osorio  SN,  et al.  A long-term follow-up evaluation of electronic health record prescribing safety.  J Am Med Inform Assoc. 2013;20(e1):e52-e58.PubMedGoogle ScholarCrossref
Kern  LM, Malhotra  S, Barrón  Y,  et al.  Accuracy of electronically reported “meaningful use” clinical quality measures: a cross-sectional study.  Ann Intern Med. 2013;158(2):77-83.PubMedGoogle ScholarCrossref
Lanham  HJ, Sittig  DF, Leykum  LK, Parchman  ML, Pugh  JA, McDaniel  RR.  Understanding differences in electronic health record (EHR) use: linking individual physicians’ perceptions of uncertainty and EHR use patterns in ambulatory care.  J Am Med Inform Assoc. 2014;21(1):73-81.PubMedGoogle ScholarCrossref