[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
Table.  Summary of Surveillance Results Showing Number of Products and Vendors Subjected to Surveillance, Surveilled With Nonconformities Identified, and Surveilled With Nonconformities Categorized as Potential Patient Harm
Summary of Surveillance Results Showing Number of Products and Vendors Subjected to Surveillance, Surveilled With Nonconformities Identified, and Surveilled With Nonconformities Categorized as Potential Patient Harm
1.
Chaudhry  B, Wang  J, Wu  S,  et al.  Systematic review: impact of health information technology on quality, efficiency, and costs of medical care.  Ann Intern Med. 2006;144(10):742-752. doi:10.7326/0003-4819-144-10-200605160-00125PubMedGoogle ScholarCrossref
2.
Howe  JL, Adams  KT, Hettinger  AZ, Ratwani  RM.  Electronic health record usability issues and potential contribution to patient harm.  JAMA. 2018;319(12):1276-1278. doi:10.1001/jama.2018.1171PubMedGoogle ScholarCrossref
3.
Resnic  FS, Normand  SL.  Postmarketing surveillance of medical devices—filling in the gaps.  N Engl J Med. 2012;366(10):875-877. doi:10.1056/NEJMp1114865PubMedGoogle ScholarCrossref
4.
Department of Health and Human Services Office of the National Coordinator. ONC Health IT Certification Program: enhanced oversight and accountability. 45 CFR 170. Effective December 19, 2016. https://www.govinfo.gov/content/pkg/FR-2016-10-19/pdf/2016-24908.pdf. Accessed July 23, 2017.
5.
Anthony  E, Posnack  S. Certification program updates to support efficiency and reduce burden. Office of the National Coordinator for Health Information Technology website. September 21, 2017. https://www.healthit.gov/buzz-blog/healthit-certification/certification-program-updates-support-efficiency-reduce-burden. Accessed July 23, 2019.
6.
Certified Health IT Product List. Department of Health and Human Services website. https://chpl.healthit.gov. Accessed August 20, 2019.
Views 2,634
Citations 0
Research Letter
December 17, 2019

Identifying Potential Patient Safety Issues From the Federal Electronic Health Record Surveillance Program

Author Affiliations
  • 1Georgetown University School of Medicine, Washington, DC
  • 2MedStar Health National Center for Human Factors in Healthcare, Washington, DC
JAMA. 2019;322(23):2339-2340. doi:10.1001/jama.2019.17242

Health care facilities adopting electronic health record (EHR) systems have realized several benefits, but there have also been patient safety risks.1,2 Similar to US Food and Drug Administration postmarket safety surveillance of medical devices, the Department of Health and Human Services’ Office of the National Coordinator (ONC) has the authority to conduct EHR surveillance.3 The ONC’s surveillance investigations seek to determine whether EHR systems, referred to as vendor products, which were certified as meeting specific design, functionality, and security standards set forth by the ONC, still meet those standards when implemented and used. Electronic health record surveillance of vendor products may occur in response to a reported issue (reactive) or can be randomly selected for investigation (randomized).4 Randomized surveillance was formerly required of at least 2% of certified products, although this is no longer mandated.5

Understanding the types and prevalence of issues identified through reactive and randomized surveillance can inform policy and advance EHR safety. We analyzed ONC surveillance data to determine whether identified issues had the potential for patient harm, and the frequency of these issues.

Methods

Each EHR product certified by the ONC receives a unique certified health product listing identification, called the product ID. Each version of a specific vendor product also receives a unique product ID. All publicly available surveillance data from January 28, 2016, to June 24, 2019, were analyzed.6 These data include the type of surveillance activity (reactive or randomized) and a brief description of the issue with the product if one is identified. Identified issues, called nonconformities, can be related to capabilities of the certified EHR product (eg, medication ordering) no longer meeting a design, functionality, or security standard of the certification program or the vendor not completing required disclosures (eg, transparency of product cost). We focused on certified capability nonconformity issues because these may have patient safety implications. Each certified capability nonconformity issue was coded as possible patient harm, defined as an issue that could be a contributing factor to a patient harm event if uncorrected, or unlikely patient harm, defined as an issue that is unlikely to have an effect on a patient’s health (Box). All issues were coded by 2 researchers (84.2% agreement; Cohen κ = 0.68), and all disagreements and possible patient harm issues were reviewed by a practicing physician.

Box Section Ref ID
Box.

Example Nonconformities Coded as Possible Patient Harm or Unlikely Patient Harm

Possible Patient Harm
  • On a dosage entry, the decimal point is removed from the entry.

  • The electronic health record is not properly recording CVX codes in specific circumstances.

  • Oral liquid medication is not restricted to milliliter dosing.

  • RXNorm codes in the clinical care document are inaccurate.

  • Patient portal does not allow download or transmission of a care summary.

  • RxNorm codes are missing in functionality—the system does not contain RxNorm codes for every single prescription medication and medication allergy, including those that represent a nondispensable medication.

  • Issues are reported with information displaying incorrectly in the patient record—laboratory results are not importing to the electronic health record properly.

Unlikely Patient Harm
  • There are blank entries on the audit log.

  • There are multiple issues with electronic clinical quality measures.

  • Promoting Interoperability Measures (measures of patient engagement and electronic exchange of information) are calculating incorrectly.

  • An incorrect user is logged in audit reports.

  • Activity log is not accessible to the patient-authorized representative.

  • The developer allows for the generation of continuity of care documents for all patients; however, the end user must request the creation of these files from the developer’s support staff.

  • Unable to export QRDA I (quality-reporting document architecture) files.

CVX codes are standardized vaccine coding for electronic systems. RxNorm is a uniform medical language for normalized names for clinical drugs.

Results

At the time of this analysis, there were a total of 7415 active certified product IDs recognized by the ONC. Six hundred ninety-seven (9.4% of total product IDs) from 195 vendors were surveilled. Three hundred fifty-eight (51.4% of all products surveilled; 4.8% of total product IDs) from 27 vendors had a certified capability nonconformity issue, and 275 (39.5% of all products surveilled; 3.7% of total product IDs) from 9 EHR vendors were coded as being associated with possible patient harm. The Table provides a summary of total, reactive, and randomized surveillance activities.

The majority of surveillance was reactive, with investigations of 653 product IDs (93.7% of all products surveilled; 8.8% of total product IDs) from 179 vendors. Three hundred fifty-two (53.9% of all products reactively surveilled; 4.7% of total product IDs) from 22 vendors had a certified capability nonconformity issue, and 273 (41.8% of all products reactively surveilled; 3.7% of total product IDs) from 7 vendors were coded as being associated with possible patient harm.

Discussion

These findings begin to estimate the extent of vendor lack of adherence to certification standards and the potential for patient harm, with 3.7% of total product IDs having a nonconformity issue that could be a contributing factor to a patient harm event. Most issues were identified through reactive surveillance, with 41.8% of product IDs reactively surveilled having a possible patient harm issue, although randomized surveillance was infrequently performed. One limitation of the study is that the coding of issues relied on coder clinical experience. Another is that although products associated with possible patient harm issues are widely used by health care facilities, it is unknown whether these issues resulted in actual patient harm.

Section Editor: Jody W. Zylke, MD, Deputy Editor.
Back to top
Article Information

Accepted for Publication: October 1, 2019.

Corresponding Author: Raj M. Ratwani, PhD, MedStar Health National Center for Human Factors in Healthcare, 3007 Tilden St, Ste 6N, Washington, DC 20008 (raj.m.ratwani@medstar.net).

Author Contributions: Mr Pacheco and Dr Ratwani had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: All authors.

Obtained funding: Ratwani.

Administrative, technical, or material support: Hettinger, Ratwani.

Supervision: Hettinger, Ratwani.

Conflict of Interest Disclosures: None reported.

Funding/Support: Mr Pacheco is supported by the Frank S. Pellegrini, MD Medical Student Research Scholarship Endowment. Dr Hettinger is supported by Agency for Healthcare Research and Quality (AHRQ) grant R01HS025136. Dr Ratwani is supported by AHRQ grants R01HS023701 and R01HS025136.

Role of the Funder/Sponsor: The funding agency had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

References
1.
Chaudhry  B, Wang  J, Wu  S,  et al.  Systematic review: impact of health information technology on quality, efficiency, and costs of medical care.  Ann Intern Med. 2006;144(10):742-752. doi:10.7326/0003-4819-144-10-200605160-00125PubMedGoogle ScholarCrossref
2.
Howe  JL, Adams  KT, Hettinger  AZ, Ratwani  RM.  Electronic health record usability issues and potential contribution to patient harm.  JAMA. 2018;319(12):1276-1278. doi:10.1001/jama.2018.1171PubMedGoogle ScholarCrossref
3.
Resnic  FS, Normand  SL.  Postmarketing surveillance of medical devices—filling in the gaps.  N Engl J Med. 2012;366(10):875-877. doi:10.1056/NEJMp1114865PubMedGoogle ScholarCrossref
4.
Department of Health and Human Services Office of the National Coordinator. ONC Health IT Certification Program: enhanced oversight and accountability. 45 CFR 170. Effective December 19, 2016. https://www.govinfo.gov/content/pkg/FR-2016-10-19/pdf/2016-24908.pdf. Accessed July 23, 2017.
5.
Anthony  E, Posnack  S. Certification program updates to support efficiency and reduce burden. Office of the National Coordinator for Health Information Technology website. September 21, 2017. https://www.healthit.gov/buzz-blog/healthit-certification/certification-program-updates-support-efficiency-reduce-burden. Accessed July 23, 2019.
6.
Certified Health IT Product List. Department of Health and Human Services website. https://chpl.healthit.gov. Accessed August 20, 2019.
×