[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.  Study Participant Flowchart
Study Participant Flowchart
Table 1.  Characteristics of Patients Who Read 1 or More Notes in the Past 12 Months by Whether They Reported Finding a Mistake
Characteristics of Patients Who Read 1 or More Notes in the Past 12 Months by Whether They Reported Finding a Mistake
Table 2.  Multivariable Analysis of Patient Characteristics and Likelihood of Reporting a Somewhat Serious or Very Serious Mistake in Visit Notesa
Multivariable Analysis of Patient Characteristics and Likelihood of Reporting a Somewhat Serious or Very Serious Mistake in Visit Notesa
Table 3.  Categories and Examples of Very Serious Mistakes Reported by Patients
Categories and Examples of Very Serious Mistakes Reported by Patients
1.
Henry J Kaiser Family Foundation. KFF health tracking poll. 2019. Accessed May 3, 2019. http://files.kff.org/attachment/Topline-KFF-Health-Tracking-Poll-January-2019
2.
Koppel  R. Patient safety and health information technology: learning from our mistakes perspective. AHRQ Patient Saf Network. 2012;(4):5-8. Accessed April 18, 2020. https://psnet.ahrq.gov/perspective/patient-safety-and-health-information-technology-learning-our-mistakes
3.
Weir  CR, Hurdle  JF, Felgar  MA, Hoffman  JM, Roth  B, Nebeker  JR.  Direct text entry in electronic progress notes. An evaluation of input errors.   Methods Inf Med. 2003;42(1):61-67. doi:10.1055/s-0038-1634210PubMedGoogle ScholarCrossref
4.
Suresh  G.  Don’t believe everything you read in the patient’s chart.   Pediatrics. 2003;111(5 pt 1):1108-1109. doi:10.1542/peds.111.5.1108 PubMedGoogle ScholarCrossref
5.
Staroselsky  M, Volk  LA, Tsurikova  R,  et al.  An effort to improve electronic health record medication list accuracy between visits: patients’ and physicians’ response.   Med Inform (Lond). 2008;77(3):153-160. doi:10.1016/j.ijmedinf.2007.03.001PubMedGoogle ScholarCrossref
6.
Kaboli  PJ, McClimon  BJ, Hoth  AB, Barnett  MJ.  Assessing the accuracy of computerized medication histories.   Am J Manag Care. 2004;10(11 pt 2):872-877.PubMedGoogle Scholar
7.
Graber  ML, Byrne  C, Johnston  D.  The impact of electronic health records on diagnosis.   Diagnosis (Berl). 2017;4(4):211-223. doi:10.1515/dx-2017-0012 PubMedGoogle ScholarCrossref
8.
Yadav  S, Kazanji  N, K C  N,  et al.  Comparison of accuracy of physical examination findings in initial progress notes between paper charts and a newly implemented electronic health record.   J Am Med Inform Assoc. 2017;24(1):140-144. doi:10.1093/jamia/ocw067 PubMedGoogle ScholarCrossref
9.
Payne  TH, Alonso  WD, Markiel  JA,  et al.  Using voice to create inpatient progress notes: effects on note timeliness, quality, and physician satisfaction.   JAMIA Open. 2018;1(2):218-226. doi:10.1093/jamiaopen/ooy036 PubMedGoogle ScholarCrossref
10.
Secretary of Health and Human Services. Report to Congress on Health IT Progress: Examining the HITECH Era and the Future of Health IT. Vol 3001. Dept of Health and Human Services; 2016. https://dashboard.healthit.gov/report-to-congress/2016-report-congress-examining-hitech-era-future-health-information-technology.php
11.
Bell  SK, Mejilla  R, Anselmo  M,  et al.  When doctors share visit notes with patients: a study of patient and doctor perceptions of documentation errors, safety opportunities and the patient-doctor relationship.   BMJ Qual Saf. 2017;26(4):262-270. doi:10.1136/bmjqs-2015-004697 PubMedGoogle ScholarCrossref
12.
Walker  JM, Carayon  P, Leveson  N,  et al.  EHR safety: the way forward to safe and effective systems.   J Am Med Inform Assoc. 2008;15(3):272-277. doi:10.1197/jamia.M2618 PubMedGoogle ScholarCrossref
13.
Colicchio  TK, Cimino  JJ, Del Fiol  G.  Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the post-meaningful use era.   J Med internet Res. 2019;21(6):e13313. doi:10.2196/13313 PubMedGoogle Scholar
14.
McDonald  CJ.  Computerization can create safety hazards: a bar-coding near miss.   Ann Intern Med. 2006;144(7):510-516. doi:10.7326/0003-4819-144-7-200604040-00010 PubMedGoogle ScholarCrossref
15.
Koppel  R, Metlay  JP, Cohen  A,  et al.  Role of computerized physician order entry systems in facilitating medication errors.   JAMA. 2005;293(10):1197-1203. doi:10.1001/jama.293.10.1197 PubMedGoogle ScholarCrossref
16.
OpenNotes. OpenNotes Map. Accessed April 27, 2020. https://www.opennotes.org/join/map/
17.
Delbanco  T, Walker  J, Bell  SK,  et al.  Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead.   Ann Intern Med. 2012;157(7):461-470. doi:10.7326/0003-4819-157-7-201210020-00002 PubMedGoogle ScholarCrossref
18.
Bell  SK, Gerard  M, Fossa  A,  et al.  A patient feedback reporting tool for OpenNotes: implications for patient-clinician safety and quality partnerships.   BMJ Qual Saf. 2017;26(4):312-322. doi:10.1136/bmjqs-2016-006020 PubMedGoogle ScholarCrossref
19.
Nazi  KM, Turvey  CL, Klein  DM, Hogan  TP, Woods  SSVA.  VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes.   J Am Med Inform Assoc. 2015;22(2):380-389. doi:10.1136/amiajnl-2014-003144 PubMedGoogle ScholarCrossref
20.
Chimowitz  H, Gerard  M, Fossa  A, Bourgeois  F, Bell  SK.  Empowering informal caregivers with health information: OpenNotes as a safety strategy.   Jt Comm J Qual Patient Saf. 2018;44(3):130-136. doi:10.1016/j.jcjq.2017.09.004 PubMedGoogle Scholar
21.
Bell  SK, Folcarelli  P, Fossa  A,  et al.  Tackling ambulatory safety risks through patient engagement: what 10,000 patients and families say about safety-related knowledge, behaviors, and attitudes after reading visit notes.   J Patient Saf. 2018;00(00):1-9. doi:10.1097/PTS.0000000000000494 PubMedGoogle Scholar
22.
Wright  E, Darer  J, Tang  X,  et al.  Sharing physician notes through an electronic portal is associated with improved medication adherence: quasi-experimental study.   J Med internet Res. 2015;17(10):e226. doi:10.2196/jmir.4872 PubMedGoogle Scholar
23.
Walker  J, Leveille  S, Bell  S,  et al.  OpenNotes after 7 years: patient experiences with ongoing access to their clinicians’ outpatient visit notes.   J Med internet Res. 2019;21(5):e13876. doi:10.2196/13876 PubMedGoogle Scholar
24.
DesRoches  CM, Bell  SK, Dong  Z,  et al.  Patients managing medications and reading their visit notes: a survey of OpenNotes participants.   Ann Intern Med. 2019;171(1):69-71. doi:10.7326/M18-3197 PubMedGoogle ScholarCrossref
25.
Wolff  JL, Berger  A, Clarke  D,  et al.  Patients, care partners, and shared access to the patient portal: online practices at an integrated health system.   J Am Med Inform Assoc. 2016;23(6):1150-1158. doi:10.1093/jamia/ocw025 PubMedGoogle ScholarCrossref
26.
Sarkar  U, Bates  DW.  Care partners and online patient portals.   JAMA. 2014;311(4):357-358. doi:10.1001/jama.2013.285825 PubMedGoogle ScholarCrossref
27.
Bell  SK, Folcarelli  PH, Anselmo  MK, Crotty  BH, Flier  LA, Walker  J.  Connecting patients and clinicians: the anticipated effects of Open Notes on patient safety and quality of care.   Jt Comm J Qual Patient Saf. 2015;41(8):378-384. doi:10.1016/S1553-7250(15)41049-9 PubMedGoogle Scholar
28.
Herlihy  M, Harcourt  K, Fossa  A, Folcarelli  PH, Golen  T, Bell  S.  An opportunity to engage obstetric and gynecology patients in safety and quality through shared visit notes.   Obstet Gynecol. 2019;133(1):S1-S3. doi:10.1097/01.AOG.0000559076.53573.ab Google ScholarCrossref
29.
Gillespie  A, Reader  TW.  Patient-centered insights.   Milbank Q. 2018;96(3):530-567. doi:10.1111/1468-0009.12338 PubMedGoogle ScholarCrossref
30.
Nickel  WK, Weinberger  SE, Guze  PA; Patient Partnership in Healthcare Committee of the American College of Physicians.  Principles for patient and family partnership in care: an American College of Physicians position paper.   Ann Intern Med. 2018;169(11):796-799. doi:10.7326/M18-0018 PubMedGoogle ScholarCrossref
31.
Donaldson  LJ.  The wisdom of patients and families: ignore it at our peril.   BMJ Qual Saf. 2015;24(10):603-604. doi:10.1136/bmjqs-2015-004573PubMedGoogle ScholarCrossref
32.
National Patient Safety Foundation. Safety Is Personal: Partnering with Patients and Families for the Safest Care: The National Patient Safety Foundation’s Lucian Leape Institute Report of the Roundtable on Consumer Engagement in Patient Safety. National Patient Safety Foundation; 2014. Accessed December 18, 2018. http://www.ihi.org/resources/Pages/Publications/Safety-Is-Personal-Partnering-with-Patients-and-Families-for-the-Safest-Care.aspx
33.
Balogh  EP, Miller  BT, Ball  JR, eds.  Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, The National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Quality Chasm Series. National Academy of Medicine; 2015.
34.
Khan  A, Spector  ND, Baird  JD,  et al.  Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study.   BMJ. 2018;363:k4764. doi:10.1136/bmj.k4764 PubMedGoogle ScholarCrossref
35.
Giardina  TD, Haskell  H, Menon  S,  et al.  Learning from patients’ experiences related to diagnostic errors is essential for progress in patient safety.   Health Aff (Millwood). 2018;37(11):1821-1827. doi:10.1377/hlthaff.2018.0698 PubMedGoogle ScholarCrossref
36.
Sokol-Hessner  L, Folcarelli  PH, Sands  KEF.  Emotional harm from disrespect: the neglected preventable harm.   BMJ Qual Saf. 2015;24(9):550-553. doi:10.1136/bmjqs-2015-004034 PubMedGoogle ScholarCrossref
37.
Weissman  JS, Schneider  EC, Weingart  SN,  et al.  Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not?   Ann Intern Med. 2008;149(2):100-108. doi:10.7326/0003-4819-149-2-200807150-00006 PubMedGoogle ScholarCrossref
38.
Harrison  R, Walton  M, Manias  E,  et al.  The missing evidence: a systematic review of patients’ experiences of adverse events in health care.   Int J Qual Health Care. 2015;27(6):424-442. doi:10.1093/intqhc/mzv075 PubMedGoogle ScholarCrossref
39.
Khan  A, Furtak  SL, Melvin  P, Rogers  JE, Schuster  MA, Landrigan  CP.  Parent-reported errors and adverse events in hospitalized children.   jama Pediatr. 2016;170(4):e154608. doi:10.1001/jamapediatrics.2015.4608 PubMedGoogle Scholar
40.
Khan  A, Coffey  M, Litterer  KP,  et al; the Patient and Family Centered I-PASS Study Group.  Families as partners in hospital error and adverse event surveillance.   JAMA Pediatr. 2017;171(4):372-381. doi:10.1001/jamapediatrics.2016.4812 PubMedGoogle ScholarCrossref
41.
Brady  PW, Zix  J, Brilli  R,  et al.  Developing and evaluating the success of a family activated medical emergency team: a quality improvement report.   BMJ Qual Saf. 2015;24(3):203-211. doi:10.1136/bmjqs-2014-003001 PubMedGoogle ScholarCrossref
42.
Weingart  SN, Pagovich  O, Sands  DZ,  et al.  What can hospitalized patients tell us about adverse events? learning from patient-reported incidents.   J Gen Intern Med. 2005;20(9):830-836. doi:10.1111/j.1525-1497.2005.0180.x PubMedGoogle ScholarCrossref
43.
Daniels  JP, Hunc  K, Cochrane  DD,  et al.  Identification by families of pediatric adverse events and near misses overlooked by health care providers.   CMAJ. 2012;184(1):29-34. doi:10.1503/cmaj.110393 PubMedGoogle ScholarCrossref
44.
Mazor  KM, Roblin  DW, Greene  SM,  et al.  Toward patient-centered cancer care: patient perceptions of problematic events, impact, and response.   J Clin Oncol. 2012;30(15):1784-1790. doi:10.1200/JCO.2011.38.1384 PubMedGoogle ScholarCrossref
45.
McNutt  L-A, Wu  C, Xue  X, Hafner  JP.  Estimating the relative risk in cohort studies and clinical trials of common outcomes.   Am J Epidemiol. 2003;157(10):940-943. doi:10.1093/aje/kwg074 PubMedGoogle ScholarCrossref
46.
Glaser  B, Strauss  A.  The Discovery of Grounded TheoryL Strategies for Qualitative Theory. Routledge; 1967.
47.
The Commonwealth Fund. The New York Times, Harvard T.H. Chan School of Public Health. Being Seriously Ill in America Today. 2018. Accessed February 1, 2019. https://cdn1.sph.harvard.edu/wp-content/uploads/sites/94/2018/10/CMWF-NYT-HSPH-Seriously-Ill-Poll-Report.pdf
48.
Zhou  L, Blackley  SV, Kowalski  L,  et al.  Analysis of errors in dictated clinical documents assisted by speech recognition software and professional transcriptionists.   JAMA Netw Open. 2018;1(3):e180530. doi:10.1001/jamanetworkopen.2018.0530 PubMedGoogle Scholar
49.
Southwick  FS, Cranley  NM, Hallisy  JA.  A patient-initiated voluntary online survey of adverse medical events: the perspective of 696 injured patients and families.   BMJ Qual Saf. 2015;24(10):620-629. doi:10.1136/bmjqs-2015-003980 PubMedGoogle ScholarCrossref
50.
Sokol-Hessner  L, Kane  GJ, Annas  CL,  et al.  Development of a framework to describe patient and family harm from disrespect and promote improvements in quality and safety: a scoping review.   Int J Qual Health Care. 2019;31(9):657-668. doi:10.1093/intqhc/mzy231PubMedGoogle Scholar
51.
Lang  S, Velasco Garrido  M, Heintze  C.  Patients’ views of adverse events in primary and ambulatory care: a systematic review to assess methods and the content of what patients consider to be adverse events.   BMC Fam Pract. 2016;17(6):6. doi:10.1186/s12875-016-0408-0 PubMedGoogle ScholarCrossref
52.
Singh  H, Meyer  AN, Thomas  EJ.  The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.   BMJ Qual Saf. 2014;23(9):727-731. doi:10.1136/bmjqs-2013-002627 PubMedGoogle ScholarCrossref
53.
Roy  CL, Poon  EG, Karson  AS,  et al.  Patient safety concerns arising from test results that return after hospital discharge.   Ann Intern Med. 2005;143(2):121-128. doi:10.7326/0003-4819-143-2-200507190-00011 PubMedGoogle ScholarCrossref
54.
Wahls  T, Haugen  T, Cram  P.  The continuing problem of missed test results in an integrated health system with an advanced electronic medical record.   Jt Comm J Qual Patient Saf. 2007;33(8):485-492. doi:10.1016/S1553-7250(07)33052-3 PubMedGoogle Scholar
55.
Singh  H, Thomas  EJ, Mani  S,  et al.  Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?   Arch Intern Med. 2009;169(17):1578-1586. doi:10.1001/archinternmed.2009.263 PubMedGoogle ScholarCrossref
56.
Kessels  RPC.  Patients’ memory for medical information.   J R Soc Med. 2003;96(5):219-222.PubMedGoogle Scholar
57.
Bates  DW, Singh  H.  Two decades since to err is human: an assessment of progress and emerging priorities in patient safety.   Health Aff (Millwood). 2018;37(11):1736-1743. doi:10.1377/hlthaff.2018.0738 PubMedGoogle ScholarCrossref
58.
Davis  RE, Jacklin  R, Sevdalis  N, Vincent  CA.  Patient involvement in patient safety: what factors influence patient participation and engagement?   Health Expect. 2007;10(3):259-267. doi:10.1111/j.1369-7625.2007.00450.x PubMedGoogle ScholarCrossref
59.
Fisher  KA, Smith  KM, Gallagher  TH, Huang  JC, Borton  JC, Mazor  KM.  We want to know: patient comfort speaking up about breakdowns in care and patient experience.   BMJ Qual Saf. 2019;28(3):190-197. doi:10.1136/bmjqs-2018-008159 PubMedGoogle ScholarCrossref
60.
Bell  SK, Roche  SD, Mueller  A,  et al.  Speaking up about care concerns in the ICU: patient and family experiences, attitudes and perceived barriers.   BMJ Qual Saf. 2018;27(11):928-936. doi:10.1136/bmjqs-2017-007525PubMedGoogle ScholarCrossref
61.
Weiner  SJ, Wang  S, Kelly  B, Sharma  G, Schwartz  A.  How accurate is the medical record? A comparison of the physician’s note with a concealed audio recording in unannounced standardized patient encounters.   J Am Med Inform Assoc. 2020;ocaa027. Published online April 24, 2020. doi:10.1093/jamia/ocaa027PubMedGoogle Scholar
62.
NORC. Methodology Report: 2014-2015 Nationwide CAHPS Surveys of Adults Enrolled in Medicaid between October and December 2013. NORC; 2015. https://www.medicaid.gov/medicaid/quality-of-care/downloads/performance-measurement/methodology-report.pdf
63.
Goel  MS, Brown  TL, Williams  A, Hasnain-Wynia  R, Thompson  JA, Baker  DW.  Disparities in enrollment and use of an electronic patient portal.   J Gen Intern Med. 2011;26(10):1112-1116. doi:10.1007/s11606-011-1728-3 PubMedGoogle ScholarCrossref
64.
Etchegaray  JM, Ottosen  MJ, Aigbe  A,  et al.  Patients as partners in learning from unexpected events.   Health Serv Res. 2016;51(6)(suppl 3):2600-2614. doi:10.1111/1475-6773.12593 PubMedGoogle ScholarCrossref
65.
Gillespie  A, Reader  TW.  The Healthcare Complaints Analysis Tool: development and reliability testing of a method for service monitoring and organisational learning.   BMJ Qual Saf. 2016;25(12):937-946. doi:10.1136/bmjqs-2015-004596 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Health Informatics
    June 9, 2020

    Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes

    Author Affiliations
    • 1Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
    • 2Harvard Medical School, Boston, Massachusetts
    • 3Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles
    • 4Department of Epidemiology, University of Michigan, Ann Arbor
    • 5Department of Nursing, College of Nursing and Health Sciences, University of Massachusetts, Boston
    • 6Department of Medicine, University of Washington School of Medicine, Seattle
    • 7Steele Institute for Health Innovation, Geisinger, Danville, Pennsylvania
    JAMA Netw Open. 2020;3(6):e205867. doi:10.1001/jamanetworkopen.2020.5867
    Key Points español 中文 (chinese)

    Question  How often do patients who read open ambulatory visit notes perceive mistakes, and what types of mistakes do they report?

    Findings  In this survey study of 136 815 patients, 29 656 provided a response, and 1 in 5 patients who read a note reported finding a mistake and 40% perceived the mistake as serious. Among patient-reported very serious errors, the most common characterizations were mistakes in diagnoses, medical history, medications, physical examination, test results, notes on the wrong patient, and sidedness.

    Meaning  As health information transparency increases, patients may perceive important errors in their visit notes, and inviting them to report mistakes that they believe are very serious may be associated with improved record accuracy and patient engagement in safety.

    Abstract

    Importance  As health information transparency increases, patients more often seek their health data. More than 44 million patients in the US can now readily access their ambulatory visit notes online, and the practice is increasing abroad. Few studies have assessed documentation errors that patients identify in their notes and how these may inform patient engagement and safety strategies.

    Objective  To assess the frequency and types of errors identified by patients who read open ambulatory visit notes.

    Design, Setting, and Participants  In this survey study, a total of 136 815 patients at 3 US health care organizations with open notes, including 79 academic and community ambulatory care practices, received invitations to an online survey from June 5 to October 20, 2017. Patients who had at least 1 ambulatory note and had logged onto the portal at least once in the past 12 months were included. Data analysis was performed from July 3, 2018, to April 27, 2020.

    Exposures  Access to ambulatory care open notes through patient portals for up to 7 years (2010-2017).

    Main Outcomes and Measures  Proportion of patients reporting a mistake and how serious they perceived the mistake to be, factors associated with finding errors characterized by patients as serious, and categories of patient-reported errors.

    Results  Of 136 815 patients who received survey invitations, 29 656 (21.7%) responded and 22 889 patients (mean [SD] age, 55.16 [15.96] years; 14 447 [63.1%] female; 18 301 [80.0%] white) read 1 or more notes in the past 12 months and completed error questions. Of these patients, 4830 (21.1%) reported a perceived mistake and 2043 (42.3%) reported that the mistake was serious (somewhat serious: 1563 [32.4%]; very serious: 480 [9.9%]). In multivariable analysis, female patients (relative risk [RR], 1.79; 95% CI, 1.72-1.85), more educated patients (RR, 1.38; 95% CI, 1.29-1.48), sicker patients (RR, 1.89; 95% CI, 1.84-1.94), those aged 45 to 64 years (RR, 2.23; 95% CI, 2.06-2.42), those 65 years or older (RR, 2.00; 95% CI, 1.73-2.32), and those who read more than 1 note (2-3 notes: RR, 1.82; 95% CI, 1.34-2.47; ≥4 notes: RR, 3.09; 95% CI, 2.02-4.73) were more likely to report a mistake that they found to be serious compared with their reference groups. After categorization of patient-reported very serious mistakes, those specifically mentioning the word diagnosis or describing a specific error in current or past diagnoses were most common (98 of 356 [27.5%]), followed by inaccurate medical history (85 of 356 [23.9%]), medications or allergies (50 of 356 [14.0%]), and tests, procedures, or results (30 of 356 [8.4%]). A total of 23 (6.5%) reflected notes reportedly written on the wrong patient. Of 433 very serious errors, 255 (58.9%) included at least 1 perceived error potentially associated with the diagnostic process (eg, history, physical examination, tests, referrals, and communication).

    Conclusions and Relevance  In this study, patients who read ambulatory notes online perceived mistakes, a substantial proportion of which they found to be serious. Older and sicker patients were twice as likely to report a serious error compared with younger and healthier patients, indicating important safety and quality implications. Sharing notes with patients may help engage them to improve record accuracy and health care safety together with practitioners.

    Introduction

    Errors in electronic health records (EHRs) are common.1,2 At least half of EHRs may contain an error, many related to medications.2-8 Overburdened practitioners may import inaccurate medication lists, propagate other erroneous information electronically by copying and pasting older parts of the record, or enter erroneous examination findings.2,8,9 EHRs may also lack critical information (errors of omission) because of limited interoperability among health care sites.10 Among primary care physicians sharing notes with patients, 26% anticipated that patients would find nontrivial errors.11 Despite these known problems, systems for checking the accuracy of notes are almost nonexistent. As practitioners integrate EHR data into decision-making, such errors could therefore lead to medication errors, wasteful duplication, unnecessary or incorrect treatment, and delayed diagnoses.7,12-15

    Cultural shifts toward transparency along with nationwide growth of secure electronic patient portals may be associated with increases in patients accessing their medical records, including visit notes (open notes). Beginning in 2010, with 20 000 patients participating in a demonstration research project, open notes are now available to more than 44 million patients in 200 health care centers in every state in the US, and this practice is extending globally (eAppendix in the Supplement).16,17 Because of the 21st Century Cures Act to ensure patient access to electronic health information along with subsequent actions by the Office of the National Coordinator for Health Information Technology addressing patients’ legal right to access their notes through patient portals or third-party apps, this number is likely to increase.

    Reading open notes may be associated with enhanced patient engagement and with improved patient safety and care quality.11,17-24 Patients reported that they understand notes and that reading notes helps them remember next steps (such as tests and referrals).18,21 They also reported that note reading enables timely follow-up of results and supports family or friend care partners with information.18,20,21,25,26 In addition, sharing notes with patients creates a new mechanism for patients to identify documentation errors.27,28

    Patients and families hold unique knowledge about themselves and their care, and their reports have potential for improving individual and organizational safety.29-34 They experience aspects of care not seen by practitioners, such as events that transpire between visits or care transitions.18,29,35,36 Patients and families may also detect breakdowns in care, including some events missed by practitioners,18,29,37-44 and aggregate patient reports can identify organizational strengths and weaknesses.29 We aimed to assess the proportion of patients who perceive mistakes in ambulatory notes and how serious they perceive the mistakes to be, patient factors associated with finding mistakes that they considered somewhat or very serious, and the types of mistakes patients describe as very serious.

    Methods
    Participants

    For this study, from June 5 to October 20, 2017, we surveyed patients in 3 health systems sharing notes with patients for up to 7 years17: Beth Israel Deaconess Medical Center, an academic medical center in Massachusetts and 6 affiliated sites; Geisinger, an integrated health system in rural Pennsylvania and southern New Jersey, including 7 hospitals and 53 community practices; and University of Washington Medicine, an urban health system in Washington with 3 hospitals and 9 freestanding practices.23 Each of these organizations participated in the initial open notes study in 2010. By 2014, they had expanded open notes access to almost every ambulatory medical and surgical practice and to all types of practitioners (eAppendix in the Supplement).23 Data analysis was performed from July 3, 2018, to April 27, 2020. The institutional review boards at Beth Israel Deaconess Medical Center, University of Washington, and Geisinger approved the study protocols at their sites. Completing the survey was voluntary, and all answers were confidential; therefore, the institutional review boards determined that informed consent was implied by survey participation. This study followed the American Association for Public Opinion Research (AAPOR) reporting guideline.

    Participants were 18 years or older, had at least 1 ambulatory note, and had logged onto the portal at least once in the past 12 months. We did not include patients who had been invited to participate in other open notes research projects in the prior 12 months or who had previously opted out of all research, based on organizational limits on recruiting patients.

    Survey

    The survey was adapted from the original OpenNotes questionnaire.17 To help focus patients on their notes, the survey included a screenshot of the location of notes on each organization’s patient portal. The survey included 4 questions about mistakes: “How confident do you feel in your ability to find mistakes in your visit notes?” (4-point ordinal responses: not at all to very confident) and “Have you ever found anything in your visit notes you thought was a mistake (not counting misspellings or typographical errors)?” (responses: no, yes, or don’t know/not sure). Those who answered yes were asked, “How important was the most serious mistake you found?” (responses: not at all serious, somewhat serious, or very serious). At 2 sites, those who described the mistake as somewhat or very serious were asked, “Please describe the most serious mistake” (free text). Respondents identified themselves as Spanish, Hispanic, Latino ethnicity (categorized as Hispanic) or not and as white, black or African American (categorized as black), American Indian or Pacific Native, Asian, Native Hawaiian or Pacific Islander, or other. Additional survey details have been published, and the questionnaire is available on request.23,24

    Statistical Analysis

    We defined note readers as those with concordant self-reported note reading and portal tracking data. We used administrative data to compare respondent and nonrespondent characteristics. We used descriptive statistics for showing the number and proportions of note readers who reported mistakes in their notes and how important they thought the mistakes were. In this article, the terms mistake and error are used interchangeably, consonant with the variability of the terms used in the literature. All patient-reported errors reflect perceived errors.

    We examined respondents’ self-reported sociodemographic and health characteristics and note reading experiences both overall and according to whether they found a mistake. For reading experience, we examined both duration (weeks, months, or years) and number of notes read (1 to ≥4). We calculated frequencies and proportions for all categorical and ordinal variables.

    We modeled the probability that a patient found a somewhat serious or very serious mistake using a generalized estimating equation with a log link and binomial error.45 Independent variables in our model were race, ethnicity, self-reported health, educational attainment, age, sex, number of notes read, length of time reading notes, and primary language spoken at home, reflecting the hypotheses that white, more educated, and sicker patients and those who read more notes or speak English primarily at home would be more likely to report a mistake.

    We used a working exchangeable correlation structure45 (specifying that all observations within a given cluster have equal variance and are equally correlated within each cluster) in our model to account for clustering within study site and to calculate the relative risks (RRs) and 95% CIs and checked for collinearity across covariates. All analyses were conducted using SAS statistical software, version 9.4 (SAS Institute Inc).

    Drawing on a classification system previously used to analyze patient-reported note errors,18 we analyzed a subset of all free-text descriptions, focusing on the mistakes reported by patients as very serious. We reviewed all patient responses and added codes to the classification system to reflect new emerging categories in the data46 and defined each code in detail (eTable in the Supplement). Three coders (including S.K.B., K.H.) then independently reviewed a subset of the data, applying the revised codebook, with further category refinement after consensus discussion. One physician (S.K.B.) with experience in patient-reported errors then applied the final codebook to the entire data set, assigning up to 2 mistake types for each free-text description. We excluded comments that were irrelevant or indecipherable. In cases in which the mistake type was not immediately evident, we used an adjudication process with a second reviewer (C.M.D). Both reviewers discussed the text until they reached consensus on the assigned categories. Coding was done in REDCap (Research Electronic Data Capture).

    Results
    Respondents and Patient-Reported Mistakes

    Of 136 815 patients who received survey invitations, 29 656 (21.7%) responded and 22 889 patients (mean [SD] age, 55.16 [15.96] years; 14 447 [63.1%] female; 18 301 [80.0%] white) read 1 or more notes in the past 12 months and completed error questions, comprising the study population (Figure). A total of 16 810 patients (73.4%) reported reading notes for at least 1 year, and 11 391 (49.8%) reported reading 4 or more notes. Among all patients, 18 437 (80.5%) reported that they were confident or very confident in their ability to find mistakes whether or not they reported a mistake in their notes. In total, 4830 of 22 889 note readers (21.1%) perceived a mistake in their notes. Univariate associations between patient characteristics and reporting a perceived error are given in Table 1.

    Characteristics of Patients Who Found Somewhat Serious or Very Serious Mistakes in Notes

    Among 4830 patients who perceived mistakes in notes, 2043 (42.3%) described them as serious (1563 [32.4%] were somewhat serious and 480 [9.9%] were very serious). In multivariable analysis, female patients (RR, 1.79; 95% CI, 1.72-1.85), more educated patients (RR, 1.38; 95% CI, 1.29-1.48), sicker patients (RR, 1.89; 95% CI, 1.84-1.94), those aged 45 to 64 years (RR, 2.23; 95% CI, 2.06-2.42), those 65 years or older (RR, 2.00; 95% CI, 1.73-2.32), and those who read more than 1 note (2-3 notes: RR, 1.82; 95% CI, 1.34-2.47; ≥4 notes: RR, 3.09; 95% CI, 2.02-4.73) were more likely to report a serious mistake compared with their reference groups (Table 2).

    Asian patients were less likely than white patients to report finding a serious mistake (RR, 0.63; 95% CI, 0.57-0.70), but no significant differences were found between patients identifying as black vs white or between Hispanic vs non-Hispanic patients. Patients reporting multiple races were more likely to report serious errors (RR, 1.28; 95% CI, 1.19-1.37). Patients who reported speaking multiple languages (RR, 0.86; 95% CI, 0.85-0.87) or a language other than English or Spanish as their primary language at home (RR, 0.6; 95% CI, 0.57-0.63) were less likely to report a serious error, but no meaningful differences were found between patients who primarily spoke English or Spanish at home.

    Categories of Mistakes Described by Patients as Very Serious

    Among 480 patients who reported finding a very serious mistake, 463 (96.5%) were respondents in the 2 organizations where patients were asked to describe the event, and 433 (93.5%) provided a free-text description. Seventy-seven comments did not contain enough information to be classified (ie, “It was already discussed with the provider”) and were excluded.

    Of the remaining 356 patient reports, the most common category of mistakes reported were those specifically mentioning the word diagnosis or describing a perceived error in current or past diagnoses (98 of 356 [27.5%]) (Table 3). Other very serious patient-reported mistakes included inaccurate description of medical history (85 of 356 [23.9%]); medications or allergies (50 of 356 [14.0%]); tests, procedures, or results (30 of 356 [8.4%]); and perceived errors pertaining to the physical examination, including elements of the examination that, according to the patient, were documented but not done (24 of 356 [6.7%]). In addition, 24 patients (6.7%) described failed communication (issues that the practitioner documented as said or done but that the patient perceived did not happen at the visit), such as informed consent or counseling on specific topics. A total of 23 patients (6.5%) reported reading notes on the wrong patient, and 12 (3.4%) detected errors in sidedness (left vs right).

    We examined mistakes that could be associated with the diagnostic process as outlined in the National Academy of Medicine conceptual model.33 These mistakes included the following study categories: diagnosis specifically mentioned; medical history; tests, procedures, or results; physical examination; family history; failed communication; wrong patient; and sidedness. Using this broader view, of all patient-reported mistakes, 255 of 433 (58.9%) described at least 1 perceived error that could potentially affect the diagnostic process. A summary of the more common types of patient-reported errors follows.

    Diagnosis Specifically Mentioned

    Patient-reported diagnosis-related mistakes included specific mention of the word diagnosis or perceived errors in specific medical diagnoses, including conditions that patients did not have, diagnoses that patients had and thought were relevant but were not recorded, problems or delays in the diagnostic process, or inaccuracy of existing diagnoses (Table 3). Examples included erroneous documentation of diabetes, cancer, and, in 1 instance, HIV infection. One patient wrote, “A history of DCIS [ductal carcinoma in situ] was written in the note as 'disseminated cancer.' I received condolences from another physician.” Another reported a potentially preventable diagnosis of osteoporosis established several years after a missed finding of osteopenia. Several patients reported delayed diagnoses, such as “score added up incorrectly… on depression survey, showing mild depression when really showed severe, mistake found by me, then by psychiatrist reviewing… about a year later.” In some cases, patients noted that diagnoses were made but not communicated to them, such as atrial fibrillation or a “heart attack.”

    Medical History

    Patients frequently described mistakes in their medical history, such as symptoms: “…all marked as negative when in fact... I indicated I have chest pain, tightness, palpitations.” Patients also described situations in which practitioners missed the most important reason for the visit. In 1 example, a pianist reported the physician noted no hand pain when that was the reason for the visit (Table 3). Patients were surprised to find documentation of responses to questions that they perceived were never asked, such as, “My cardiologist repeatedly says that I 'deny' symptoms (such as shortness of breath, etc) that he never asked me about and that I never denied having.”

    Some patients noted mistakes in dates or types of operations, including documentation of operations they reported they never had (ie, gall bladder removal, gastric bypass, or hysterectomy). They also reported conflicting information among practitioner notes or between one part of a particular note and another.

    Medications

    In this category, patients described prescription medications that appeared active but that the patient was no longer taking, new prescription medications that the patient was taking but that were missing, and wrong dosages (Table 3). Occasionally, patients reported finding medication not intended for them. There were also many reported errors related to medication allergies, including omission of severe or anaphylactic allergic reactions. Some patients detected discrepancies in medication documentation within the note, with dosages correct in one place but incorrect in another.

    Tests, Procedures, and Results

    Patients identified that some practitioners reported the wrong test result in the note and others who were not aware that more recent results or reports existed. For example, “The provider put the wrong CD4 cell count in my chart. She states 399, however lab results show my CD4 at 219.” Patients also reported mistakes in radiology results or practitioner summaries of radiology reports that made it difficult to determine whether there was clinical improvement or deterioration. In 1 example, different units of measurement resulted in ambiguity: “MRI [magnetic resonance imaging] reads dimensions now 5 × 3 cm compared to prior 5 × 4 mm.” Another commented, “Pathology report summary stated I had 2 positive lymph nodes but [the] detailed report stated 3 positive lymph nodes. That changes the staging and the treatment options. My physician had only read the summary and didn't realize I had 3 positive lymph nodes.” In this case, the patient reportedly notified the physician of the additional positive node, thereby changing the treatment plan because of the wider spread of the cancer. Patients also reported errors of omission, such as a missed lesion in the liver.

    Other Perceived Errors

    Other errors, reported by 53 participants (14.9%), most commonly reflected patients’ perception of something important that was missing from the note (such as active health issues, delivery of vaccines, or recent test results), errors stemming from copy and paste of prior electronic notes, and billing mistakes, such as erroneous codes that implied conditions the patient reportedly did not have. A few patients mentioned errors related to mental health or substance abuse. Several patients reported errors attributable to EHR glitches, such as missing medications after EHR vendor changes, attribution of all tests or vaccination dates to the patient’s birthdate, or a single date for all treatments or operations that reflected the patient’s date of transfer to the organization.

    Patient Experience With Perceived Mistakes and Attempts to Correct Them

    Some patients reported rapid resolution after notifying the practitioner about the perceived error, and several expressed appreciation: “Seeing the notes was very useful ... because it led to not taking an erroneous action... which would otherwise never have been fixed, had the notes not been shared with me.” However, in other instances, patients described experiences of disrespect, such as being derided or ignored (Table 3). One patient commented, “There is a notation that I have 'missed appointments'... despite me correcting her that the clinic actually cancelled on me multiple times in a row, it is still written up in a way that has me appear noncompliant.” Some patients also commented that perceived errors led to emotional or psychological distress, delayed diagnosis or treatment, or lost days at work.

    Several patients expressed exhaustion, frustration, and sometimes despair in trying to correct perceived errors: “…repeatedly attempted to fix this, STILL wrong” and “It is almost impossible to change incorrect information [about surgical candidacy]. It took months to do so.” Some were informed that the error could not be corrected or received no response after pointing out perceived errors, such as, “Notes claim … 'BP [blood pressure] in clinic today is 86/56 but she denies any symptoms of lightheadedness or dizziness.' That was inaccurate, I told the nurse who took my vitals I felt faint, lightheaded and weak…I did however call and let the cardiology [department] know about the incorrect report. Unfortunately I don't believe anything was done about it. I never heard back from anyone and the visit note... was never corrected.”

    Several patients attributed communication errors, especially those pertaining to events they thought did not occur at the visit, to misunderstanding or misrepresentation; others found the practitioner’s account disingenuous. In a few instances, patients reported seeking a new health care practitioner, especially if their attempts to correct errors were ignored: “This prompted me to find a new provider. Her attitude was that she was not incorrect.”

    Discussion

    Reports from more than 22 000 patients who read their practitioners’ notes at 3 US health care centers suggest that patients may play an important role in identifying errors in their records. More than 1 in 5 patients perceived mistakes in their notes, judging more than 40% of them as serious. To our knowledge, no other large-scale, cross-sectional surveys have assessed the frequency of patient-reported errors in ambulatory notes, and these data may help build on findings in smaller pilot programs.18,28 Older patients and those with poorer health—patients for whom consequences of errors may be most profound—were approximately twice as likely as younger and healthier patients to identify errors they considered serious, suggesting that note sharing may have particularly important safety implications for those with a heavier burden of illness. In addition, we received reports of mistakes that patients perceived as very serious, suggesting that a feedback mechanism for such findings may be useful for engaging patients in safety.

    Despite patients’ rights to review their medical records through the Health Insurance Portability and Accountability Act, systematic checks on the content of notes has been almost absent from clinical documentation. Patient-reported mistakes may help prevent medication errors, diagnostic and treatment delay, and duplicated diagnostic tests and procedures, which occur for approximately 25% to 30% of seriously ill patients.47 Patient-reported mistakes in this study surpassed the previously reported error rates of 7% in speech-recognition clinical documents,48 suggesting that patients can detect mistakes beyond transcription errors. Lack of routine review of notes by patients may be a missed opportunity not only for EHR accuracy but also for organizational learning.29,35,49-51

    Experts estimate that 12 million Americans experience diagnostic error annually,52 and diagnostic errors represent the most frequent category of paid malpractice claims.33 Patients who read notes can help ensure that the physician correctly heard and recorded the patient’s symptoms and may also identify mistakes associated with tests and results—known vulnerabilities in some practitioner evaluations.53-55 Shared notes may also help practitioners to communicate their thought process about a diagnosis.56 As safety leaders seek ways to engage patients and families actively in the diagnostic process, sharing clinical notes may be a scalable first step, especially because patient-reported errors related to diagnosis, medical history, physical examination, and tests and results were among the most common patient-reported errors in this study.

    Safety experts emphasize learning from patient reports.57 However, realizing the potential for shared notes to enhance safety will require broad outreach and education for patients of all ages, races/ethnicities, and educational and health literacy levels. Practitioner support for patient feedback about errors is also important.58,59 As reported in this study and elsewhere,60 lack of meaningful change after reporting perceived mistakes may be associated with patient frustration and hesitancy to further engage with practitioners. Organizations will need systematic mechanisms for triaging and responding to patient-reported errors, particularly as EHR transparency increases and more patients access their records.

    Some patient-reported errors may reflect disagreements between patients and practitioners or may not be errors. However, similar to prior reports,18,28,40 the types of very serious mistakes described by patients generally appeared to have relevant clinical implications (Table 3). In addition, in a study61 that compared physician notes with audio recordings of unannounced standardized patients, some findings recorded in physician notes did not take place at the visit, similar to patient reports in this study (“Failed communication: did not happen at visit”). Such errors may be associated with use of templated notes if all elements that were not done during the visit (ie, full review of systems, complete physical examination, and time spent counseling) are not accurately edited.

    Some patients will identify errors that they themselves do not think are important. Focusing attention instead on errors that they think are most important may be associated with an improved patient experience, even if practitioners believe some of those reports are not errors. However, some concerns that patients do not deem serious may have important health consequences, underscoring the need for further research, patient education, and report triage. As a first step, soliciting patient-reported very serious errors may complement existing error surveillance mechanisms and raise awareness about (at least some) problems that may otherwise go undetected.

    Limitations

    This study has limitations. Although similar to other online patient surveys,62 the response rate was low at 21.7%. Some patients do not have access to internet or data plans, patient portals, or notes, and results may not reflect the views of patients who do not have access to or do not read their notes online. True patient-reported EHR error rates could therefore be higher or lower than reported here. In addition, although the study involved 3 US health care centers, the patient population overall was predominantly white and educated, characteristics associated with higher likelihood of finding mistakes. Further research with more diverse patient populations is needed.63

    The data are further limited by self-report and errors unverified by EHR review or patient outcomes, processes beyond our scope. Moreover, patients may have described errors as serious that their practitioners might have categorized differently. However, many of the errors that patients cited were compelling, and the proportion of patients reporting mistakes was similar to reporting tool findings and national surveys of patient-reported errors in EHRs (21% in each).1,18 One benefit of patient reports is that they may bring new information about breakdowns in care that are not in the record because they may be missed by practitioners and therefore would not be documented in notes.29,37,40,64,65 As a result, not all patient-reported errors can be verified by EHR review. Future research should examine associations between patient-reported errors and safety outcomes, including potential prevention of diagnostic errors.

    Conclusions

    The findings suggest that inviting patients to report perceived mistakes in shared visit notes, particularly those that patients believe are very serious, may be associated with improved record accuracy and patient engagement in diagnosis. Developing efficient mechanisms to respond to such reports appears to be important. At a time when patient demand for data is increasing along with federal support for providing patients easy access to health information, shared notes may help enlist patients, families, and practitioners in pursuing greater patient safety collaboratively.

    Back to top
    Article Information

    Accepted for Publication: March 14, 2020.

    Published: June 9, 2020. doi:10.1001/jamanetworkopen.2020.5867

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Bell SK et al. JAMA Network Open.

    Corresponding Author: Sigall K. Bell, MD, Division of General Medicine, Beth Israel Deaconess Medical Center, 133 Brookline Ave, Annex 2, Room 2200, Boston, MA 02215 (sbell1@bidmc.harvard.edu).

    Author Contributions: Mr Fossa had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Bell, Delbanco, Fossa, Leveille, Payne, Walker, DesRoches.

    Acquisition, analysis, or interpretation of data: Delbanco, Elmore, Fitzgerald, Fossa, Harcourt, Leveille, Payne, Stametz, Walker, DesRoches.

    Drafting of the manuscript: Bell, Delbanco, Elmore, Fitzgerald, Fossa, Harcourt, DesRoches.

    Critical revision of the manuscript for important intellectual content: Delbanco, Fossa, Leveille, Payne, Stametz, Walker, DesRoches.

    Statistical analysis: Bell, Fitzgerald, Fossa, DesRoches.

    Obtained funding: Delbanco, Walker, DesRoches.

    Administrative, technical, or material support: Elmore, Fitzgerald, Fossa, Harcourt, Payne, Stametz.

    Supervision: Fitzgerald, Leveille, Payne, DesRoches.

    Conflict of Interest Disclosures: All authors reported receiving grants from the Robert Wood Johnson Foundation, the Gordon and Betty Moore Foundation, the Peterson Center on Healthcare, and the Cambia Health Foundation during the conduct of the study. Dr Bell reported receiving royalties from Clio for a published book outside the scope of work; and receiving a grant from the Agency for Healthcare Research and Quality for related but separate work. No other disclosures were reported.

    Additional Contributions: Topister Bonyo, MPH, Massachusetts Eye and Ear, Boston, Massachusetts, contributed to the development of the patient-reported error categories and Zhiyong Dong, MS, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, provided data analysis review. They were compensated for their work.

    References
    1.
    Henry J Kaiser Family Foundation. KFF health tracking poll. 2019. Accessed May 3, 2019. http://files.kff.org/attachment/Topline-KFF-Health-Tracking-Poll-January-2019
    2.
    Koppel  R. Patient safety and health information technology: learning from our mistakes perspective. AHRQ Patient Saf Network. 2012;(4):5-8. Accessed April 18, 2020. https://psnet.ahrq.gov/perspective/patient-safety-and-health-information-technology-learning-our-mistakes
    3.
    Weir  CR, Hurdle  JF, Felgar  MA, Hoffman  JM, Roth  B, Nebeker  JR.  Direct text entry in electronic progress notes. An evaluation of input errors.   Methods Inf Med. 2003;42(1):61-67. doi:10.1055/s-0038-1634210PubMedGoogle ScholarCrossref
    4.
    Suresh  G.  Don’t believe everything you read in the patient’s chart.   Pediatrics. 2003;111(5 pt 1):1108-1109. doi:10.1542/peds.111.5.1108 PubMedGoogle ScholarCrossref
    5.
    Staroselsky  M, Volk  LA, Tsurikova  R,  et al.  An effort to improve electronic health record medication list accuracy between visits: patients’ and physicians’ response.   Med Inform (Lond). 2008;77(3):153-160. doi:10.1016/j.ijmedinf.2007.03.001PubMedGoogle ScholarCrossref
    6.
    Kaboli  PJ, McClimon  BJ, Hoth  AB, Barnett  MJ.  Assessing the accuracy of computerized medication histories.   Am J Manag Care. 2004;10(11 pt 2):872-877.PubMedGoogle Scholar
    7.
    Graber  ML, Byrne  C, Johnston  D.  The impact of electronic health records on diagnosis.   Diagnosis (Berl). 2017;4(4):211-223. doi:10.1515/dx-2017-0012 PubMedGoogle ScholarCrossref
    8.
    Yadav  S, Kazanji  N, K C  N,  et al.  Comparison of accuracy of physical examination findings in initial progress notes between paper charts and a newly implemented electronic health record.   J Am Med Inform Assoc. 2017;24(1):140-144. doi:10.1093/jamia/ocw067 PubMedGoogle ScholarCrossref
    9.
    Payne  TH, Alonso  WD, Markiel  JA,  et al.  Using voice to create inpatient progress notes: effects on note timeliness, quality, and physician satisfaction.   JAMIA Open. 2018;1(2):218-226. doi:10.1093/jamiaopen/ooy036 PubMedGoogle ScholarCrossref
    10.
    Secretary of Health and Human Services. Report to Congress on Health IT Progress: Examining the HITECH Era and the Future of Health IT. Vol 3001. Dept of Health and Human Services; 2016. https://dashboard.healthit.gov/report-to-congress/2016-report-congress-examining-hitech-era-future-health-information-technology.php
    11.
    Bell  SK, Mejilla  R, Anselmo  M,  et al.  When doctors share visit notes with patients: a study of patient and doctor perceptions of documentation errors, safety opportunities and the patient-doctor relationship.   BMJ Qual Saf. 2017;26(4):262-270. doi:10.1136/bmjqs-2015-004697 PubMedGoogle ScholarCrossref
    12.
    Walker  JM, Carayon  P, Leveson  N,  et al.  EHR safety: the way forward to safe and effective systems.   J Am Med Inform Assoc. 2008;15(3):272-277. doi:10.1197/jamia.M2618 PubMedGoogle ScholarCrossref
    13.
    Colicchio  TK, Cimino  JJ, Del Fiol  G.  Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the post-meaningful use era.   J Med internet Res. 2019;21(6):e13313. doi:10.2196/13313 PubMedGoogle Scholar
    14.
    McDonald  CJ.  Computerization can create safety hazards: a bar-coding near miss.   Ann Intern Med. 2006;144(7):510-516. doi:10.7326/0003-4819-144-7-200604040-00010 PubMedGoogle ScholarCrossref
    15.
    Koppel  R, Metlay  JP, Cohen  A,  et al.  Role of computerized physician order entry systems in facilitating medication errors.   JAMA. 2005;293(10):1197-1203. doi:10.1001/jama.293.10.1197 PubMedGoogle ScholarCrossref
    16.
    OpenNotes. OpenNotes Map. Accessed April 27, 2020. https://www.opennotes.org/join/map/
    17.
    Delbanco  T, Walker  J, Bell  SK,  et al.  Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead.   Ann Intern Med. 2012;157(7):461-470. doi:10.7326/0003-4819-157-7-201210020-00002 PubMedGoogle ScholarCrossref
    18.
    Bell  SK, Gerard  M, Fossa  A,  et al.  A patient feedback reporting tool for OpenNotes: implications for patient-clinician safety and quality partnerships.   BMJ Qual Saf. 2017;26(4):312-322. doi:10.1136/bmjqs-2016-006020 PubMedGoogle ScholarCrossref
    19.
    Nazi  KM, Turvey  CL, Klein  DM, Hogan  TP, Woods  SSVA.  VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes.   J Am Med Inform Assoc. 2015;22(2):380-389. doi:10.1136/amiajnl-2014-003144 PubMedGoogle ScholarCrossref
    20.
    Chimowitz  H, Gerard  M, Fossa  A, Bourgeois  F, Bell  SK.  Empowering informal caregivers with health information: OpenNotes as a safety strategy.   Jt Comm J Qual Patient Saf. 2018;44(3):130-136. doi:10.1016/j.jcjq.2017.09.004 PubMedGoogle Scholar
    21.
    Bell  SK, Folcarelli  P, Fossa  A,  et al.  Tackling ambulatory safety risks through patient engagement: what 10,000 patients and families say about safety-related knowledge, behaviors, and attitudes after reading visit notes.   J Patient Saf. 2018;00(00):1-9. doi:10.1097/PTS.0000000000000494 PubMedGoogle Scholar
    22.
    Wright  E, Darer  J, Tang  X,  et al.  Sharing physician notes through an electronic portal is associated with improved medication adherence: quasi-experimental study.   J Med internet Res. 2015;17(10):e226. doi:10.2196/jmir.4872 PubMedGoogle Scholar
    23.
    Walker  J, Leveille  S, Bell  S,  et al.  OpenNotes after 7 years: patient experiences with ongoing access to their clinicians’ outpatient visit notes.   J Med internet Res. 2019;21(5):e13876. doi:10.2196/13876 PubMedGoogle Scholar
    24.
    DesRoches  CM, Bell  SK, Dong  Z,  et al.  Patients managing medications and reading their visit notes: a survey of OpenNotes participants.   Ann Intern Med. 2019;171(1):69-71. doi:10.7326/M18-3197 PubMedGoogle ScholarCrossref
    25.
    Wolff  JL, Berger  A, Clarke  D,  et al.  Patients, care partners, and shared access to the patient portal: online practices at an integrated health system.   J Am Med Inform Assoc. 2016;23(6):1150-1158. doi:10.1093/jamia/ocw025 PubMedGoogle ScholarCrossref
    26.
    Sarkar  U, Bates  DW.  Care partners and online patient portals.   JAMA. 2014;311(4):357-358. doi:10.1001/jama.2013.285825 PubMedGoogle ScholarCrossref
    27.
    Bell  SK, Folcarelli  PH, Anselmo  MK, Crotty  BH, Flier  LA, Walker  J.  Connecting patients and clinicians: the anticipated effects of Open Notes on patient safety and quality of care.   Jt Comm J Qual Patient Saf. 2015;41(8):378-384. doi:10.1016/S1553-7250(15)41049-9 PubMedGoogle Scholar
    28.
    Herlihy  M, Harcourt  K, Fossa  A, Folcarelli  PH, Golen  T, Bell  S.  An opportunity to engage obstetric and gynecology patients in safety and quality through shared visit notes.   Obstet Gynecol. 2019;133(1):S1-S3. doi:10.1097/01.AOG.0000559076.53573.ab Google ScholarCrossref
    29.
    Gillespie  A, Reader  TW.  Patient-centered insights.   Milbank Q. 2018;96(3):530-567. doi:10.1111/1468-0009.12338 PubMedGoogle ScholarCrossref
    30.
    Nickel  WK, Weinberger  SE, Guze  PA; Patient Partnership in Healthcare Committee of the American College of Physicians.  Principles for patient and family partnership in care: an American College of Physicians position paper.   Ann Intern Med. 2018;169(11):796-799. doi:10.7326/M18-0018 PubMedGoogle ScholarCrossref
    31.
    Donaldson  LJ.  The wisdom of patients and families: ignore it at our peril.   BMJ Qual Saf. 2015;24(10):603-604. doi:10.1136/bmjqs-2015-004573PubMedGoogle ScholarCrossref
    32.
    National Patient Safety Foundation. Safety Is Personal: Partnering with Patients and Families for the Safest Care: The National Patient Safety Foundation’s Lucian Leape Institute Report of the Roundtable on Consumer Engagement in Patient Safety. National Patient Safety Foundation; 2014. Accessed December 18, 2018. http://www.ihi.org/resources/Pages/Publications/Safety-Is-Personal-Partnering-with-Patients-and-Families-for-the-Safest-Care.aspx
    33.
    Balogh  EP, Miller  BT, Ball  JR, eds.  Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, The National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Quality Chasm Series. National Academy of Medicine; 2015.
    34.
    Khan  A, Spector  ND, Baird  JD,  et al.  Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study.   BMJ. 2018;363:k4764. doi:10.1136/bmj.k4764 PubMedGoogle ScholarCrossref
    35.
    Giardina  TD, Haskell  H, Menon  S,  et al.  Learning from patients’ experiences related to diagnostic errors is essential for progress in patient safety.   Health Aff (Millwood). 2018;37(11):1821-1827. doi:10.1377/hlthaff.2018.0698 PubMedGoogle ScholarCrossref
    36.
    Sokol-Hessner  L, Folcarelli  PH, Sands  KEF.  Emotional harm from disrespect: the neglected preventable harm.   BMJ Qual Saf. 2015;24(9):550-553. doi:10.1136/bmjqs-2015-004034 PubMedGoogle ScholarCrossref
    37.
    Weissman  JS, Schneider  EC, Weingart  SN,  et al.  Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not?   Ann Intern Med. 2008;149(2):100-108. doi:10.7326/0003-4819-149-2-200807150-00006 PubMedGoogle ScholarCrossref
    38.
    Harrison  R, Walton  M, Manias  E,  et al.  The missing evidence: a systematic review of patients’ experiences of adverse events in health care.   Int J Qual Health Care. 2015;27(6):424-442. doi:10.1093/intqhc/mzv075 PubMedGoogle ScholarCrossref
    39.
    Khan  A, Furtak  SL, Melvin  P, Rogers  JE, Schuster  MA, Landrigan  CP.  Parent-reported errors and adverse events in hospitalized children.   jama Pediatr. 2016;170(4):e154608. doi:10.1001/jamapediatrics.2015.4608 PubMedGoogle Scholar
    40.
    Khan  A, Coffey  M, Litterer  KP,  et al; the Patient and Family Centered I-PASS Study Group.  Families as partners in hospital error and adverse event surveillance.   JAMA Pediatr. 2017;171(4):372-381. doi:10.1001/jamapediatrics.2016.4812 PubMedGoogle ScholarCrossref
    41.
    Brady  PW, Zix  J, Brilli  R,  et al.  Developing and evaluating the success of a family activated medical emergency team: a quality improvement report.   BMJ Qual Saf. 2015;24(3):203-211. doi:10.1136/bmjqs-2014-003001 PubMedGoogle ScholarCrossref
    42.
    Weingart  SN, Pagovich  O, Sands  DZ,  et al.  What can hospitalized patients tell us about adverse events? learning from patient-reported incidents.   J Gen Intern Med. 2005;20(9):830-836. doi:10.1111/j.1525-1497.2005.0180.x PubMedGoogle ScholarCrossref
    43.
    Daniels  JP, Hunc  K, Cochrane  DD,  et al.  Identification by families of pediatric adverse events and near misses overlooked by health care providers.   CMAJ. 2012;184(1):29-34. doi:10.1503/cmaj.110393 PubMedGoogle ScholarCrossref
    44.
    Mazor  KM, Roblin  DW, Greene  SM,  et al.  Toward patient-centered cancer care: patient perceptions of problematic events, impact, and response.   J Clin Oncol. 2012;30(15):1784-1790. doi:10.1200/JCO.2011.38.1384 PubMedGoogle ScholarCrossref
    45.
    McNutt  L-A, Wu  C, Xue  X, Hafner  JP.  Estimating the relative risk in cohort studies and clinical trials of common outcomes.   Am J Epidemiol. 2003;157(10):940-943. doi:10.1093/aje/kwg074 PubMedGoogle ScholarCrossref
    46.
    Glaser  B, Strauss  A.  The Discovery of Grounded TheoryL Strategies for Qualitative Theory. Routledge; 1967.
    47.
    The Commonwealth Fund. The New York Times, Harvard T.H. Chan School of Public Health. Being Seriously Ill in America Today. 2018. Accessed February 1, 2019. https://cdn1.sph.harvard.edu/wp-content/uploads/sites/94/2018/10/CMWF-NYT-HSPH-Seriously-Ill-Poll-Report.pdf
    48.
    Zhou  L, Blackley  SV, Kowalski  L,  et al.  Analysis of errors in dictated clinical documents assisted by speech recognition software and professional transcriptionists.   JAMA Netw Open. 2018;1(3):e180530. doi:10.1001/jamanetworkopen.2018.0530 PubMedGoogle Scholar
    49.
    Southwick  FS, Cranley  NM, Hallisy  JA.  A patient-initiated voluntary online survey of adverse medical events: the perspective of 696 injured patients and families.   BMJ Qual Saf. 2015;24(10):620-629. doi:10.1136/bmjqs-2015-003980 PubMedGoogle ScholarCrossref
    50.
    Sokol-Hessner  L, Kane  GJ, Annas  CL,  et al.  Development of a framework to describe patient and family harm from disrespect and promote improvements in quality and safety: a scoping review.   Int J Qual Health Care. 2019;31(9):657-668. doi:10.1093/intqhc/mzy231PubMedGoogle Scholar
    51.
    Lang  S, Velasco Garrido  M, Heintze  C.  Patients’ views of adverse events in primary and ambulatory care: a systematic review to assess methods and the content of what patients consider to be adverse events.   BMC Fam Pract. 2016;17(6):6. doi:10.1186/s12875-016-0408-0 PubMedGoogle ScholarCrossref
    52.
    Singh  H, Meyer  AN, Thomas  EJ.  The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.   BMJ Qual Saf. 2014;23(9):727-731. doi:10.1136/bmjqs-2013-002627 PubMedGoogle ScholarCrossref
    53.
    Roy  CL, Poon  EG, Karson  AS,  et al.  Patient safety concerns arising from test results that return after hospital discharge.   Ann Intern Med. 2005;143(2):121-128. doi:10.7326/0003-4819-143-2-200507190-00011 PubMedGoogle ScholarCrossref
    54.
    Wahls  T, Haugen  T, Cram  P.  The continuing problem of missed test results in an integrated health system with an advanced electronic medical record.   Jt Comm J Qual Patient Saf. 2007;33(8):485-492. doi:10.1016/S1553-7250(07)33052-3 PubMedGoogle Scholar
    55.
    Singh  H, Thomas  EJ, Mani  S,  et al.  Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?   Arch Intern Med. 2009;169(17):1578-1586. doi:10.1001/archinternmed.2009.263 PubMedGoogle ScholarCrossref
    56.
    Kessels  RPC.  Patients’ memory for medical information.   J R Soc Med. 2003;96(5):219-222.PubMedGoogle Scholar
    57.
    Bates  DW, Singh  H.  Two decades since to err is human: an assessment of progress and emerging priorities in patient safety.   Health Aff (Millwood). 2018;37(11):1736-1743. doi:10.1377/hlthaff.2018.0738 PubMedGoogle ScholarCrossref
    58.
    Davis  RE, Jacklin  R, Sevdalis  N, Vincent  CA.  Patient involvement in patient safety: what factors influence patient participation and engagement?   Health Expect. 2007;10(3):259-267. doi:10.1111/j.1369-7625.2007.00450.x PubMedGoogle ScholarCrossref
    59.
    Fisher  KA, Smith  KM, Gallagher  TH, Huang  JC, Borton  JC, Mazor  KM.  We want to know: patient comfort speaking up about breakdowns in care and patient experience.   BMJ Qual Saf. 2019;28(3):190-197. doi:10.1136/bmjqs-2018-008159 PubMedGoogle ScholarCrossref
    60.
    Bell  SK, Roche  SD, Mueller  A,  et al.  Speaking up about care concerns in the ICU: patient and family experiences, attitudes and perceived barriers.   BMJ Qual Saf. 2018;27(11):928-936. doi:10.1136/bmjqs-2017-007525PubMedGoogle ScholarCrossref
    61.
    Weiner  SJ, Wang  S, Kelly  B, Sharma  G, Schwartz  A.  How accurate is the medical record? A comparison of the physician’s note with a concealed audio recording in unannounced standardized patient encounters.   J Am Med Inform Assoc. 2020;ocaa027. Published online April 24, 2020. doi:10.1093/jamia/ocaa027PubMedGoogle Scholar
    62.
    NORC. Methodology Report: 2014-2015 Nationwide CAHPS Surveys of Adults Enrolled in Medicaid between October and December 2013. NORC; 2015. https://www.medicaid.gov/medicaid/quality-of-care/downloads/performance-measurement/methodology-report.pdf
    63.
    Goel  MS, Brown  TL, Williams  A, Hasnain-Wynia  R, Thompson  JA, Baker  DW.  Disparities in enrollment and use of an electronic patient portal.   J Gen Intern Med. 2011;26(10):1112-1116. doi:10.1007/s11606-011-1728-3 PubMedGoogle ScholarCrossref
    64.
    Etchegaray  JM, Ottosen  MJ, Aigbe  A,  et al.  Patients as partners in learning from unexpected events.   Health Serv Res. 2016;51(6)(suppl 3):2600-2614. doi:10.1111/1475-6773.12593 PubMedGoogle ScholarCrossref
    65.
    Gillespie  A, Reader  TW.  The Healthcare Complaints Analysis Tool: development and reliability testing of a method for service monitoring and organisational learning.   BMJ Qual Saf. 2016;25(12):937-946. doi:10.1136/bmjqs-2015-004596 PubMedGoogle ScholarCrossref
    ×