[Skip to Content]
[Skip to Content Landing]
Figure 1.
Systematic Surveillance of Errors and Adverse Events (AEs)
Systematic Surveillance of Errors and Adverse Events (AEs)

The established 2-step, prospective, systematic surveillance methodology currently considered highest yield for detecting errors and AEs in safety surveillance research.5,14,15 Notably, patients and families are absent from this process. Our study integrated family safety reports into the first step of this process.

Figure 2.
Sources of Errors and Adverse Events (AEs)
Sources of Errors and Adverse Events (AEs)

Sources of medical errors as validated through 2-step methodology (research clinician review followed by review by 2 physicians) across all 4 sites. Additional sources of medical errors included observation (eg, by study nurse while on unit; n = 12) and other (n = 8). Additional sources of AEs included observation (n = 7) and other (n = 4).

aCategories are not mutually exclusive, so numbers do not sum to 179 errors and 113 AEs.

bThere were 0 unique medical errors reported through hospital incident reports.

Table 1.  
Patient, Parent/Caregiver, and Clinician Characteristics
Patient, Parent/Caregiver, and Clinician Characteristics
Table 2.  
Examples of Errors and Adverse Events (AEs) by Reporting Source
Examples of Errors and Adverse Events (AEs) by Reporting Source
Table 3.  
Overall Error and AE Rates Without vs With Family Reportinga
Overall Error and AE Rates Without vs With Family Reportinga
Table 4.  
Clinician vs Family Contributions to Overall Error and AE Ratesa
Clinician vs Family Contributions to Overall Error and AE Ratesa
Table 5.  
Hospital Incident Report vs Family Contributions to Overall Error and AE Ratesa,b
Hospital Incident Report vs Family Contributions to Overall Error and AE Ratesa,b
1.
Levinson  DR; Office of Inspector General. Adverse events in hospitals: national incidence among Medicare beneficiaries. https://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf. Accessed July 28, 2016.
2.
Kohn  L, Corrigan  J, Donaldson  MS; Institute of Medicine. To err is human: building a safer health system. November http://iom.nationalacademies.org/~/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf. Accessed July 18, 2016.
3.
Makary  MA, Daniel  M.  Medical error: the third leading cause of death in the US.  BMJ. 2016;353:i2139.PubMedGoogle ScholarCrossref
4.
James  JT.  A new, evidence-based estimate of patient harms associated with hospital care.  J Patient Saf. 2013;9(3):122-128.PubMedGoogle ScholarCrossref
5.
Classen  DC, Lloyd  RC, Provost  L, Griffin  FA, Resar  R.  Development and evaluation of the Institute for Healthcare Improvement Global Trigger Tool.  J Patient Saf. 2008;4(3):169-177. doi:10.1097/PTS.0b013e318183a475Google ScholarCrossref
6.
Sharek  PJ, Parry  G, Goldmann  D,  et al.  Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients.  Health Serv Res. 2011;46(2):654-678.PubMedGoogle ScholarCrossref
7.
National Patient Safety Foundation. Free from harm: accelerating patient safety improvement fifteen years after To Err is Human. http://www.npsf.org/?freefromharm. Accessed July 30, 2016.
8.
Levinson  DR; Office of Inspector General. Hospital incident reporting systems do not capture most patient harm. http://oig.hhs.gov/oei/reports/oei-06-09-00091.pdf. Accessed September 18, 2016.
9.
Sari  AB-A, Sheldon  TA, Cracknell  A, Turnbull  A.  Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review.  BMJ. 2007;334(7584):79.PubMedGoogle ScholarCrossref
10.
Farley  DO, Haviland  A, Champagne  S,  et al.  Adverse-event-reporting practices by US hospitals: results of a national survey.  Qual Saf Health Care. 2008;17(6):416-423.PubMedGoogle ScholarCrossref
11.
Evans  SM, Berry  JG, Smith  BJ,  et al.  Attitudes and barriers to incident reporting: a collaborative hospital study.  Qual Saf Health Care. 2006;15(1):39-43.PubMedGoogle ScholarCrossref
12.
Uribe  CL, Schweikhart  SB, Pathak  DS, Dow  M, Marsh  GB.  Perceived barriers to medical-error reporting: an exploratory investigation.  J Healthc Manag. 2002;47(4):263-279.PubMedGoogle ScholarCrossref
13.
Landrigan  CP, Stockwell  D, Toomey  SL,  et al.  Performance of the Global Assessment of Pediatric Patient Safety (GAPPS) Tool  [published online May 21, 2016].  Pediatrics. 2016;137(6):e20154076. doi:10.1542/peds.2015-4076PubMedGoogle ScholarCrossref
14.
Classen  DC, Resar  R, Griffin  F,  et al.  ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured.  Health Aff (Millwood). 2011;30(4):581-589.PubMedGoogle ScholarCrossref
15.
Rozich  JD, Haraden  CR, Resar  RK.  Adverse drug event trigger tool: a practical methodology for measuring medication related harm.  Qual Saf Health Care. 2003;12(3):194-200.PubMedGoogle ScholarCrossref
16.
Weingart  SN, Pagovich  O, Sands  DZ,  et al.  What can hospitalized patients tell us about adverse events? learning from patient-reported incidents.  J Gen Intern Med. 2005;20(9):830-836.PubMedGoogle ScholarCrossref
17.
Weissman  JS, Schneider  EC, Weingart  SN,  et al.  Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not?  Ann Intern Med. 2008;149(2):100-108.PubMedGoogle ScholarCrossref
18.
Khan  A, Furtak  SL, Melvin  P, Rogers  JE, Schuster  MA, Landrigan  CP.  Parent-reported errors and adverse events in hospitalized children.  JAMA Pediatr. 2016;170(4):e154608.PubMedGoogle ScholarCrossref
19.
Daniels  JP, Hunc  K, Cochrane  DD,  et al.  Identification by families of pediatric adverse events and near misses overlooked by health care providers.  CMAJ. 2012;184(1):29-34.PubMedGoogle ScholarCrossref
20.
Matlow  AG, Baker  GR, Flintoft  V,  et al.  Adverse events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study.  CMAJ. 2012;184(13):E709-E718.PubMedGoogle ScholarCrossref
21.
Starmer  AJ, Spector  ND, Srivastava  R,  et al; I-PASS Study Group.  Changes in medical errors after implementation of a handoff program.  N Engl J Med. 2014;371(19):1803-1812.PubMedGoogle ScholarCrossref
22.
Starmer  AJ, Sectish  TC, Simon  DW,  et al.  Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle.  JAMA. 2013;310(21):2262-2270.PubMedGoogle ScholarCrossref
23.
Brennan  TA, Leape  LL, Laird  NM,  et al.  Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I.  N Engl J Med. 1991;324(6):370-376.PubMedGoogle ScholarCrossref
24.
Leape  LL, Brennan  TA, Laird  N,  et al.  The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II.  N Engl J Med. 1991;324(6):377-384.PubMedGoogle ScholarCrossref
25.
Bates  DW, Boyle  DL, Vander Vliet  MB, Schneider  J, Leape  L.  Relationship between medication errors and adverse drug events.  J Gen Intern Med. 1995;10(4):199-205.PubMedGoogle ScholarCrossref
26.
Kaushal  R, Bates  DW, Landrigan  C,  et al.  Medication errors and adverse drug events in pediatric inpatients.  JAMA. 2001;285(16):2114-2120.PubMedGoogle ScholarCrossref
27.
National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP). NCC MERP index for categorizing medication errors. http://www.nccmerp.org/sites/default/files/algorColor2001-06-12.pdf. Accessed July 18, 2016.
28.
Feudtner  C, Christakis  DA, Connell  FA.  Pediatric deaths attributable to complex chronic conditions: a population-based study of Washington State, 1980-1997.  Pediatrics. 2000;106(1, pt 2)(suppl 1):205-209.PubMedGoogle Scholar
29.
Firth  D.  Bias reduction of maximum likelihood estimates.  Biometrika. 1993;80(1):27-38. doi:10.1093/biomet/80.1.27Google ScholarCrossref
30.
Lipsitz  SR, Fitzmaurice  GM, Regenbogen  SE, Sinha  D, Ibrahim  JG, Gawande  AA.  Bias correction for the proportional odds logistic regression model with application to a study of surgical complications.  J R Stat Soc Ser C Appl Stat. 2013;62(2):233-250.PubMedGoogle ScholarCrossref
31.
Moore  CG, Lipsitz  SR, Addy  CL, Hussey  JR, Fitzmaurice  G, Natarajan  S.  Logistic regression with incomplete covariate data in complex survey sampling: application of reweighted estimating equations.  Epidemiology. 2009;20(3):382-390.PubMedGoogle ScholarCrossref
32.
Henry  AJ, Hevelone  ND, Lipsitz  S, Nguyen  LL.  Comparative methods for handling missing data in large databases.  J Vasc Surg. 2013;58(5):1353-1359.e6.PubMedGoogle ScholarCrossref
33.
Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support.  J Biomed Inform. 2009;42(2):377-381.PubMedGoogle ScholarCrossref
34.
Jeffe  DB, Dunagan  WC, Garbutt  J,  et al.  Using focus groups to understand physicians’ and nurses’ perspectives on error reporting in hospitals.  Jt Comm J Qual Saf. 2004;30(9):471-479.PubMedGoogle ScholarCrossref
35.
Elder  NC, Graham  D, Brandt  E, Hickner  J.  Barriers and motivators for making error reports from family medicine offices: a report from the American Academy of Family Physicians National Research Network (AAFP NRN).  J Am Board Fam Med. 2007;20(2):115-123.PubMedGoogle ScholarCrossref
36.
Agency for Healthcare Research and Quality. Developing and testing the Health Care Safety Hotline: a prototype consumer reporting system for patient safety events. http://www.ahrq.gov/professionals/quality-patient-safety/patient-family-engagement/hotline/index.html. Accessed July 18, 2016.
37.
Kraman  SS, Hamm  G.  Risk management: extreme honesty may be the best policy.  Ann Intern Med. 1999;131(12):963-967.PubMedGoogle ScholarCrossref
38.
Kachalia  A, Kaufman  SR, Boothman  R,  et al.  Liability claims and costs before and after implementation of a medical error disclosure program.  Ann Intern Med. 2010;153(4):213-221.PubMedGoogle ScholarCrossref
39.
Stewart  RM, Corneille  MG, Johnston  J,  et al.  Transparent and open discussion of errors does not increase malpractice risk in trauma patients.  Ann Surg. 2006;243(5):645-649.PubMedGoogle ScholarCrossref
40.
Painter  LM, Kidwell  KM, Kidwell  RP,  et al.  Do written disclosures of serious events increase risk of malpractice claims? one health care system’s experience  [published online March 31, 2015].  J Patient Saf.PubMedGoogle Scholar
41.
Macrae  C.  The problem with incident reporting.  BMJ Qual Saf. 2016;25(2):71-75.PubMedGoogle ScholarCrossref
42.
Edwards  JD, Houtrow  AJ, Vasilevskis  EE,  et al.  Chronic conditions among children admitted to U.S. pediatric intensive care units: their prevalence and impact on risk for mortality and prolonged length of stay.  Crit Care Med. 2012;40(7):2196-2203.PubMedGoogle ScholarCrossref
43.
Feinstein  JA, Feudtner  C, Kempe  A.  Adverse drug event-related emergency department visits associated with complex chronic conditions.  Pediatrics. 2014;133(6):e1575-e1585.PubMedGoogle ScholarCrossref
44.
Pennarola  BW, Rodday  AM, Mayer  DK,  et al; HSCT-CHESS Study.  Factors associated with parental activation in pediatric hematopoietic stem cell transplant.  Med Care Res Rev. 2012;69(2):194-214.PubMedGoogle ScholarCrossref
Original Investigation
April 2017

Families as Partners in Hospital Error and Adverse Event Surveillance

Author Affiliations
  • 1Division of General Pediatrics, Department of Medicine, Boston Children’s Hospital, Boston, Massachusetts
  • 2Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
  • 3Centre for Quality Improvement and Patient Safety, Department of Paediatrics, Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada
  • 4Center for Families, Boston Children’s Hospital, Boston, Massachusetts
  • 5Department of Nursing, Cardiovascular, and Critical Care Services, Boston Children’s Hospital, Boston, Massachusetts
  • 6Family-Centered Care, Lucile Packard Children’s Hospital, Palo Alto, California
  • 7Section of Critical Care, Department of Pediatrics, St Christopher’s Hospital for Children, Drexel University College of Medicine, Philadelphia, Pennsylvania
  • 8Section of Hospital Medicine, Department of Pediatrics, St Christopher’s Hospital for Children, Drexel University College of Medicine, Philadelphia, Pennsylvania
  • 9Department of Pediatrics, Cincinnati Children’s Hospital, University of Cincinnati College of Medicine, Cincinnati, Ohio
  • 10Department of Pediatrics, Benioff Children’s Hospital, University of California-San Francisco School of Medicine, San Francisco
  • 11Division of Pediatric Hospital Medicine, Lucile Packard Children’s Hospital, Stanford University School of Medicine, Palo Alto, California
  • 12Department of Pediatrics, Primary Children’s Hospital, University of Utah School of Medicine, Salt Lake City
  • 13Department of Pediatrics, Walter Reed National Military Medical Center, Uniformed Services University of the Health Sciences, Bethesda, Maryland
  • 14Department of Medicine, Harvard Medical School, Boston, Massachusetts
  • 15The Center for Patient Safety Research and Practice, Division of General Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
  • 16Institute for Healthcare Delivery Research, Intermountain Healthcare, Salt Lake City, Utah
  • 17Section of General Pediatrics, Department of Pediatrics, St Christopher’s Hospital for Children, Drexel University College of Medicine, Philadelphia, Pennsylvania
  • 18Division of Sleep Medicine, Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
JAMA Pediatr. 2017;171(4):372-381. doi:10.1001/jamapediatrics.2016.4812
Key Points

Question  How do rates of family-reported errors and adverse events (AEs) compare with those detected by other sources of hospital safety reporting that do not typically include families?

Findings  In this cohort study including 746 parents/caregivers of 989 hospitalized pediatric patients, families reported similar rates of errors and AEs as clinicians, and families reported 5-fold more errors and 3-fold more AEs than hospital incident reports. Including families in prospective systematic surveillance increased overall error detection rates by 16% and AE detection rates by 10%.

Meaning  Families provide unique safety information and have the potential to be valuable partners in safety surveillance conducted by both hospitals and researchers.

Abstract

Importance  Medical errors and adverse events (AEs) are common among hospitalized children. While clinician reports are the foundation of operational hospital safety surveillance and a key component of multifaceted research surveillance, patient and family reports are not routinely gathered. We hypothesized that a novel family-reporting mechanism would improve incident detection.

Objective  To compare error and AE rates (1) gathered systematically with vs without family reporting, (2) reported by families vs clinicians, and (3) reported by families vs hospital incident reports.

Design, Setting, and Participants  We conducted a prospective cohort study including the parents/caregivers of 989 hospitalized patients 17 years and younger (total 3902 patient-days) and their clinicians from December 2014 to July 2015 in 4 US pediatric centers. Clinician abstractors identified potential errors and AEs by reviewing medical records, hospital incident reports, and clinician reports as well as weekly and discharge Family Safety Interviews (FSIs). Two physicians reviewed and independently categorized all incidents, rating severity and preventability (agreement, 68%-90%; κ, 0.50-0.68). Discordant categorizations were reconciled. Rates were generated using Poisson regression estimated via generalized estimating equations to account for repeated measures on the same patient.

Main Outcomes and Measures  Error and AE rates.

Results  Overall, 746 parents/caregivers consented for the study. Of these, 717 completed FSIs. Their median (interquartile range) age was 32.5 (26-40) years; 380 (53.0%) were nonwhite, 566 (78.9%) were female, 603 (84.1%) were English speaking, and 380 (53.0%) had attended college. Of 717 parents/caregivers completing FSIs, 185 (25.8%) reported a total of 255 incidents, which were classified as 132 safety concerns (51.8%), 102 nonsafety-related quality concerns (40.0%), and 21 other concerns (8.2%). These included 22 preventable AEs (8.6%), 17 nonharmful medical errors (6.7%), and 11 nonpreventable AEs (4.3%) on the study unit. In total, 179 errors and 113 AEs were identified from all sources. Family reports included 8 otherwise unidentified AEs, including 7 preventable AEs. Error rates with family reporting (45.9 per 1000 patient-days) were 1.2-fold (95% CI, 1.1-1.2) higher than rates without family reporting (39.7 per 1000 patient-days). Adverse event rates with family reporting (28.7 per 1000 patient-days) were 1.1-fold (95% CI, 1.0-1.2; P =.006) higher than rates without (26.1 per 1000 patient-days). Families and clinicians reported similar rates of errors (10.0 vs 12.8 per 1000 patient-days; relative rate, 0.8; 95% CI, .5-1.2) and AEs (8.5 vs 6.2 per 1000 patient-days; relative rate, 1.4; 95% CI, 0.8-2.2). Family-reported error rates were 5.0-fold (95% CI, 1.9-13.0) higher and AE rates 2.9-fold (95% CI, 1.2-6.7) higher than hospital incident report rates.

Conclusions and Relevance  Families provide unique information about hospital safety and should be included in hospital safety surveillance in order to facilitate better design and assessment of interventions to improve safety.

Introduction

Between 44 000 and 440 000 patients are estimated to die yearly in the United States due to medical errors,1-4 making medical errors a leading cause of death. Detecting medical errors is important for identifying causative factors and measuring the effectiveness of prevention strategies. Error detection methodologies have greatly advanced over the past few decades.5-7 However, implementation of error detection, or safety surveillance, varies across hospitals.

Hospital incident reports, which are voluntary and suffer from underreporting, are commonly used but only capture a small subset of errors and adverse events (AEs), ie, harms due to medical care.8-12 Trigger tools are higher yield and becoming more sophisticated at identifying events but are not yet widely used.13 Prospective systematic surveillance, in which trained research staff conduct daily reviews of patient medical records (Figure 1), is the most effective way to measure errors but is expensive and typically limited to research settings.5,14,15

Patients and families are absent both from hospital incident reports and systematic surveillance. Limited adult studies suggest that patients are a fruitful source of safety surveillance.16,17 Small, single-center pediatric studies suggest similar findings; however, the role of patients and families in pediatric safety reporting has not been widely examined.18-20 Additionally, to our knowledge, a rigorous patient and family error-reporting methodology has not been developed or operationalized.

We sought to fill these gaps by developing a family error-reporting methodology and testing its utility and effectiveness in systematic safety surveillance across 4 US pediatric hospitals. We hypothesized that families would report errors and AEs not discovered by other methods and that adding a family reporting mechanism to systematic surveillance methodology would increase error and AE detection. If these hypotheses are correct, then adding families to safety surveillance systems would provide an important new way to detect errors, identify their causes, and measure the effectiveness of efforts to prevent them.

Methods
Data, Setting, and Study Population

We conducted a prospective cohort study in 4 pediatric hospitals. Data were collected from December 2014 to July 2015 coincident with data collection for the Patient and Family Centered I-PASS Study, an ongoing multicenter investigation of clinician-family communication and safety built on prior communication and safety research.21

Study participants included parents/guardians or caregivers (eg, grandparents living in the home) and clinicians (resident-physicians and nurses) of 989 hospitalized medical patients 17 years and younger on inpatient general pediatric and subspecialty units. We obtained written informed consent from clinicians, verbal consent using an information sheet from parents/caregivers, and a waiver of informed consent to review patient records. The Boston Children’s Hospital Institutional Review Board and each participating institution’s institutional review board approved the study.

Inclusions

Given limited translation and interpretation resources, we included parents/caregivers speaking English, Spanish, Arabic, Chinese, or Russian (the most commonly spoken languages across study sites).

Family Safety Interviews

We developed the Family Safety Interview (FSI) using a modified Delphi method with input from experts in survey methodology, patient safety, and health literacy, as well as family partners. Draft versions of the FSI were pilot tested through cognitive interviews with parents/caregivers at Boston Children’s Hospital.

To orient families to types of information to report, the FSI began with descriptions, definitions, and examples of safety events related to medications, miscommunications, diagnoses, delays in care, complications of care, and equipment. Using closed-ended and open-ended questions, it then asked parents/caregivers to indicate whether the child’s illness worsened or almost worsened because (1) of medical care (ie, an AE), (2) something was not done that should have been (ie, a preventable AE due to an error of omission), or (3) something was done that should not have been (ie, a preventable AE due to an error of commission). It also asked if a mistake occurred that did not result in harm (ie, a nonharmful error) and whether anything else happened or almost happened that was upsetting or could have harmed the child.

Research assistants administered this semistructured interview to parents/caregivers of eligible patients every 7 days while hospitalized and before discharge. Interviews typically lasted 3 to 5 minutes when no concerns were reported and 10 to 15 minutes when concerns were reported. Research clinicians (nurses or physicians) then classified family responses as safety concerns (ie, potential errors and AEs), nonsafety-related quality concerns (eg, an unpleasant interaction with a physician), or other concerns (eg, a difficult intravenous placement). Off-unit (eg, emergency department) safety concerns were subsequently excluded from analyses.

Clinician Event Reporting Surveys

Clinician event reporting was conducted by modifying an instrument used in previous studies.21,22 This less-than-1-minute survey asked respondents to describe errors and AEs or procedures, medications, fluids, or other therapies that were unnecessary or questionably beneficial, delayed, involved in an error or AE, or ordered erroneously but intercepted before reaching the patient (near-misses). They also asked about resident or nurse sign-out omissions or inaccuracies leading to problems with patient care. Every weekday morning, research clinicians administered the survey (verbally or on paper) to outgoing overnight residents. Surveys were also posted on the study units to allow unit staff, particularly nurses, not enrolled in the study to voluntarily report anonymously.

Error Validation/Classification

To identify potential errors and AEs, research clinicians conducted systematic surveillance using a validated methodology21-26 (ie, daily review of all study-unit patients’ medical records, hospital incident reports, and clinician event reporting surveys) (Figure 1). These data were supplemented with incidents collected during FSIs. For potential errors and AEs, research clinicians recorded reporting source(s) (ie, family, clinician, observation by study personnel, or other), preceding events, patient outcomes, harm level (using modified National Coordinating Council for Medication Error Reporting and Prevention categorization27), preventability (if harmful), error category, and follow-up actions.

To validate and categorize events, physician reviewer pairs independently reviewed all research clinician–collected events (including those obtained from FSIs) as AEs (ie, harms), nonharmful errors, or exclusions (preconsensus agreement, 67.6%; κ, 0.50). Physician raters additionally assessed AEs for harm level27 (preconsensus agreement, 89.5%; κ, 0.68) and preventability (preconsensus agreement, 83.7%; κ, 0.60). Physician reviewer pairs reconciled discordant categorizations through consensus.

Demographic Data

We collected clinician and parent/caregiver demographic information through surveys. We obtained patient clinical and sociodemographic characteristics data from hospital administrative data.

Outcomes and Predictors

Our primary outcome was overall medical error and AE rates per 1000 patient-days (days hospitalized on the study unit) both including and excluding family reporting (ie, FSI data). We also analyzed error and AE rates per 100 admissions. We secondarily analyzed error and AE rates reported by families vs clinicians and by families vs hospital incident reports. We identified the following a priori variables as potential predictors of family reporting: patient age, sex, race/ethnicity, insurance, complex chronic conditions (CCCs, a marker of medical complexity according to International Classification of Diseases, Ninth Revision, Clinical Modification28), and parent/caregiver age, sex, race/ethnicity, language, education, and household income.

Statistical Analyses

To compare error and AE rates (overall with vs without family data, family vs clinician, and family vs hospital incident report), we used Poisson regression estimated via generalized estimating equations to account for correlation arising from collecting parent/caregiver, clinician, and hospital incident reports on the same patient. All 989 sample patients were used for the overall and family vs clinician analyses. The family vs hospital incident report analysis only included the 3 sites with available hospital incident report data.

In unadjusted and adjusted analyses, we compared family-reported error and AE rates per 1000 patient-days using bias-corrected Poisson regression, which has better statistical properties than usual Poisson regression when the total number of events is not large.29,30 A priori parent/caregiver-level and patient-level predictors of interest that were significant (P < .05) in unadjusted analyses were fit in an adjusted model. The final adjusted model only included predictors with P < .05. Site effects were considered as fixed effects in the adjusted model. For consistency, final adjusted models for errors and AEs contained the same predictors.

Unadjusted and adjusted analyses included only the 717 patients whose parents/caregivers completed FSIs, which we refer to as a complete-case analysis. Because FSIs were not completed for all study-unit patients, we compared patient characteristics among those with and without completed FSIs. Additionally, we conducted a sensitivity analysis using a reweighted estimating equation missing data approach to determine if complete-cases produced biased results. In the reweighted estimating equations approach, we fit a logistic regression model to determine predictors of a parent/caregiver completing an interview (using all nonmissing variables). Then, complete-cases were reweighted by the inverse probability of a parent/caregiver completing an interview. Thus, patients whose parents/caregivers completed interviews despite being less likely to complete them were up-weighted to account for those similar parents/caregivers who did not complete interviews.31,32 We used REDcap33 to collect and manage study data and SAS version 9.3 (SAS Institute) for analyses.

Results
Sample Characteristics

Of 1024 patients on the study units, 989 met study criteria (eFigure in the Supplement). Parents/caregivers of 782 eligible patients (79.1%) were available and therefore approached. Most parents/caregivers (746 [95.4%]) who were approached consented for the study. Of these, 717 parents/caregivers completed 763 interviews (35 parents/caregivers completed more than 1 interview). Among clinicians, 146 nurses (98.6%) and 207 resident-physicians (95.4%) consented for the study. A total of 717 of 782 eligible parents/caregivers (91.7%) completed FSIs. Overall, 77 residents completed a total of 284 of 327 (86.9%) solicited postshift event reporting surveys, with residents completing 0-11 surveys each.

Patient, parent/caregiver, and clinician demographic data are reported in Table 1. Patients and parents/caregivers were from varied racial/ethnic and socioeconomic backgrounds. Overall, 243 of 989 patients (24.6%) had 1 or more CCCs. Patients without completed FSIs (primarily because parents/caregivers were not present) were similar to patients with completed FSIs in terms of age, CCCs, race/ethnicity, insurance, and error and AE rates. Patient sex was the only statistically significant difference between these 2 groups.

Errors and AEs

We found a total of 179 errors and 113 AEs on our study units through all sources. Families reported 39 of 179 total errors (21.8%; 19 [10.6%] uniquely and 20 [11.2%] in combination with another source) and 33 of 113 total AEs (29.2%; 8 [7.1%] uniquely and 25 [22.1%] in combination with another source) (Figure 2).

Family Reports

Overall, 185 parents/caregivers (25.8%) reported 255 total incidents, of which we classified 132 (51.8%) as safety concerns, 102 (40.0%) as nonsafety-related quality concerns, and 21 (8.2%) as other concerns. Of reported safety concerns, 35 occurred off-unit and were excluded from further analyses; 97 (73.5%) occurred on-unit. On 2-step review, 50 family-reported on-unit errors and/or AEs were confirmed, including 22 preventable AEs, 17 nonharmful medical errors, and 11 nonpreventable AEs.

Family reports included 8 otherwise unidentified AEs, including 7 preventable AEs. Unique family-reported AEs included multiple needle sticks, inadequate suctioning, and adverse effects from medication (Table 2). Of 39 validated family-reported errors (ie, preventable AEs and nonharmful errors), 20 were also detected through research clinician medical record review, 3 through resident report, 2 through nurse report, and 1 through hospital incident report. Of 33 validated family-reported AEs, 25 were also detected through medical record review, 3 through nurse report, 2 through resident report, and 2 through hospital incident report.

Family-reported AEs most commonly involved temporary patient harm requiring intervention (28 [84.9%]) or prolonging hospitalization (5 [15.2%]). Both family-reported and nonfamily-reported errors and AEs were primarily medication related.

Overall Error and AE Rates With vs Without Family-Reporting

Overall error rates with family reporting were 15.5% (95% CI, 9.0%-22.3%) higher than without. Overall AE rates with family reporting were 9.8% (95% CI, 3.1%-16.9%) higher than without (Table 3).

Family-Reported vs Clinician-Reported Errors and AEs

Family-reported error rates were equivalent to clinician-reported rates (relative rate [RR], 0.8; 95% CI, 0.5-1.2). Family-reported AE rates were also equivalent to clinician-reported rates (RR, 1.4; 95% CI, 0.8-2.2) (Table 4).

Family-Reported vs Hospital Incident Report Rates of Errors and AEs

Among the 3 sites with available hospital incident report data, of 111 errors detected through all sources, hospital incident reports detected 5 errors (4.5%) and family reports detected 25 errors (22.5%). Of 71 AEs detected through all sources, hospital incident reports detected 7 AEs (9.9%) and family reports detected 20 AEs (28.2%).

Family-reported error rates were 5.0-fold (95% CI, 1.9-13.0) higher than hospital incident report rates across these sites. Family-reported AE rates were 2.9-fold (95% CI, 1.2-6.7) higher than hospital incident report rates (Table 5).

Predictors of Family-Reported Errors and AEs

Results of the weighted estimated approach accounting for missing data were similar to unweighted results; for simplicity, we report only unweighted results. Unadjusted predictors of family-reported errors included patient age, 1 or more CCCs, parent/caregiver proficiency in written and spoken English, and higher parent/caregiver education (eTable 1 in the Supplement). Adjusted predictors of family-reported errors included younger patient age (RR, 0.9 per 1-year increment; 95% CI, 0.9-1.0), 1 or more CCCs (RR, 2.4; 95% CI, 1.2-4.8), older parent/caregiver age (RR, 1.0; 95% CI, 1.0-1.1), and higher parent/caregiver education (less than high school vs college education: RR, 0.2; 95% CI, 0.1-0.3) (eTable 2 in the Supplement).

Unadjusted predictors of family-reported AEs include 1 or more CCCs, higher parent/caregiver education, and greater family presence during hospitalization. Adjusted predictors of family-reported AEs included 1 or more CCCs (RR, 2.3; 95% CI, 1.1-4.8) and higher parent/caregiver education (less than high school vs college education: RR, 0.3; 95% CI, 0.2-0.4).

Discussion

In a study of 4 US hospitals, solicited family reporting yielded 5-fold more errors and 3-fold more AEs than the voluntary hospital incident reports that most hospitals use as their primary patient safety surveillance tool. Additionally, we found that family reporting substantively enhanced rigorous multifaceted patient safety surveillance methods, with families reporting patient safety incidents that were not otherwise detected. Our finding that family reporting increased overall error detection by 16% and AE detection by 10%—compared with what is typically considered the highest-yield methodology in safety surveillance research—supports our hypothesis that including families in safety surveillance improves safety detection. This represents an important patient safety innovation. By including families in safety reporting, we can identify otherwise unrecognized errors and AEs, providing new opportunities to prevent them from occurring.

Our study’s family-reported error and AE rates are far higher than rates detected through clinician-only, voluntary hospital incident reports, which typically only detect 1% to 14% of AEs.8,14 We similarly found that hospital incident reports only captured 5% of errors and 10% of AEs. In contrast, our family-reporting methodology was much higher yield, detecting 22% of errors and 29% of AEs. This may be because families are available to provide patient safety information and may in fact have more opportunity to provide such reports than clinicians, who might be prevented from reporting by competing time demands.34,35 We additionally found that 49% of family-reported errors and 24% of family-reported AEs were not present in the medical record, consistent with prior studies.14,16-18

Based on other studies, patient and family safety-reporting rates are variable and appear related to mode of reporting. The percentage of families in our study reporting incidents (26%) was approximately 3-fold higher than in our prior single-center study (9%),18 in which we used written family discharge experience surveys (vs the active semistructured family interviews used in our current study). Additionally, our current study’s reporting rate is more than 600-fold higher than that of a voluntary prototype consumer-reporting hotline that was tested with funding from the Agency for Healthcare Research and Quality in 2 hospital institutions in a single state and administered via the web and telephone (0.04%; Denise Quigley, PhD, email communication, June 2016).36

Prior studies using active patient and family reporting similarly found that 23% to 49% of patients and families reported safety concerns.16,17,19 These studies used various methods, including postdischarge patient telephone interviews,16 computer-based safety-reporting parent discharge surveys,17 and in-person patient/caregiver interviews 2 to 3 times per week.19 An important difference between our study and these is that we subsequently validated family safety reporting using a rigorous 2-step methodology. Our validation step represents an important advance in medical error and AE reporting methodology.

Further research is needed to determine the best process for operationalizing patient and family reporting in hospitals and to assess its feasibility, success, and safety implications. While less intensive than systematic surveillance, gathering patient and family safety reports requires hospital personnel time and effort. Time and effort may vary depending on whether postdischarge interviews or in-hospital surveys are used, for instance. Further research is required to compare and weigh yield and costs of different approaches. While we employed research assistants for this purpose in our study, it is plausible that, if they felt it worthwhile, hospitals might use existing quality improvement staff to administer family safety surveys or interviews. Hospitals might additionally consider incorporating safety reporting into their existing family surveys, thereby leveraging existing resources, although the sensitivity of this approach would need to be further evaluated.

The effects of patient and family safety reporting on outcomes, such as malpractice risk, parent experience, and subsequent hospital improvement efforts, are unknown. Research suggests that disclosing errors of which patients are unaware does not lead to more malpractice cases.37-40 It seems unlikely that gathering patient and family reports of errors of which patients are already aware would increase malpractice risk. Moreover, engaging patients and families may drive hospitals to bridge the gap between safety reporting and improvement. Involving patients and families may increase transparency and accountability of safety reporting and thereby address criticisms that safety reports are not consistently acted on.41

Given our finding that families of children with 1 or more CCCs were more likely to report errors and AEs, actively partnering with families of patients with CCCs may be a high-yield approach to detecting and perhaps preventing medical errors and AEs. Patients with CCCs experience higher rates of errors and AEs owing to increased illness complexity and length of stay.17,18,42,43 Their families may also be more activated or better understand the medical system,44 making them particularly high-yield reporters. However, it is also important for hospitals to make concerted efforts to actively engage all families in safety reporting and, in particular, to engage less educated and other vulnerable populations.

Limitations

Our study had several limitations. We included patients from general pediatric and subspecialty services, not surgical services, at 4 hospitals. Therefore, our reporting rates may not be generalizable to all hospitals or services. However, family and caregiver reporting may be broadly applicable to other patient populations (eg, adult, surgical, and geriatric). Although we used a rigorous methodology using a well-established 2-step review process with kappas similar to or higher than prior studies,21 rating errors and AEs is complex. Additionally, families reported a number of off-unit (eg, emergency department) safety concerns beyond our study’s scope to further investigate. Our rates of validated family-reported errors and AEs may have been even higher had we done so.

Conclusions

We developed and studied a novel family-reporting methodology that suggests families may be useful partners in hospital safety reporting. They report events not otherwise detected or documented, including preventable AEs. Family reporting increases error and AE rates detected through voluntary hospital incident reporting systems used by most hospitals and systematic safety surveillance used in research—both of which typically exclude patients and families. Actively surveying families about safety may be a fruitful way to gather errors and AEs—for both hospital safety improvement and research.

Back to top
Article Information

Corresponding Author: Alisa Khan, MD, MPH, Division of General Pediatrics, Department of Medicine, Boston Children’s Hospital, 21 Autumn St, 200.2, Boston, MA 02215 (alisa.khan@childrens.harvard.edu).

Accepted for Publication: December 5, 2016.

Correction: This article was corrected on January 8, 2018, to fix Rebecca L. Blankenburg’s name.

Published Online: February 27, 2017. doi:10.1001/jamapediatrics.2016.4812

The Patient and Family Centered I-PASS Study Group, in addition to the listed authors, includes the following individuals: Brenda K. Allair, BA, MEd; Claire Alminde, MSN, RN; Wilma Alvarado-Little, MA, MSW; Marisa Atsatt, MS; Megan E. Aylor, MD; James F. Bale Jr, MD; Dorene Balmer, PhD; Kevin T. Barton, MD; Carolyn Beck, MD, FRCPC; Zia Bismilla, MD, FRCPC, MEd; Rebecca L. Blankenburg, MD, MPH; Debra Chandler, RN; Amanda Choudhary, MHA; Eileen Christensen, BA; Sally Coghlan-McDonald, JD; F. Sessions Cole, MD; Elizabeth Corless, RN; Sharon Cray, BBA; Roxi Da Silva, MSN, RN; Devesh Dahale, BS, MSc; Benard Dreyer, MD; Amanda S. Growdon, MD; LeAnn Gubler, RN; Amy Guiot, MD, MEd; Roben Harris; Helen Haskell, MA; Irene Kocolas, MD, MS; Elizabeth Kruvand, BS; Michele Marie Lane, RN; Kathleen Langrish, RN, MN; Christy J. W. Ledford, PhD; Kheyandra Lewis, MD; Joseph O. Lopreiato, MD, MPH; Christopher G. Maloney, MD, PhD; Amanda Mangan, RN; Peggy Markle, BA; Fernando Mendoza, MD; Dale Ann Micalizzi, AAS; Vineeta Mittal, MD, MBA; Maria Obermeyer, RN; Katherine A. O’Donnell, MD; Mary Ottolini, MD, MPH; Shilpa J. Patel, MD; Rita Pickler, PhD, RN; Jayne Elizabeth Rogers, MSN, RN, NEA-BC; Lee M. Sanders, MD; Kimberly Sauder, RN; Samir S. Shah, MD, MSCE; Meesha Sharma, MD, MPH; Arabella Simpkin, BMBCh, MMSc; Anupama Subramony, MD, MBA; E. Douglas Thompson Jr, MD; Laura Trueman, RN; Tanner Trujillo, MHA; Michael P. Turmelle, MD; Cindy Warnick, BS; Chelsea Welch, BS; Andrew J. White, MD; Matthew F. Wien, BS; Ariel S. Winn, MD; Stephanie Wintch, RN; Michael Wolf, PhD, MPH; H. Shonna Yin, MD, MS; Clifton E. Yu, MD.

Affiliations of The Patient and Family Centered I-PASS Study Group: Boston Children’s Hospital, Boston, Massachusetts (Allair, Rogers, Sharma); St Christopher’s Hospital for Children, Philadelphia, Pennsylvania (Alminde, Cray); Alvarado-Little Consulting LLC, Albany, New York (Alvarado-Little); Lucile Packard Children’s Hospital, Stanford, California (Atsatt, Wintch); Doernbecher Children’s Hospital, Oregon Health and Science University, Portland (Aylor); Primary Children’s Hospital, Intermountain Healthcare, University of Utah School of Medicine, Salt Lake City (Bale, Kocolas, Maloney); Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia (Balmer); St Louis Children’s Hospital, Washington University School of Medicine, St Louis, Missouri (Barton, Cole, Turmelle, White); Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada (Beck, Bismilla, Langrish); Lucile Packard Children’s Hospital, Stanford University, Stanford, California (Blankenburg, Mendoza, Sanders); Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio (Chandler, Dahale, Obermeyer, Trueman); Primary Children’s Hospital, Salt Lake City, Utah (Choudhary, Christensen, Corless, Gubler, Trujillo, Warnick, Welch); Benioff Children’s Hospital, San Francisco, California (Coghlan-McDonald, Mangan); Walter Reed National Military Medical Center, Bethesda, Maryland (Da Silva, Markle); New York University Langone Medical Center, New York University School of Medicine, New York (Dreyer, Yin); Boston Children’s Hospital, Harvard Medical School, Boston, Massachusetts (Growdon, O’Donnell, Winn); Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio (Guiot, Shah); St Louis Children’s Hospital, St Louis, Missouri (Harris, Kruvand, Lane, Sauder); Mothers Against Medical Error, Columbia, South Carolina (Haskell); Walter Reed National Military Medical Center, Uniformed Services University of the Health Sciences, Bethesda, Maryland (Ledford, Lopreiato, Yu); St Christopher’s Hospital for Children, Drexel University College of Medicine, Philadelphia, Pennsylvania (Lewis, Thompson); Justin’s HOPE Project, Task Force for Global Health, Decatur, Georgia (Micalizzi); Children’s Medical Center Dallas, University of Texas Southwestern Medical Center, Dallas (Mittal); Children’s National Health System, George Washington University School of Medicine, Washington, DC (Ottolini); Kapi’olani Medical Center for Women and Children, University of Hawai’i John A. Burns School of Medicine, Honolulu (Patel); Ohio State University, Columbus (Pickler); Harvard Medical School, Boston, Massachusetts (Simpkin); Cohen Children’s Medical Center, Hofstra Northwell School of Medicine, East Garden City, New York (Subramony); Brigham and Women’s Hospital, Boston, Massachusetts (Wien); Northwestern University Feinberg School of Medicine, Evanston, Illinois (Wolf).

Author Contributions: Dr Lipsitz and Ms Yoon had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Khan, Landrigan.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Lipsitz, Yoon.

Obtained funding: Khan, Landrigan.

Administrative, technical, or material support: Furtak, Garcia.

Supervision: Landrigan.

Conflict of Interest Disclosures: Drs Landrigan, Sectish, Spector, Srivastava, Starmer, and West have consulted with and hold equity in the I-PASS Institute, which seeks to train institutions in best handoff practices and aid in their implementation. Drs Sectish, Spector, Srivastava, Starmer, and West have received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for teaching and consulting on physician performance and handoffs. Drs Landrigan and Srivastava are supported in part by the Children’s Hospital Association for their work as Executive Council members of the Pediatric Research in Inpatient Settings (PRIS) network. Dr Landrigan has also served as a paid consultant to Virgin Pulse to help develop a Sleep and Health Program. In addition, Dr Landrigan has received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for teaching and consulting on sleep deprivation, physician performance, handoffs, and safety and has served as an expert witness in cases regarding patient safety and sleep deprivation. No other disclosures were reported.

Funding/Support: This project was supported by grant CDR-1306-03556 from the Patient-Centered Outcomes Research Institute (Dr Landrigan). Dr Khan’s time was supported by K12 grant K12HS022986 from the Agency for Healthcare Research and Quality (principal investigator: Jonathan Finkelstein, MD; Boston Children’s Hospital, Boston, Massachusetts). Dr Baird’s time was supported by grant 5T32HS00063-21 from the Agency for Healthcare Research and Quality (principal investigator: Jonathan Finkelstein, MD).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Information: The authors listed in the byline are also members of the Patient and Family Centered I-PASS Study Group.

Additional Contributions: We thank the patients, families, and caregivers involved in this project, as well as the research assistants, research clinicians, and unit staff. We also thank Cindy Brach, MPP (Agency for Healthcare Research and Quality, Rockville, Maryland), for her assistance reviewing study instruments and providing guidance on health literacy best practices. Ms Brach was not compensated for her work.

References
1.
Levinson  DR; Office of Inspector General. Adverse events in hospitals: national incidence among Medicare beneficiaries. https://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf. Accessed July 28, 2016.
2.
Kohn  L, Corrigan  J, Donaldson  MS; Institute of Medicine. To err is human: building a safer health system. November http://iom.nationalacademies.org/~/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf. Accessed July 18, 2016.
3.
Makary  MA, Daniel  M.  Medical error: the third leading cause of death in the US.  BMJ. 2016;353:i2139.PubMedGoogle ScholarCrossref
4.
James  JT.  A new, evidence-based estimate of patient harms associated with hospital care.  J Patient Saf. 2013;9(3):122-128.PubMedGoogle ScholarCrossref
5.
Classen  DC, Lloyd  RC, Provost  L, Griffin  FA, Resar  R.  Development and evaluation of the Institute for Healthcare Improvement Global Trigger Tool.  J Patient Saf. 2008;4(3):169-177. doi:10.1097/PTS.0b013e318183a475Google ScholarCrossref
6.
Sharek  PJ, Parry  G, Goldmann  D,  et al.  Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients.  Health Serv Res. 2011;46(2):654-678.PubMedGoogle ScholarCrossref
7.
National Patient Safety Foundation. Free from harm: accelerating patient safety improvement fifteen years after To Err is Human. http://www.npsf.org/?freefromharm. Accessed July 30, 2016.
8.
Levinson  DR; Office of Inspector General. Hospital incident reporting systems do not capture most patient harm. http://oig.hhs.gov/oei/reports/oei-06-09-00091.pdf. Accessed September 18, 2016.
9.
Sari  AB-A, Sheldon  TA, Cracknell  A, Turnbull  A.  Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review.  BMJ. 2007;334(7584):79.PubMedGoogle ScholarCrossref
10.
Farley  DO, Haviland  A, Champagne  S,  et al.  Adverse-event-reporting practices by US hospitals: results of a national survey.  Qual Saf Health Care. 2008;17(6):416-423.PubMedGoogle ScholarCrossref
11.
Evans  SM, Berry  JG, Smith  BJ,  et al.  Attitudes and barriers to incident reporting: a collaborative hospital study.  Qual Saf Health Care. 2006;15(1):39-43.PubMedGoogle ScholarCrossref
12.
Uribe  CL, Schweikhart  SB, Pathak  DS, Dow  M, Marsh  GB.  Perceived barriers to medical-error reporting: an exploratory investigation.  J Healthc Manag. 2002;47(4):263-279.PubMedGoogle ScholarCrossref
13.
Landrigan  CP, Stockwell  D, Toomey  SL,  et al.  Performance of the Global Assessment of Pediatric Patient Safety (GAPPS) Tool  [published online May 21, 2016].  Pediatrics. 2016;137(6):e20154076. doi:10.1542/peds.2015-4076PubMedGoogle ScholarCrossref
14.
Classen  DC, Resar  R, Griffin  F,  et al.  ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured.  Health Aff (Millwood). 2011;30(4):581-589.PubMedGoogle ScholarCrossref
15.
Rozich  JD, Haraden  CR, Resar  RK.  Adverse drug event trigger tool: a practical methodology for measuring medication related harm.  Qual Saf Health Care. 2003;12(3):194-200.PubMedGoogle ScholarCrossref
16.
Weingart  SN, Pagovich  O, Sands  DZ,  et al.  What can hospitalized patients tell us about adverse events? learning from patient-reported incidents.  J Gen Intern Med. 2005;20(9):830-836.PubMedGoogle ScholarCrossref
17.
Weissman  JS, Schneider  EC, Weingart  SN,  et al.  Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not?  Ann Intern Med. 2008;149(2):100-108.PubMedGoogle ScholarCrossref
18.
Khan  A, Furtak  SL, Melvin  P, Rogers  JE, Schuster  MA, Landrigan  CP.  Parent-reported errors and adverse events in hospitalized children.  JAMA Pediatr. 2016;170(4):e154608.PubMedGoogle ScholarCrossref
19.
Daniels  JP, Hunc  K, Cochrane  DD,  et al.  Identification by families of pediatric adverse events and near misses overlooked by health care providers.  CMAJ. 2012;184(1):29-34.PubMedGoogle ScholarCrossref
20.
Matlow  AG, Baker  GR, Flintoft  V,  et al.  Adverse events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study.  CMAJ. 2012;184(13):E709-E718.PubMedGoogle ScholarCrossref
21.
Starmer  AJ, Spector  ND, Srivastava  R,  et al; I-PASS Study Group.  Changes in medical errors after implementation of a handoff program.  N Engl J Med. 2014;371(19):1803-1812.PubMedGoogle ScholarCrossref
22.
Starmer  AJ, Sectish  TC, Simon  DW,  et al.  Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle.  JAMA. 2013;310(21):2262-2270.PubMedGoogle ScholarCrossref
23.
Brennan  TA, Leape  LL, Laird  NM,  et al.  Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I.  N Engl J Med. 1991;324(6):370-376.PubMedGoogle ScholarCrossref
24.
Leape  LL, Brennan  TA, Laird  N,  et al.  The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II.  N Engl J Med. 1991;324(6):377-384.PubMedGoogle ScholarCrossref
25.
Bates  DW, Boyle  DL, Vander Vliet  MB, Schneider  J, Leape  L.  Relationship between medication errors and adverse drug events.  J Gen Intern Med. 1995;10(4):199-205.PubMedGoogle ScholarCrossref
26.
Kaushal  R, Bates  DW, Landrigan  C,  et al.  Medication errors and adverse drug events in pediatric inpatients.  JAMA. 2001;285(16):2114-2120.PubMedGoogle ScholarCrossref
27.
National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP). NCC MERP index for categorizing medication errors. http://www.nccmerp.org/sites/default/files/algorColor2001-06-12.pdf. Accessed July 18, 2016.
28.
Feudtner  C, Christakis  DA, Connell  FA.  Pediatric deaths attributable to complex chronic conditions: a population-based study of Washington State, 1980-1997.  Pediatrics. 2000;106(1, pt 2)(suppl 1):205-209.PubMedGoogle Scholar
29.
Firth  D.  Bias reduction of maximum likelihood estimates.  Biometrika. 1993;80(1):27-38. doi:10.1093/biomet/80.1.27Google ScholarCrossref
30.
Lipsitz  SR, Fitzmaurice  GM, Regenbogen  SE, Sinha  D, Ibrahim  JG, Gawande  AA.  Bias correction for the proportional odds logistic regression model with application to a study of surgical complications.  J R Stat Soc Ser C Appl Stat. 2013;62(2):233-250.PubMedGoogle ScholarCrossref
31.
Moore  CG, Lipsitz  SR, Addy  CL, Hussey  JR, Fitzmaurice  G, Natarajan  S.  Logistic regression with incomplete covariate data in complex survey sampling: application of reweighted estimating equations.  Epidemiology. 2009;20(3):382-390.PubMedGoogle ScholarCrossref
32.
Henry  AJ, Hevelone  ND, Lipsitz  S, Nguyen  LL.  Comparative methods for handling missing data in large databases.  J Vasc Surg. 2013;58(5):1353-1359.e6.PubMedGoogle ScholarCrossref
33.
Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support.  J Biomed Inform. 2009;42(2):377-381.PubMedGoogle ScholarCrossref
34.
Jeffe  DB, Dunagan  WC, Garbutt  J,  et al.  Using focus groups to understand physicians’ and nurses’ perspectives on error reporting in hospitals.  Jt Comm J Qual Saf. 2004;30(9):471-479.PubMedGoogle ScholarCrossref
35.
Elder  NC, Graham  D, Brandt  E, Hickner  J.  Barriers and motivators for making error reports from family medicine offices: a report from the American Academy of Family Physicians National Research Network (AAFP NRN).  J Am Board Fam Med. 2007;20(2):115-123.PubMedGoogle ScholarCrossref
36.
Agency for Healthcare Research and Quality. Developing and testing the Health Care Safety Hotline: a prototype consumer reporting system for patient safety events. http://www.ahrq.gov/professionals/quality-patient-safety/patient-family-engagement/hotline/index.html. Accessed July 18, 2016.
37.
Kraman  SS, Hamm  G.  Risk management: extreme honesty may be the best policy.  Ann Intern Med. 1999;131(12):963-967.PubMedGoogle ScholarCrossref
38.
Kachalia  A, Kaufman  SR, Boothman  R,  et al.  Liability claims and costs before and after implementation of a medical error disclosure program.  Ann Intern Med. 2010;153(4):213-221.PubMedGoogle ScholarCrossref
39.
Stewart  RM, Corneille  MG, Johnston  J,  et al.  Transparent and open discussion of errors does not increase malpractice risk in trauma patients.  Ann Surg. 2006;243(5):645-649.PubMedGoogle ScholarCrossref
40.
Painter  LM, Kidwell  KM, Kidwell  RP,  et al.  Do written disclosures of serious events increase risk of malpractice claims? one health care system’s experience  [published online March 31, 2015].  J Patient Saf.PubMedGoogle Scholar
41.
Macrae  C.  The problem with incident reporting.  BMJ Qual Saf. 2016;25(2):71-75.PubMedGoogle ScholarCrossref
42.
Edwards  JD, Houtrow  AJ, Vasilevskis  EE,  et al.  Chronic conditions among children admitted to U.S. pediatric intensive care units: their prevalence and impact on risk for mortality and prolonged length of stay.  Crit Care Med. 2012;40(7):2196-2203.PubMedGoogle ScholarCrossref
43.
Feinstein  JA, Feudtner  C, Kempe  A.  Adverse drug event-related emergency department visits associated with complex chronic conditions.  Pediatrics. 2014;133(6):e1575-e1585.PubMedGoogle ScholarCrossref
44.
Pennarola  BW, Rodday  AM, Mayer  DK,  et al; HSCT-CHESS Study.  Factors associated with parental activation in pediatric hematopoietic stem cell transplant.  Med Care Res Rev. 2012;69(2):194-214.PubMedGoogle ScholarCrossref
×