[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Views 159
Citations 0
Original Investigation
December 26, 2018

Use of Computerized Provider Order Entry Events for Postoperative Complication Surveillance

Author Affiliations
  • 1Division of Pediatric Surgery, Department of Surgery, University of Utah School of Medicine, Salt Lake City
  • 2Department of Biomedical Informatics, University of Utah School of Medicine, Salt Lake City
  • 3Care Transformation, Intermountain Healthcare, Salt Lake City, Utah
  • 4Division of General Surgery, Department of Surgery, University of Utah School of Medicine, Salt Lake City
  • 5Division of Clinical Epidemiology, Department of Medicine, University of Utah School of Medicine, Salt Lake City
  • 6VA Salt Lake City Healthcare System, IDEAS Center 2.0, Salt Lake City, Utah
JAMA Surg. Published online December 26, 2018. doi:10.1001/jamasurg.2018.4874
Key Points

Question  Can computerized provider order entry events be used to identify patients who would need manual medical record review to detect postoperative complications?

Findings  In a cohort study of 21 775 patients who had undergone surgical procedures, the use of computerized provider order entry to screen for postoperative complications appeared to decrease the burden of manual medical record review by 55.4% to 90.3%.

Meaning  Monitoring computerized provider order entry events may augment manual medical record review for selected postoperative complications by excluding from review those patients with low likelihood of complications.


Importance  Conventional approaches for tracking postoperative adverse events requires manual medical record review, thus limiting the scalability of such efforts.

Objective  To determine if a surveillance system using computerized provider order entry (CPOE) events for selected medications as well as laboratory, microbiologic, and radiologic orders can decrease the manual medical record review burden for surveillance of postoperative complications.

Design, Setting, and Participants  This cohort study reviewed the medical records of 21 775 patients who underwent surgical procedures at a university-based tertiary referral center (University of Utah, Salt Lake City) from July 1, 2007, to August 31, 2017. Patients were included if their case was selected for review by a surgical clinical reviewer as part of the National Surgical Quality Improvement Program. Patients were excluded if they had incomplete follow-up data.

Main Outcomes and Measures  Thirty-day postoperative occurrences of superficial surgical site infection, deep surgical site infection, organ space surgical site infection, urinary tract infection, pneumonia, sepsis, septic shock, deep vein thrombosis requiring therapy, and pulmonary embolism, as defined by the National Surgical Quality Improvement Program. A logistic regression model was developed for each postoperative complication using CPOE features as predictors on a development set, and performance was measured on a holdout internal validation set. The models were internally validated using bootstrapping with 10 000 replications to determine the sensitivity, specificity, positive predictive value, and negative predictive value of CPOE-based surveillance system.

Results  The study included 21 775 patients who underwent surgical procedures. Among these patients, 11 855 (54.4%) were women and 9920 (45.6%) were men, with a mean (SD) age of 51.7 (16.8) years. Overall, the prevalence of postoperative complications was low, ranging from 0.2% (pulmonary embolism) to 2.6% (superficial surgical site infection). Use of CPOE events to detect patients who experienced at least 1 complication had a sensitivity of 74.8% (95% CI, 71.1%-78.4%), specificity of 86.8% (95% CI, 85.5%-88.3%), positive predictive value of 33.8% (95% CI, 31.2%-36.4%), negative predictive value of 97.5% (95% CI, 97.1%-97.8%), and area under the curve of 0.808 (95% CI, 0.791-0.824). The negative predictive value for individual complications ranged from 98.7% to 100%. Use of CPOE events to screen for adverse events was estimated to diminish the burden of manual medical record review by 55.4% to 90.3%. A CPOE-based surveillance system performed well for both inpatient and outpatient procedures.

Conclusions and Relevance  A CPOE-based surveillance of postoperative complications has high negative predictive value, which demonstrates that this approach can augment the currently used, resource-intensive manual medical record review process.

Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words