Automated Identification of Postoperative Complications Within an Electronic Medical Record Using Natural Language Processing | Surgery | JAMA | JAMA Network
[Skip to Navigation]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
[Skip to Navigation Landing]
Original Contribution
August 24/31, 2011

Automated Identification of Postoperative Complications Within an Electronic Medical Record Using Natural Language Processing

Author Affiliations

Author Affiliations: Tennessee Valley Healthcare System, Veterans Affairs Medical Center, Nashville (Drs Murff, FitzHenry, Matheny, Dittus, Brown, and Speroff and Ms Gentry); Division of General Internal Medicine and Public Health (Drs Murff, Matheny, Dittus, and Speroff) and Departments of Biomedical Informatics (Drs FitzHenry, Matheny, and Brown) and Biostatistics (Ms Kotter and Drs Crimin and Speroff), Vanderbilt University, Nashville; Center for Organization, Leadership and Management Research, VA Boston Healthcare System, and Department of Health Policy and Management, Boston University School of Public Health, Boston, Massachusetts (Dr Rosen); and Mount Sinai School of Medicine, New York City, New York (Dr Elkin).

JAMA. 2011;306(8):848-855. doi:10.1001/jama.2011.1204

Context Currently most automated methods to identify patient safety occurrences rely on administrative data codes; however, free-text searches of electronic medical records could represent an additional surveillance approach.

Objective To evaluate a natural language processing search–approach to identify postoperative surgical complications within a comprehensive electronic medical record.

Design, Setting, and Patients Cross-sectional study involving 2974 patients undergoing inpatient surgical procedures at 6 Veterans Health Administration (VHA) medical centers from 1999 to 2006.

Main Outcome Measures Postoperative occurrences of acute renal failure requiring dialysis, deep vein thrombosis, pulmonary embolism, sepsis, pneumonia, or myocardial infarction identified through medical record review as part of the VA Surgical Quality Improvement Program. We determined the sensitivity and specificity of the natural language processing approach to identify these complications and compared its performance with patient safety indicators that use discharge coding information.

Results The proportion of postoperative events for each sample was 2% (39 of 1924) for acute renal failure requiring dialysis, 0.7% (18 of 2327) for pulmonary embolism, 1% (29 of 2327) for deep vein thrombosis, 7% (61 of 866) for sepsis, 16% (222 of 1405) for pneumonia, and 2% (35 of 1822) for myocardial infarction. Natural language processing correctly identified 82% (95% confidence interval [CI], 67%-91%) of acute renal failure cases compared with 38% (95% CI, 25%-54%) for patient safety indicators. Similar results were obtained for venous thromboembolism (59%, 95% CI, 44%-72% vs 46%, 95% CI, 32%-60%), pneumonia (64%, 95% CI, 58%-70% vs 5%, 95% CI, 3%-9%), sepsis (89%, 95% CI, 78%-94% vs 34%, 95% CI, 24%-47%), and postoperative myocardial infarction (91%, 95% CI, 78%-97%) vs 89%, 95% CI, 74%-96%). Both natural language processing and patient safety indicators were highly specific for these diagnoses.

Conclusion Among patients undergoing inpatient surgical procedures at VA medical centers, natural language processing analysis of electronic medical records to identify postoperative complications had higher sensitivity and lower specificity compared with patient safety indicators based on discharge coding.