[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.  Flow Diagram
Flow Diagram

Patients taking opioids in the context of cancer treatment and/or for hospice care were excluded prior to group identification. Shading indicates the subset of patients whose medical records were randomly selected for manual review. GMMP indicates Geisinger medication monitoring program; and ICD, International Classification of Diseases, Ninth Revision and/or International Statistical Classification of Diseases and Related Health Problems, Tenth Revision.

Figure 2.  Percentage of Patients With Each Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM) Severity Score From Medical Record Review
Percentage of Patients With Each Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM) Severity Score From Medical Record Review

A, Percentage of patients with each DSM severity score from medical record review for opioid-exposed controls (n = 200) and all patients in the Geisinger medication monitoring program (GMMP) (n = 200). B, Percentage of patients with each DSM severity score from medical record review for those enrolled in the GMMP who maintained the contract (GMMP-M; n = 100) and those enrolled in the GMMP who violated the contract (GMMP-V; n = 100). OUD indicates opioid use disorder.

Table 1.  Electronic Health Record Search Categories Defined for Identifying 9 DSM-5 Criteria for OUD
Electronic Health Record Search Categories Defined for Identifying 9 DSM-5 Criteria for OUD
Table 2.  Demographic and Health Record Summary Data for Health Record Review Subsample
Demographic and Health Record Summary Data for Health Record Review Subsample
Table 3.  Demographic and Health Record Summary Data for Entire Opioid-Using, Geisinger-Specific Medication Monitoring Program and Control Population
Demographic and Health Record Summary Data for Entire Opioid-Using, Geisinger-Specific Medication Monitoring Program and Control Population
1.
Schuchat  A, Houry  D, Guy  GP  Jr.  New data on opioid use and prescribing in the United States.   JAMA. 2017;318(5):425-426. doi:10.1001/jama.2017.8913 PubMedGoogle ScholarCrossref
2.
Seth  P, Scholl  L, Rudd  RA, Bacon  S.  Overdose deaths involving opioids, cocaine, and psychostimulants—United States, 2015-2016.   MMWR Morb Mortal Wkly Rep. 2018;67(12):349-358. doi:10.15585/mmwr.mm6712a1 PubMedGoogle ScholarCrossref
3.
Roxburgh  A, Hall  WD, Gisev  N, Degenhardt  L.  Characteristics and circumstances of heroin and pharmaceutical opioid overdose deaths: comparison across opioids.   Drug Alcohol Depend. 2019;205:107533. doi:10.1016/j.drugalcdep.2019.06.035 PubMedGoogle Scholar
4.
Walley  AY, Bernson  D, Larochelle  MR, Green  TC, Young  L, Land  T.  The contribution of prescribed and illicit opioids to fatal overdoses in Massachusetts, 2013-2015.   Public Health Rep. 2019;134(6):667-674. doi:10.1177/0033354919878429 PubMedGoogle ScholarCrossref
5.
Noto  K, Troxclair  D, Williams  M,  et al.  Surge in fentanyl toxicity deaths in Jefferson Parish, LA, 2013-2018.   Ochsner J. 2019;19(4):314-318. doi:10.31486/toj.19.0039 PubMedGoogle ScholarCrossref
6.
Eeckhaut  MCW, Wagner  J, Neitzke-Spruill  L, Walker  R, Anderson  TL.  Is the gender gap in overdose deaths (still) decreasing? an examination of opioid deaths in Delaware, 2013-2017.   J Stud Alcohol Drugs. 2020;81(1):68-73. doi:10.15288/jsad.2020.81.68 PubMedGoogle ScholarCrossref
7.
Substance Abuse and Mental Health Services Administration. Behavioral health trends in the United States: results from the 2014 National Survey on Drug Use and Health. HHS publication SMA 15-4927, NSDUH series H-50. Published September 2015. Accessed May 30, 2020. https://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf
8.
Cicero  TJ, Kasper  ZA, Ellis  MS.  Increased use of heroin as an initiating opioid of abuse: further considerations and policy implications.   Addict Behav. 2018;87:267-271. doi:10.1016/j.addbeh.2018.05.030 PubMedGoogle ScholarCrossref
9.
Substance Abuse and Mental Health Services Administration.  Key Substance Use and Mental Health Indicators in the United States: Results From the 2018 National Survey on Drug Use and Health. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2019.
10.
Han  B, Compton  WM, Blanco  C, Crane  E, Lee  J, Jones  CM.  Prescription opioid use, misuse, and use disorders in U.S. adults: 2015 National Survey on Drug Use and Health.   Ann Intern Med. 2017;167(5):293-301. doi:10.7326/M17-0865 PubMedGoogle ScholarCrossref
11.
Guy  GP  Jr, Zhang  K.  Opioid prescribing by specialty and volume in the U.S.   Am J Prev Med. 2018;55(5):e153-e155. doi:10.1016/j.amepre.2018.06.008 PubMedGoogle ScholarCrossref
12.
Castro  VM, Minnier  J, Murphy  SN,  et al; International Cohort Collection for Bipolar Disorder Consortium.  Validation of electronic health record phenotyping of bipolar disorder cases and controls.   Am J Psychiatry. 2015;172(4):363-372. doi:10.1176/appi.ajp.2014.14030423 PubMedGoogle ScholarCrossref
13.
Global  RPh. Medical calculators. Accessed July 27, 2020. https://globalrph.com/medcalcs/
14.
Saha  TD, Kerridge  BT, Goldstein  RB,  et al.  Nonmedical prescription opioid use and DSM-5 nonmedical prescription opioid use disorder in the United States.   J Clin Psychiatry. 2016;77(6):772-780. doi:10.4088/JCP.15m10386 PubMedGoogle ScholarCrossref
15.
The R Project for Statistical Computing. Getting started. Accessed July 27, 2020. https://www.r-project.org/
16.
Campbell  CI, Bahorik  AL, VanVeldhuisen  P, Weisner  C, Rubinstein  AL, Ray  GT.  Use of a prescription opioid registry to examine opioid misuse and overdose in an integrated health system.   Prev Med. 2018;110:31-37. doi:10.1016/j.ypmed.2018.01.019 PubMedGoogle ScholarCrossref
17.
Carrell  DS, Cronkite  D, Palmer  RE,  et al.  Using natural language processing to identify problem usage of prescription opioids.   Int J Med Inform. 2015;84(12):1057-1064. doi:10.1016/j.ijmedinf.2015.09.002 PubMedGoogle ScholarCrossref
18.
Palmer  RE, Carrell  DS, Cronkite  D,  et al.  The prevalence of problem opioid use in patients receiving chronic opioid therapy: computer-assisted review of electronic health record clinical notes.   Pain. 2015;156(7):1208-1214. doi:10.1097/j.pain.0000000000000145 PubMedGoogle ScholarCrossref
19.
Goesling  J, Henry  MJ, Moser  SE,  et al.  Symptoms of depression are associated with opioid use regardless of pain severity and physical functioning among treatment-seeking patients with chronic pain.   J Pain. 2015;16(9):844-851. doi:10.1016/j.jpain.2015.05.010 PubMedGoogle ScholarCrossref
20.
Manchikanti  L, Pampati  V, Damron  KS, McManus  CD.  Evaluation of variables in illicit drug use: does a controlled substance abuse screening tool identify illicit drug use?   Pain Physician. 2004;7(1):71-75.PubMedGoogle Scholar
21.
Boscarino  JA, Hoffman  SN, Han  JJ.  Opioid-use disorder among patients on long-term opioid therapy: impact of final DSM-5 diagnostic criteria on prevalence and correlates.   Subst Abuse Rehabil. 2015;6:83-91. doi:10.2147/SAR.S85667PubMedGoogle ScholarCrossref
22.
Brady  KT, McCauley  JL, Back  SE.  Prescription opioid misuse, abuse, and treatment in the United States: an update.   Am J Psychiatry. 2016;173(1):18-26. doi:10.1176/appi.ajp.2015.15020262 PubMedGoogle ScholarCrossref
23.
McHugh  RK, Nielsen  S, Weiss  RD.  Prescription drug abuse: from epidemiology to public policy.   J Subst Abuse Treat. 2015;48(1):1-7. doi:10.1016/j.jsat.2014.08.004 PubMedGoogle ScholarCrossref
24.
Barclay  JS, Owens  JE, Blackhall  LJ.  Screening for substance abuse risk in cancer patients using the Opioid Risk Tool and urine drug screen.   Support Care Cancer. 2014;22(7):1883-1888. doi:10.1007/s00520-014-2167-6 PubMedGoogle ScholarCrossref
25.
Becker  WC, Sullivan  LE, Tetrault  JM, Desai  RA, Fiellin  DA.  Non-medical use, abuse and dependence on prescription opioids among U.S. adults: psychiatric, medical and substance use correlates.   Drug Alcohol Depend. 2008;94(1-3):38-47. doi:10.1016/j.drugalcdep.2007.09.018 PubMedGoogle ScholarCrossref
26.
Han  B, Compton  WM, Jones  CM, Cai  R.  Nonmedical prescription opioid use and use disorders among adults aged 18 through 64 years in the United States, 2003-2013.   JAMA. 2015;314(14):1468-1478. doi:10.1001/jama.2015.11859 PubMedGoogle ScholarCrossref
27.
Cheatle  MD.  Prescription opioid misuse, abuse, morbidity, and mortality: balancing effective pain management and safety.   Pain Med. 2015;16(suppl 1):S3-S8. doi:10.1111/pme.12904 PubMedGoogle ScholarCrossref
28.
Smith  SM, Dart  RC, Katz  NP,  et al; Analgesic, Anesthetic, and Addiction Clinical Trials, Translations, Innovations, Opportunities, and Networks (ACTTION) Public-Private Partnership.  Classification and definition of misuse, abuse, and related events in clinical trials: ACTTION systematic review and recommendations.   Pain. 2013;154(11):2287-2296. doi:10.1016/j.pain.2013.05.053 PubMedGoogle ScholarCrossref
29.
Vowles  KE, McEntee  ML, Julnes  PS, Frohe  T, Ney  JP, van der Goes  DN.  Rates of opioid misuse, abuse, and addiction in chronic pain: a systematic review and data synthesis.   Pain. 2015;156(4):569-576. doi:10.1097/01.j.pain.0000460357.01998.f1 PubMedGoogle ScholarCrossref
30.
Webster  LR.  Risk factors for opioid-use disorder and overdose.   Anesth Analg. 2017;125(5):1741-1748. doi:10.1213/ANE.0000000000002496 PubMedGoogle ScholarCrossref
31.
Pickens  RW, Preston  KL, Miles  DR,  et al.  Family history influence on drug abuse severity and treatment outcome.   Drug Alcohol Depend. 2001;61(3):261-270. doi:10.1016/S0376-8716(00)00146-0 PubMedGoogle ScholarCrossref
32.
Passik  SD.  Issues in long-term opioid therapy: unmet needs, risks, and solutions.   Mayo Clin Proc. 2009;84(7):593-601. doi:10.1016/S0025-6196(11)60748-9 PubMedGoogle ScholarCrossref
33.
Jamison  RN, Mao  J.  Opioid analgesics.   Mayo Clin Proc. 2015;90(7):957-968. doi:10.1016/j.mayocp.2015.04.010 PubMedGoogle ScholarCrossref
34.
Gisev  N, Pearson  S-A, Karanges  EA,  et al.  To what extent do data from pharmaceutical claims under-estimate opioid analgesic utilisation in Australia?   Pharmacoepidemiol Drug Saf. 2018;27(5):550-555. doi:10.1002/pds.4329 PubMedGoogle ScholarCrossref
35.
Karhade  AV, Ogink  PT, Thio  QCBS,  et al.  Machine learning for prediction of sustained opioid prescription after anterior cervical discectomy and fusion.   Spine J. 2019;19(6):976-983. doi:10.1016/j.spinee.2019.01.009 PubMedGoogle ScholarCrossref
36.
Karhade  AV, Cha  TD, Fogel  HA,  et al.  Predicting prolonged opioid prescriptions in opioid-naïve lumbar spine surgery patients.   Spine J. 2020;20(6):888-895. doi:10.1016/j.spinee.2019.12.019 PubMedGoogle ScholarCrossref
37.
Karhade  AV, Ogink  PT, Thio  QCBS,  et al.  Development of machine learning algorithms for prediction of prolonged opioid prescription after surgery for lumbar disc herniation.   Spine J. 2019;19(11):1764-1771. doi:10.1016/j.spinee.2019.06.002 PubMedGoogle ScholarCrossref
38.
Karhade  AV, Schwab  JH, Bedair  HS.  Development of machine learning algorithms for prediction of sustained postoperative opioid prescriptions after total hip arthroplasty.   J Arthroplasty. 2019;34(10):2272-2277.e1. doi:10.1016/j.arth.2019.06.013 PubMedGoogle ScholarCrossref
39.
Shah  AS, Blackwell  RH, Kuo  PC, Gupta  GN.  Rates and risk factors for opioid dependence and overdose after urological surgery.   J Urol. 2017;198(5):1130-1136. doi:10.1016/j.juro.2017.05.037 PubMedGoogle ScholarCrossref
40.
Brummett  CM, Waljee  JF, Goesling  J,  et al.  New persistent opioid use after minor and major surgical procedures in US adults.   JAMA Surg. 2017;152(6):e170504. doi:10.1001/jamasurg.2017.0504 PubMedGoogle Scholar
41.
Calcaterra  SL, Scarbro  S, Hull  ML, Forber  AD, Binswanger  IA, Colborn  KL.  Prediction of future chronic opioid use among hospitalized patients.   J Gen Intern Med. 2018;33(6):898-905. doi:10.1007/s11606-018-4335-8 PubMedGoogle ScholarCrossref
42.
Chartash  D, Paek  H, Dziura  JD,  et al.  Identifying opioid use disorder in the emergency department: multi-system electronic health record-based computable phenotype derivation and validation study.   JMIR Med Inform. 2019;7(4):e15794. doi:10.2196/15794 PubMedGoogle Scholar
43.
Che  Z, St Sauver  J, Liu  H, Liu  Y.  Deep learning solutions for classifying patients on opioid use.   AMIA Annu Symp Proc. 2018;2017:525-534.PubMedGoogle Scholar
44.
Canan  C, Polinski  JM, Alexander  GC, Kowal  MK, Brennan  TA, Shrank  WH.  Automatable algorithms to identify nonmedical opioid use using electronic data: a systematic review.   J Am Med Inform Assoc. 2017;24(6):1204-1210. doi:10.1093/jamia/ocx066 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,227
    Citations 0
    Original Investigation
    Substance Use and Addiction
    September 4, 2020

    Assessment of Probable Opioid Use Disorder Using Electronic Health Record Documentation

    Author Affiliations
    • 1Department of Biomedical Science, Schmidt College of Medicine of Florida Atlantic University, Boca Raton
    • 2Geisinger Clinic, Geisinger, Danville, Pennsylvania
    • 3Department of Molecular and Functional Genomics, Geisinger, Danville, Pennsylvania
    • 4Center for Neurobiology and Behavior, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia
    • 5Department of Biomedical Sciences, Cooper Medical School of Rowan University, Camden, New Jersey
    • 6Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia
    • 7Department of Imaging Science and Innovation, Geisinger, Danville, Pennsylvania
    • 8Neuroscience Institute, Geisinger, Danville, Pennsylvania
    • 9Department of Basic Sciences, Geisinger Commonwealth School of Medicine, Scranton, Pennsylvania
    JAMA Netw Open. 2020;3(9):e2015909. doi:10.1001/jamanetworkopen.2020.15909
    Key Points español 中文 (chinese)

    Question  Are medication monitoring programs within a hospital associated with more accurate identification of patients with opioid use disorder through the use of proxy Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-5) criteria for opioid use disorder extracted from electronic health records?

    Findings  This cross-sectional study demonstrated that DSM-5 criteria for opioid use disorder can be extracted through review of electronic health records and that patients who are part of a drug monitoring program had a higher mean prevalence of opiod use disorder and a higher mean number of psychiatric comorbidities associated with opioid use disorder.

    Meaning  Proxy measures that rely on multiple sources of data, including prescription drug history and notes in the electronic health record, may help identify patients with opioid use disorder who have not received a diagnosis.

    Abstract

    Importance  Electronic health records are a potentially valuable source of information for identifying patients with opioid use disorder (OUD).

    Objective  To evaluate whether proxy measures from electronic health record data can be used reliably to identify patients with probable OUD based on Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-5) criteria.

    Design, Setting, and Participants  This retrospective cross-sectional study analyzed individuals within the Geisinger health system who were prescribed opioids between December 31, 2000, and May 31, 2017, using a mixed-methods approach. The cohort was identified from 16 253 patients enrolled in a contract-based, Geisinger-specific medication monitoring program (GMMP) for opioid use, including patients who maintained or violated contract terms, as well as a demographically matched control group of 16 253 patients who were prescribed opioids but not enrolled in the GMMP. Substance use diagnoses and psychiatric comorbidities were assessed using automated electronic health record summaries. A manual medical record review procedure using DSM-5 criteria for OUD was completed for a subset of patients. The analysis was conducted beginning from June 5, 2017, until May 29, 2020.

    Main Outcomes and Measures  The primary outcome was the prevalence of OUD as defined by proxy measures for DSM-5 criteria for OUD as well as the prevalence of comorbidities among patients prescribed opioids within an integrated health system.

    Results  Among the 16 253 patients enrolled in the GMMP (9309 women [57%]; mean [SD] age, 52 [14] years), OUD diagnoses as defined by diagnostic codes were present at a much lower rate than expected (291 [2%]), indicating the necessity for alternative diagnostic strategies. The DSM-5 criteria for OUD can be assessed using manual medical record review; a manual review of 200 patients in the GMMP and 200 control patients identifed a larger percentage of patients with probable moderate to severe OUD (GMMP, 145 of 200 [73%]; and control, 27 of 200 [14%]) compared with the prevalence of OUD assessed using diagnostic codes.

    Conclusions and Relevance  These results suggest that patients with OUD may be identified using information available in the electronic health record, even when diagnostic codes do not reflect this diagnosis. Furthermore, the study demonstrates the utility of coding for DSM-5 criteria from medical records to generate a quantitative DSM-5 score that is associated with OUD severity.

    Introduction

    Opioid use disorder (OUD) is an epidemic that has been escalating in the United States for the past 2 decades. Although the rate of prescribing opioid analgesics has been decreasing since 2012,1 the number of synthetic opioid–related deaths has been exponentially increasing,2 and this trend is anticipated to continue. Most patients with OUD use heroin and/or fentanyl,3-6 but 50% to 90% of patients with OUD were exposed to a prescription opioid first.7,8 The prevalence estimate of OUD in the US in 2018 is 2 million individuals,9 similar to the previous year’s prevalence estimates.10 However, OUD is likely underdiagnosed within the health system setting. This underdiagnosis may be due, in part, to the reticence of practitioners who lack the specialized training in addiction medicine required to diagnose and treat OUD despite the fact that the most common source of opioid prescriptions is from primary care settings.11

    Typically, OUD is diagnosed during a patient-physician consultation during which the addiction-trained practitioner uses dialogue with the patient or questionnaires to evaluate whether the patient exhibits symptoms of OUD based on Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-5) OUD criteria. These criteria are based on the assessment of whether opioid use causes significant impairment in physical and social functioning, as well as aspects of craving and unsuccessful efforts to reduce or control use. The presentation of 2 or more of the 11 DSM-5 criteria for OUD within a 12-month period warrants an OUD diagnosis. More important, the practitioner typically relies on the self-report of the patient but may consult a significant other or relative of the patient.

    Electronic health record (EHR) data provide a wealth of information, including patients’ previous health care encounters, demographic characteristics, and prescription history. Within an integrated health care system, such as Geisinger, where patients seek primary and specialty care in the same network, these variables could be particularly important to consider for frequently underdiagnosed conditions, such as OUD.

    The goal of this research is to use the comprehensive EHR data of patients who are prescribed opioids to develop proxy measures for the DSM-5 criteria for OUD. To accomplish this, we used EHR data and a contract-based medication monitoring program that exists within Geisinger, a large, integrated health system. We hypothesized that patients who violated the terms of this contract would have more clinical characteristics of OUD compared with those who maintained their contract.

    Methods
    Data Sources and Patient Cohort

    This retrospective, observational, cross-sectional study was implemented in 2 primary groups. We identified a large cohort of patients at Geisinger, an integrated health system in central Pennsylvania that has used the EPIC EHR system since 1996, who were treated with opioids for nonprogressive musculoskeletal pain. A subset of 400 patients was randomly selected from the cohort for a manual medical record review (Figure 1). The medical record review portion of this project was approved as human participant research by the Geisinger Institutional Review Board, and a waiver of Health Insurance Portability and Accountability Act authorization and research consent was granted owing to the retrospective nature of the study, absence of direct risk posed to the participants, and limited nature of the data set (small subsample included in the review). A second part of this study was designed to measure the prevalence of psychiatric and substance use phenotypes in both the medical record review and the larger cohort using automated EHR data extraction methods (ie, International Classification of Diseases, Ninth Revision [ICD-9] and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision [ICD-10] codes). The automated data extraction portion of this project was deemed exempt by the Geisinger Institutional Review Board because all variables were extracted and deidentified using an approved data broker. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Geisinger instituted their own medication monitoring program (GMMP), aimed at standardizing opioid prescribing practices and enhancing patient adherence with medication instructions. The contract requires patients to submit to random urine drug screening tests and to take only the opioid and opioid dose prescribed by the designated Geisinger physician. Although use of the contract is encouraged by all physicians prescribing controlled substances for chronic pain, implementation of the contract is ultimately at the discretion of an individual clinician, and many patients are prescribed opioids without being enrolled in the GMMP. Documentation of program enrollment is recorded in EPIC with a Geisinger-specific code, along with a PDF file of the patient’s signed contract. Patients who violate the terms of the contract can be reported by the clinician and are designated in the EHR with another Geisinger-specific EPIC code. Full details of the contract are provided in the eAppendix in the Supplement. Because the contract is implemented based on physician discretion, there are a substantial number of patients prescribed opioids but not enrolled in the GMMP.

    We analyzed 16 253 individuals aged 18 to 75 years enrolled in the GMMP for an opioid prescription between December 31, 2000, and May 31, 2017, using a mixed-methods approach. A Geisinger data broker was used to identify patients within the GMMP program (based on EPIC coding). The data broker also selected the matched (age, sex, and smoking history) population from a pool of all Geisinger patients who were not part of the GMMP and had at least 2 opioid prescriptions in their medication history. Patients with terminal illness and those with metastatic cancer were excluded from both groups prior to matching. We completed a medical record review on a subset of these patients, including 200 patients from the GMMP group (100 who maintained an active contract [GMMP-M] and 100 who had violated the contract [GMMP-V]) and 200 patients from the control group. In addition to the medical record review, we also assessed the clinical characteristics of both patient cohorts (N = 32 506) using ICD-9 and ICD-10 codes for psychiatric disorders and substance use.

    Outcomes

    Our primary outcome was a quantification of OUD-related symptoms consistent with DSM-5 criteria. Secondarily, we aimed to characterize psychiatric and addiction comorbidities within this population of patients using opioids.

    Demographic Factors and Disease Codes

    The ICD-9 and ICD-10 codes associated with psychiatric disorders (including substance use) were drawn from patient problem lists and clinical encounters. All ICD-9 and ICD-10 codes and their relevant categories used in this research study are in eTable 1 in the Supplement. The ICD-9 and ICD-10 psychiatric codes were categorized into 1 of 6 diagnostic categories: depression, anxiety, tobacco use disorder, alcohol use disorder, OUD, and other substance use disorder. Patients were required to have at least 3 clinical encounters with notation of an ICD-9 or ICD-10 code to be characterized as having that disorder, consistent with previous work using ICD codes for phenotyping in other psychiatric disorders.12 All opioid prescription records were also drawn from the EHR, and the morphine milligram equivalent (MME) dose was calculated using the global rPH calculator.13 See the eAppendix in the Supplement for included medications.

    Medical Record Review Variable Selection and Recording

    We characterized the clinical profile of a subset of patients receiving opioids using a manual medical record review procedure. Detailed information was extracted from the medical record for each patient, including variables associated with the GMMP agreement, such as termination cause, as well as other quantitative phenotypes, such as opioid MME, toxicology screening test pass or fail data, and number of emergency department visits (see eAppendix in the Supplement for search terms). We also adapted criteria from the DSM-514 to diagnose OUD using EHR search and review (Table 1). We excluded criteria associated with opioid tolerance and withdrawal because these criteria are based on physical dependence, which will affect any patient with long-term opioid use. It was then determined by expert clinician review (W.H.B.) whether each medical record review variable was relevant for each of the 9 DSM-5 criteria evaluated here (Table 1). After review of each patient’s medical record, the presence or absence of a given EHR search variable or behavior was recorded. The medical record reviewer was not blinded to the patient’s status in the program because it was not possible given the nature of the comprehensive manual medical record review. A score was generated in which the patient received 1 point for each of the 9 DSM-5 criteria satisfied. These scores were then stratified into the 4 standardized categories from the DSM-5: no OUD (scores <2), mild OUD (scores 2-3), moderate OUD (scores 4-5), and severe OUD (scores ≥6).

    Statistical Analysis

    Statistical analysis was conducted from June 5, 2017, to May 29, 2020. Summary scores were compared between the GMMP and control groups, and within the GMMP group between the GMMP-M and GMMP-V groups. For medical record review comparisons, differences in the frequency of individuals with each OUD severity were assessed using χ2 tests. For both the medical record review cohort and the larger group (N = 32 506), ICD-9 and ICD-10 codes were extracted from the EHR and aggregated across associated psychiatric and addiction categories to limit multiple hypothesis testing and improve power. The percentages of individuals expressing each phenotype variable were compared between the GMMP and control groups with proportion tests using the “stats” package in R Studio.15 All P values were from 2-sided tests, and results were deemed statistically significant at P < .05.

    Results
    OUD Diagnoses

    A total of 16 253 patients (9309 women [57%]; mean [SD] age, 52 [14] years) were enrolled in the GMMP. Among these patients, OUD diagnoses were present at a much lower rate than expected (291 [2%]), indicating the necessity for alternative diagnostic strategies.

    Record Review

    The medical records of 200 patients in the GMMP group (100 in GMMP-M and 100 in GMMP-V) were reviewed for DSM-5 criteria for OUD. The results of the DSM-5 criteria medical record review indicated that 67 of 100 GMMP-M patients whose medical record was reviewed (67%) and 78 of 100 GMMP-V patients whose medical record was reviewed (78%) were classified as having moderate to severe OUD (χ21 = 3.03; n = 200; P = .08). These scores are presented in Figure 2A. Violations of the contract leading to GMMP-V status were predominantly based on failed urine toxicology screening tests that were positive for unprescribed medication (64 of 100 [64%]) or negative for prescribed medications (48 of 100 [48%]), with many patients both positive for unprescribed medications and negative for prescribed medications (35 of 100 [35%]). We also identified many patients in the GMMP-M group who exhibited behaviors that violated contract terms, including urine toxicology screening test results that were positive for unprescribed medications (20 of 100 [20%]), negative for prescribed medications (17 of 100 [17%]), or both (10 of 100 [10%]). More important, 66 of 100 patients (66%) in the GMMP-M group had at least 1 documentation that indicated a contract violation (for numbers of each patient group with each violation and toxicology screening test data, see eTables 3, 4, and 5 in the Supplement). Given the large number of patients who maintained their contract despite behaviors consistent with contract violations, we then collapsed the GMMP-V and GMMP-M populations into 1 GMMP medical record review population (n = 200) and compared that GMMP group with a separate control group of patients who were prescribed opioids but who were not in the GMMP (n = 200). In the control group, 27 patients (14%) were classified as having moderate to severe OUD, compared with 145 patients (73%) in the GMMP population (χ21 = 223.9; n = 400; P < .001; Figure 2B).

    Patients in the combined medical record review GMMP group also had higher rates of depression (82 of 200 [41%] vs 33 of 200 [16%]; P < .001) and anxiety (92 of 200 [46%] vs 39 of 200 [20%]; P < .001), as well as greater nicotine use (85 of 200 [42%] vs 53 of 200 [26%]; P < .001), opioid use (6 of 200 [3%] vs 0; P = .04), and other substance use (9 of 200 [4%] vs 1 of 200 [1%]; P = .03) relative to the control population (Table 2). In addition, patients in the combined GMMP group sought out treatment in the emergency department more frequently than patients in the control group (13 vs 5 visits; P = .007).

    Large Cohort Phenotype Comparisons of ICD-9 and ICD-10 Codes

    Since the medical record review results indicated that a substantial percentage of patients in the GMMP group showed signs of OUD, we compared demographic differences and psychiatric and addiction diagnosis differences between the entire GMMP population (n = 16 253) and controls (n = 16 253) (Table 3). Within the combined GMMP group, we found higher rates of depression (5446 [34%] vs 1473 [9%]; P < .001) and anxiety (6552 [40%] vs 1605 [10%]; P < .001). We also found higher rates of tobacco use (4760 [29%] vs 1523 [9%]; P < .001), alcohol use (489 [3%] vs 137 [1%]; P < .001), and other substance use (570 [4%] vs 106 [1%]; P < .001) compared with the opioid-treated control group. Similar differences in psychiatric and substance use codes were observed in the subset of patients assessed using the medical record review procedure (Table 2) and when comparing the larger group of patients in the GMMP-V group with the patients in the GMMP-M group (eTables 2 and 3 in the Supplement).

    Discussion

    We found that patients enrolled in a contract-based, health system–specific drug monitoring program showed higher rates of OUD based on a medical record review procedure that adapted DSM-5 interview criteria. We also observed that when patients are appropriately documented as having violated the terms of the contract with an EPIC code, this code can be a useful proxy for OUD diagnosis. This finding is consistent with previous work that demonstrated the utility of a prescription monitoring program.16

    We used a manual medical record review procedure but searched for consistent and predefined search terms. This method is different from natural language processing algorithms, as a human reviewer can interpret whether certain search terms are appropriate in a given context. For example, for “substance abuse mentioned,” we included search terms such as “abuse” and “high.” A human reviewer can evaluate if those search terms appear in a context relevant to drug abuse. These search terms could serve as the basis of future natural language processing algorithms and would improve the scalability of this method. Future work may also benefit from combining search terms and ICD codes, as Carell et al17 reported that the combined use of ICD codes and natural language processing data were more effective in identifying OUD than either method alone. Other work using natural language processing has shown that more than one-third of patients with inappropriate opioid use in the setting of chronic pain did not have an ICD code associated with their opioid misuse.18 This finding is consistent with our finding that very few patients had ICD-9 or ICD-10 codes for OUD.

    We also demonstrate that psychiatric and other substance use codes are increased in patients in the drug monitoring program. This finding is consistent with epidemiologic data of OUD14 and other work in chronic pain populations.19 These results suggest the potential for assessing psychiatric and other substance use codes as an associated factor to evaluate patient risk for OUD in the chronic pain setting. Others have also confirmed the utility of assessing EHRs of patient populations to identify risk factors, such as comorbidities and illicit drug use, associated with opioid misuse and overdose.16,20

    Studies have described clinical characteristics of patients with pain who are at risk for developing prescription opioid addiction in the context of chronic opioid treatment of persistent pain.21,22 Factors associated with increased risk of OUD in cross-sectional studies include younger age, male sex, European ancestry, comorbid psychiatric disorders, higher MME, lower socioeconomic and educational achievement, exposure to violence or sexual assault, inability to work owing to disability, and a personal or family history of substance use disorders.21-26 A previous study of OUD among 705 Geisinger patients taking prescription opioids for at least 5 months in a 12-month period for noncancer pain revealed that the characteristics most frequently associated with severe OUD are age younger than 65 years, current pain impairment, sleep disturbance, suicidal thoughts, anxiety disorders, and history of substance abuse with treatment.21 In that study, approximately 13% of these 705 patients satisfied psychiatric-based criteria for moderate to severe OUD.

    Data from the present study are also relevant to the overall risk for developing OUD when being treated for chronic pain. Estimates for OUD in the context of treatment for chronic pain vary widely, with estimates ranging from less than 10% to more than 25%.21,27-29 Our estimate of OUD in controls treated with opioids (14%) is consistent with the lower end of this estimate but is much higher for those in the GMMP (73%). This difference may be, in part, because physicians are more likely to refer someone to the GMMP based on an increased suspicion that the prescribed opioid is being misused or owing to closer monitoring of patients once they are enrolled in the program. There may also be a bias toward identifying OUD once the patient is in the program given the increased monitoring of program participants.

    Limitations

    This study has some limitations. It relies on EHR data, which are dependent on physicians’ entries; thus, the information may not be standardized or available for all candidates. Some relevant items might also not be included in the EHR. For example, family history of substance use disorders is poorly captured despite being a known risk factor for developing OUD.30-33 We also did not assess differences in nonpsychiatric comorbidities, such as pain. Future work would benefit from assessing whether certain types of pain are associated with increased risk for OUD. In addition, this study was conducted in a single health care system and thus may have certain population characteristics that are unique and may be associated with the results of the study. Future work should explore the validity of applying this medical record review estimation of OUD severity in other health systems.

    Prescription and claims data are also thought to underestimate opioid use.34 These data might result in underestimates of MME for patients who are seeking additional prescriptions outside of Geisinger. Others have been successful in incorporating prescription data from national databases to assess more comprehensive OUD phenotypes surrounding prolonged use of opioids in the context of chronic pain and after surgery.18,35-40 Future work should incorporate additional prescription databases, when available. At the time of this study, the Pennsylvania State Drug Monitoring Program was not available for research use.

    Some of the DSM-5 criteria used in this study may not map directly from a more traditional interview format to those adapted to available EHR data. For example, we used disability status as a proxy for DSM-5 criteria including “work/school impact,” “interpersonal impact,” and “reduced activities because of use.” A person can become disabled for many reasons, including chronic pain and/or other injury not associated with opioid misuse. We chose to generate severity scores that would closely mirror the traditional interview format, but future iterations of this work may find it more useful to characterize severity based on the presence of an individual search term or a subset of search terms.

    Many of the individuals in the GMMP program had diagnosis codes for other substance use disorders and other psychiatric disorders. Assessing whether a psychiatric illness came before or after opioid use is challenging with EHR data, as the date of the first recorded diagnosis does not necessarily co-occur with the onset of the disorder. Future work should explore the temporality of medication use and diagnoses and/or explore different trajectories associated with distinct patient subgroups.

    Other work has used ICD codes and other EHR data to characterize various opioid use phenotypes. For example, health record data have been used to assess which patients go on to develop chronic use of opioids using opioid-related search terms, emergency department chief symptoms, prescription history, and other variables.41-44 Future work is needed to develop the most optimized algorithm for use across multiple health records.

    Conclusions

    We show in the present study that EHRs can be used to derive DSM-5 severity scores for OUD. Our methods are unique in deriving a severity score that aims to mirror severity scores from more traditional interview-based diagnostic procedures, but results are also consistent with previous work examining OUD in EHR data. Thus, this study contributes to the growing body of knowledge that emphasizes the utility of EHRs to evaluate a patient’s status or potential for opioid or other substance misuse. Opioids continue to be used for the treatment of pain. Precision medicine within integrated health systems such as Geisinger could be a major associated factor in developing more efficient pain treatments with less risk for addiction, and studies of this potential could be helped by establishing more effective proxy measures for OUD using EHR data.

    Back to top
    Article Information

    Accepted for Publication: June 8, 2020.

    Published: September 4, 2020. doi:10.1001/jamanetworkopen.2020.15909

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Palumbo SA et al. JAMA Network Open.

    Corresponding Author: Vanessa Troiani, PhD, Geisinger Clinic, 120 Hamm Dr, Ste 2A, Lewisburg, PA 17837 (vtroiani@geisinger.edu).

    Author Contributions: Dr Troiani had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Metpally, Crist, Ferraro, Berrettini, Robishaw, Troiani.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Palumbo, Krishnamurthy, Young, Ferraro, Berrettini, Robishaw, Troiani.

    Critical revision of the manuscript for important intellectual content: Palumbo, Adamson, Manoharan, Beiler, Seiwell, Metpally, Crist, Doyle, Ferraro, Li, Berrettini, Robishaw, Troiani.

    Statistical analysis: Palumbo, Krishnamurthy, Metpally, Li, Troiani.

    Obtained funding: Berrettini, Robishaw, Troiani.

    Administrative, technical, or material support: Adamson, Manoharan, Beiler, Seiwell, Young, Robishaw, Troiani.

    Supervision: Metpally, Berrettini, Robishaw, Troiani.

    Conflict of Interest Disclosures: Ms Palumbo reported receiving grants from the National Institutes of Health during the conduct of the study. Dr Ferraro reported receiving grants from the National Institute of Drug Abuse and the New Jersey Department of Health during the conduct of this study. Dr Berrettini reported receiving grants from the National Institute of Drug Abuse, Pennsylvania Department of Health, and National Institute of Mental Health during the conduct of this study and serving as a consultant for Geisinger. Dr Robishaw reported receiving grants from the National Institute of Drug Abuse, National Institute of General Medical Sciences, National Institute of Aging, and National Heart, Lung, and Blood Institute during the conduct of this study. Dr Troiani reported receiving grants from the National Institute of Drug Abuse, Geisinger Health Plan, and Pennsylvania Department of Health during the conduct of this study. Ms Adamson, Mr Krishnamurthy, Ms Beiler, Mr Seiwell, Mr Young, and Dr Metpally were all supported by grants from the National Institute of Drug Abuse, Geisinger Health Plan, and Pennsylvania Department of Health during the conduct of this study. Drs Crist, Doyle, and Li were supported by the National Institute of Drug Abuse, Pennsylvania Department of Health, and National Institute of Mental Health during the conduct of this study. No other disclosures were reported.

    Funding/Support: This study was supported by the National Institute of Drug Abuse under award DA404415 to Drs Berrettini, Robishaw, and Troiani and by a Commonwealth of Pennsylvania Department of Health grant to Dr Berrettini. This research was funded by grant R01 DA044015 from the National Institutes of Health to Drs Robishaw, Berrettini, and Troiani and by a Commonwealth of Pennsylvania Department of Health grant to Dr Berrettini.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: Joseph Chronowski, MBA, Geisinger Clinic, assisted with morphine milligram equivalent calculations from prescription data. He was compensated for his contribution.

    References
    1.
    Schuchat  A, Houry  D, Guy  GP  Jr.  New data on opioid use and prescribing in the United States.   JAMA. 2017;318(5):425-426. doi:10.1001/jama.2017.8913 PubMedGoogle ScholarCrossref
    2.
    Seth  P, Scholl  L, Rudd  RA, Bacon  S.  Overdose deaths involving opioids, cocaine, and psychostimulants—United States, 2015-2016.   MMWR Morb Mortal Wkly Rep. 2018;67(12):349-358. doi:10.15585/mmwr.mm6712a1 PubMedGoogle ScholarCrossref
    3.
    Roxburgh  A, Hall  WD, Gisev  N, Degenhardt  L.  Characteristics and circumstances of heroin and pharmaceutical opioid overdose deaths: comparison across opioids.   Drug Alcohol Depend. 2019;205:107533. doi:10.1016/j.drugalcdep.2019.06.035 PubMedGoogle Scholar
    4.
    Walley  AY, Bernson  D, Larochelle  MR, Green  TC, Young  L, Land  T.  The contribution of prescribed and illicit opioids to fatal overdoses in Massachusetts, 2013-2015.   Public Health Rep. 2019;134(6):667-674. doi:10.1177/0033354919878429 PubMedGoogle ScholarCrossref
    5.
    Noto  K, Troxclair  D, Williams  M,  et al.  Surge in fentanyl toxicity deaths in Jefferson Parish, LA, 2013-2018.   Ochsner J. 2019;19(4):314-318. doi:10.31486/toj.19.0039 PubMedGoogle ScholarCrossref
    6.
    Eeckhaut  MCW, Wagner  J, Neitzke-Spruill  L, Walker  R, Anderson  TL.  Is the gender gap in overdose deaths (still) decreasing? an examination of opioid deaths in Delaware, 2013-2017.   J Stud Alcohol Drugs. 2020;81(1):68-73. doi:10.15288/jsad.2020.81.68 PubMedGoogle ScholarCrossref
    7.
    Substance Abuse and Mental Health Services Administration. Behavioral health trends in the United States: results from the 2014 National Survey on Drug Use and Health. HHS publication SMA 15-4927, NSDUH series H-50. Published September 2015. Accessed May 30, 2020. https://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf
    8.
    Cicero  TJ, Kasper  ZA, Ellis  MS.  Increased use of heroin as an initiating opioid of abuse: further considerations and policy implications.   Addict Behav. 2018;87:267-271. doi:10.1016/j.addbeh.2018.05.030 PubMedGoogle ScholarCrossref
    9.
    Substance Abuse and Mental Health Services Administration.  Key Substance Use and Mental Health Indicators in the United States: Results From the 2018 National Survey on Drug Use and Health. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2019.
    10.
    Han  B, Compton  WM, Blanco  C, Crane  E, Lee  J, Jones  CM.  Prescription opioid use, misuse, and use disorders in U.S. adults: 2015 National Survey on Drug Use and Health.   Ann Intern Med. 2017;167(5):293-301. doi:10.7326/M17-0865 PubMedGoogle ScholarCrossref
    11.
    Guy  GP  Jr, Zhang  K.  Opioid prescribing by specialty and volume in the U.S.   Am J Prev Med. 2018;55(5):e153-e155. doi:10.1016/j.amepre.2018.06.008 PubMedGoogle ScholarCrossref
    12.
    Castro  VM, Minnier  J, Murphy  SN,  et al; International Cohort Collection for Bipolar Disorder Consortium.  Validation of electronic health record phenotyping of bipolar disorder cases and controls.   Am J Psychiatry. 2015;172(4):363-372. doi:10.1176/appi.ajp.2014.14030423 PubMedGoogle ScholarCrossref
    13.
    Global  RPh. Medical calculators. Accessed July 27, 2020. https://globalrph.com/medcalcs/
    14.
    Saha  TD, Kerridge  BT, Goldstein  RB,  et al.  Nonmedical prescription opioid use and DSM-5 nonmedical prescription opioid use disorder in the United States.   J Clin Psychiatry. 2016;77(6):772-780. doi:10.4088/JCP.15m10386 PubMedGoogle ScholarCrossref
    15.
    The R Project for Statistical Computing. Getting started. Accessed July 27, 2020. https://www.r-project.org/
    16.
    Campbell  CI, Bahorik  AL, VanVeldhuisen  P, Weisner  C, Rubinstein  AL, Ray  GT.  Use of a prescription opioid registry to examine opioid misuse and overdose in an integrated health system.   Prev Med. 2018;110:31-37. doi:10.1016/j.ypmed.2018.01.019 PubMedGoogle ScholarCrossref
    17.
    Carrell  DS, Cronkite  D, Palmer  RE,  et al.  Using natural language processing to identify problem usage of prescription opioids.   Int J Med Inform. 2015;84(12):1057-1064. doi:10.1016/j.ijmedinf.2015.09.002 PubMedGoogle ScholarCrossref
    18.
    Palmer  RE, Carrell  DS, Cronkite  D,  et al.  The prevalence of problem opioid use in patients receiving chronic opioid therapy: computer-assisted review of electronic health record clinical notes.   Pain. 2015;156(7):1208-1214. doi:10.1097/j.pain.0000000000000145 PubMedGoogle ScholarCrossref
    19.
    Goesling  J, Henry  MJ, Moser  SE,  et al.  Symptoms of depression are associated with opioid use regardless of pain severity and physical functioning among treatment-seeking patients with chronic pain.   J Pain. 2015;16(9):844-851. doi:10.1016/j.jpain.2015.05.010 PubMedGoogle ScholarCrossref
    20.
    Manchikanti  L, Pampati  V, Damron  KS, McManus  CD.  Evaluation of variables in illicit drug use: does a controlled substance abuse screening tool identify illicit drug use?   Pain Physician. 2004;7(1):71-75.PubMedGoogle Scholar
    21.
    Boscarino  JA, Hoffman  SN, Han  JJ.  Opioid-use disorder among patients on long-term opioid therapy: impact of final DSM-5 diagnostic criteria on prevalence and correlates.   Subst Abuse Rehabil. 2015;6:83-91. doi:10.2147/SAR.S85667PubMedGoogle ScholarCrossref
    22.
    Brady  KT, McCauley  JL, Back  SE.  Prescription opioid misuse, abuse, and treatment in the United States: an update.   Am J Psychiatry. 2016;173(1):18-26. doi:10.1176/appi.ajp.2015.15020262 PubMedGoogle ScholarCrossref
    23.
    McHugh  RK, Nielsen  S, Weiss  RD.  Prescription drug abuse: from epidemiology to public policy.   J Subst Abuse Treat. 2015;48(1):1-7. doi:10.1016/j.jsat.2014.08.004 PubMedGoogle ScholarCrossref
    24.
    Barclay  JS, Owens  JE, Blackhall  LJ.  Screening for substance abuse risk in cancer patients using the Opioid Risk Tool and urine drug screen.   Support Care Cancer. 2014;22(7):1883-1888. doi:10.1007/s00520-014-2167-6 PubMedGoogle ScholarCrossref
    25.
    Becker  WC, Sullivan  LE, Tetrault  JM, Desai  RA, Fiellin  DA.  Non-medical use, abuse and dependence on prescription opioids among U.S. adults: psychiatric, medical and substance use correlates.   Drug Alcohol Depend. 2008;94(1-3):38-47. doi:10.1016/j.drugalcdep.2007.09.018 PubMedGoogle ScholarCrossref
    26.
    Han  B, Compton  WM, Jones  CM, Cai  R.  Nonmedical prescription opioid use and use disorders among adults aged 18 through 64 years in the United States, 2003-2013.   JAMA. 2015;314(14):1468-1478. doi:10.1001/jama.2015.11859 PubMedGoogle ScholarCrossref
    27.
    Cheatle  MD.  Prescription opioid misuse, abuse, morbidity, and mortality: balancing effective pain management and safety.   Pain Med. 2015;16(suppl 1):S3-S8. doi:10.1111/pme.12904 PubMedGoogle ScholarCrossref
    28.
    Smith  SM, Dart  RC, Katz  NP,  et al; Analgesic, Anesthetic, and Addiction Clinical Trials, Translations, Innovations, Opportunities, and Networks (ACTTION) Public-Private Partnership.  Classification and definition of misuse, abuse, and related events in clinical trials: ACTTION systematic review and recommendations.   Pain. 2013;154(11):2287-2296. doi:10.1016/j.pain.2013.05.053 PubMedGoogle ScholarCrossref
    29.
    Vowles  KE, McEntee  ML, Julnes  PS, Frohe  T, Ney  JP, van der Goes  DN.  Rates of opioid misuse, abuse, and addiction in chronic pain: a systematic review and data synthesis.   Pain. 2015;156(4):569-576. doi:10.1097/01.j.pain.0000460357.01998.f1 PubMedGoogle ScholarCrossref
    30.
    Webster  LR.  Risk factors for opioid-use disorder and overdose.   Anesth Analg. 2017;125(5):1741-1748. doi:10.1213/ANE.0000000000002496 PubMedGoogle ScholarCrossref
    31.
    Pickens  RW, Preston  KL, Miles  DR,  et al.  Family history influence on drug abuse severity and treatment outcome.   Drug Alcohol Depend. 2001;61(3):261-270. doi:10.1016/S0376-8716(00)00146-0 PubMedGoogle ScholarCrossref
    32.
    Passik  SD.  Issues in long-term opioid therapy: unmet needs, risks, and solutions.   Mayo Clin Proc. 2009;84(7):593-601. doi:10.1016/S0025-6196(11)60748-9 PubMedGoogle ScholarCrossref
    33.
    Jamison  RN, Mao  J.  Opioid analgesics.   Mayo Clin Proc. 2015;90(7):957-968. doi:10.1016/j.mayocp.2015.04.010 PubMedGoogle ScholarCrossref
    34.
    Gisev  N, Pearson  S-A, Karanges  EA,  et al.  To what extent do data from pharmaceutical claims under-estimate opioid analgesic utilisation in Australia?   Pharmacoepidemiol Drug Saf. 2018;27(5):550-555. doi:10.1002/pds.4329 PubMedGoogle ScholarCrossref
    35.
    Karhade  AV, Ogink  PT, Thio  QCBS,  et al.  Machine learning for prediction of sustained opioid prescription after anterior cervical discectomy and fusion.   Spine J. 2019;19(6):976-983. doi:10.1016/j.spinee.2019.01.009 PubMedGoogle ScholarCrossref
    36.
    Karhade  AV, Cha  TD, Fogel  HA,  et al.  Predicting prolonged opioid prescriptions in opioid-naïve lumbar spine surgery patients.   Spine J. 2020;20(6):888-895. doi:10.1016/j.spinee.2019.12.019 PubMedGoogle ScholarCrossref
    37.
    Karhade  AV, Ogink  PT, Thio  QCBS,  et al.  Development of machine learning algorithms for prediction of prolonged opioid prescription after surgery for lumbar disc herniation.   Spine J. 2019;19(11):1764-1771. doi:10.1016/j.spinee.2019.06.002 PubMedGoogle ScholarCrossref
    38.
    Karhade  AV, Schwab  JH, Bedair  HS.  Development of machine learning algorithms for prediction of sustained postoperative opioid prescriptions after total hip arthroplasty.   J Arthroplasty. 2019;34(10):2272-2277.e1. doi:10.1016/j.arth.2019.06.013 PubMedGoogle ScholarCrossref
    39.
    Shah  AS, Blackwell  RH, Kuo  PC, Gupta  GN.  Rates and risk factors for opioid dependence and overdose after urological surgery.   J Urol. 2017;198(5):1130-1136. doi:10.1016/j.juro.2017.05.037 PubMedGoogle ScholarCrossref
    40.
    Brummett  CM, Waljee  JF, Goesling  J,  et al.  New persistent opioid use after minor and major surgical procedures in US adults.   JAMA Surg. 2017;152(6):e170504. doi:10.1001/jamasurg.2017.0504 PubMedGoogle Scholar
    41.
    Calcaterra  SL, Scarbro  S, Hull  ML, Forber  AD, Binswanger  IA, Colborn  KL.  Prediction of future chronic opioid use among hospitalized patients.   J Gen Intern Med. 2018;33(6):898-905. doi:10.1007/s11606-018-4335-8 PubMedGoogle ScholarCrossref
    42.
    Chartash  D, Paek  H, Dziura  JD,  et al.  Identifying opioid use disorder in the emergency department: multi-system electronic health record-based computable phenotype derivation and validation study.   JMIR Med Inform. 2019;7(4):e15794. doi:10.2196/15794 PubMedGoogle Scholar
    43.
    Che  Z, St Sauver  J, Liu  H, Liu  Y.  Deep learning solutions for classifying patients on opioid use.   AMIA Annu Symp Proc. 2018;2017:525-534.PubMedGoogle Scholar
    44.
    Canan  C, Polinski  JM, Alexander  GC, Kowal  MK, Brennan  TA, Shrank  WH.  Automatable algorithms to identify nonmedical opioid use using electronic data: a systematic review.   J Am Med Inform Assoc. 2017;24(6):1204-1210. doi:10.1093/jamia/ocx066 PubMedGoogle ScholarCrossref
    ×