Assessment of Potentially Inappropriate Prescribing of Opioid Analgesics Requiring Prior Opioid Tolerance

Key Points Question What proportion of patients are opioid-tolerant when starting opioids that require prior tolerance for safe use? Findings In this cohort study including 153 385 use episodes of opioid analgesics labeled only for use in people who are opioid-tolerant, more than half of patients who received these medications showed no evidence of prior opioid tolerance in health insurance claims or electronic health records. Meaning These findings suggest that many patients may be at risk of potentially serious adverse events owing to widespread use of certain extended-release opioids without prior opioid tolerance.


Episode definition
OTO episodes were defined as follows: For all opioids prescribed at doses requiring prior tolerance, an episode is defined as: start date = fill date; and end date = [(fill date + days' supply) -1].
For episodes of use of extended release oxycodone, the episode start date was defined as the first the date where cumulative daily dosage exceeded 80 mg or drug strength was equal to 60 mg or 80 mg. The episode end date was defined as the latest date where the cumulative daily dosage exceeded 80 mg or the runout of the claim of the 60 mg or 80 mg fill.

Assumptions Regarding Days' Supply: claims data analysis
Because prescription claims indicate only that medications were retrieved from the pharmacy, with no definitive information about patient behavior in taking these medications, the following assumptions were made: • Assumption 1: The quantity of medication dispensed was consumed equally over the days' supplied from pharmacy claims • Assumption 2: Patients began taking medications on the fill date. This analysis was conducted in the OLDW environment using claims data. The unit of observation was an OTO episode that met the following criteria: • The enrollee had both medical and pharmacy benefits and was enrolled in a Medicare Advantage or commercial plan.
• 6 months (183 days) of continuous enrollment in medical and pharmacy benefits prior to and including date of the OTO prescription • No evidence of an OTO prescription of the same type during the 183 days prior to OTO episode (that is, the episode is incident) • Episodes (not individuals) were excluded for any of the following reasons: -Quantity or days' supplied on the qualifying claim was ≤0. -The enrollee has a missing or unknown age, gender, insurance type, or

Structured data analyses:
In general, the analyses using EHR data followed the same procedures as the claims data analyses. There were two key exceptions, described here:

Natural language processing analysis
Because up to 80% of the information contained in the EHR/ clinical record is stored as free text in clinical notes, the analysis that included free-text EHR notes (henceforth "free-text analysis") utilized natural language processing --a branch of artificial intelligence that helps computers understand, interpret, and manipulate human languageand applied it to help evaluate the content and meaning of clinical notes. The goal was to determine whether there is additional evidence of opioid tolerance in clinical notes, in addition to that seen in claims or EHR prescribing behavior-that is, potential reasons the clinicians may have prescribed OTO medications to people who were not opioid-tolerant.
The free-text analysis occurred behind an "identifiable" data firewall and had to be conducted without access to claims data. OTO episodes and tolerance were identified according to the data available behind that firewall.
The general approach for cohort definition for the free-text analysis was to create sets of notes highly distinguishable from one another that are sound, though not necessarily complete due to the "leakage" in the EHR described above under the heading Description of Electronic

Medical Recorddata
Notes for each OTO event for which notes were available were combined for the 30 days preceding the OTO into a single large text block to use as a record of patient history and encounters.

Definitions
Standard access to the EHR data included in the OLDW does not allow direct access to the clinical notes. The free-text analysis occurred behind a firewall without access to claims and used somewhat different procedures to identify OTO episodes. Specifically: • Extended-release (ER) and immediate-release (IR) opioids were identified using a list of NDC codes from the CDC(1) • OTO Episodes were identified using EHR prescriptions written data showing prescribed OTO medication (ER oxycodone; ER hydromorphone; fentanyl • Opioid exposure was used as a proxy for opioid tolerance and defined as any opioid exposure in the 30 days preceding an OTO episode • Opioid exposed group -Those given an ER opioid after a having an IR opioid in the prior 30 days are the exposed group • Opioid unexposed groupno opioid use of any kind identified in the EHR in the 30 days prior to the OTO episode Topics: Clusters of terms that: • frequently occur together or • frequently occur in similar patterns

Opioid Exposure and Clean Periods
We used a conservative approach for identifying notes for those in the exposed and unexposed groups. Recognizing constraints imposed on the analyses resulting from the diversity of the structured data collection (data is sourced from many health systems and many EHR instances from several EHR vendors), the range of documentation processes of the contributing client organizations, and the potential incompleteness of any patient's clinical history beyond a single encounter, we used the following conservative definitions: • "Clean" periods were defined as 183 days with no OTO exposure (based on NDC codes), either prescription or administration. This conservative approach of a 183 day time frame was used to be consistent with the claims and claims + EHR structured fields analyses as well as to counteract potential misclassification of the data.
-Days' supply was often missing for prescription data so we assumed a prescription could be up to 90 days.
© 2020 Jeffery MM et al. JAMA Network Open.
-It was not always possible to determine whether refills occurred -The assumption was that a 183 day clean period without any OTO would filter out any prior tolerance • We used NDC codes to identify OTO drugs ER hydromorphone, fentanyl transdermal system, transmucosal fentanyl, and ER oxycodone • Episodes are based only prescriptions written. Medications administered in a hospital setting were not used to identify OTO medication episodes but were used to identify tolerance (exposure).
The first step of the NLP work was to create a meaningful set (or sets) of case (exposed) and control (unexposed) episodes that could be compared. The definitions need to be precise enough to allow a small signal of tolerance to emerge from all the text that was available at the episode level. Using too many notes could have the potential to dilute this signal to the point of non-detection. Because the choice of the case/control definitions was critical to the success of the project, we used a narrow definition as a first experiment, and then expanded in subsequent iterations. Because medication dose and duration cannot be utilized to create tolerance definitions similar to those used in the claims analysis, we defined prior opioid use in this phase as "opioid exposure" rather than opioid tolerance. We extracted the cohort of OTO episodes then divided the cohort into opioid-exposed or "cases" and opioid-unexposed or "control" groups based on whether they had any opioid exposure in the 30 days preceding the OTO episode date.
• OTO episodes were defined as an OTO prescription written, preceded by 183 days free of prior OTO prescriptions or administration • Case/Exposed group: Those with any IR opioid exposure in the 30 days prior to the OTO date were "cases" © 2020 Jeffery MM et al. JAMA Network Open.
• Unexposed group: Those who were completely opioid naïve in the 183 days prior to the OTO were "controls" The cohorts were developed using structured EHR data for prescribed and administered drugs.

Free-text Analysis Study Sample
The

Natural Language Processing Steps
Clinical notes differ from many types of standard text and require pre-processing to clean the notes (e.g., remove white space and punctuation, identify and remove template language and copy-paste of previous comments), prior to use. Standard pre-processing steps were followed, see Data Cleaning section below for detail.
We used the vector space model, also known as the "bag-of-words" approach, which is a commonly used method of document classification. In this method the frequency with which each word occurs is used as a feature for training for the classifier to identify and categorize the words. A list of words and terms was created, and the number of times each term appeared in each document patient note was counted. See Term Vector Creation section below for examples.
The patient-by-patient counts of how often each term appears in each patient's notes are scaled by how many other patients also had that term. This scaling makes the data more precise as a term with a high frequency may be common but not be very specific. The goal was to identify specific words that appear in relatively few documents, but seem important where they do occur (i.e., they occur frequently). The term frequency (TF) is multiplied by its inverse document frequency (IDF) to arrive at the final number used to create the topic. Words or terms were dropped if they occurred in a) less than 1% of notes and were considered rare or b) more than 80% of notes and were considered common.
We used both a combination of machine learning, allowing the computer algorithm to determine the words, and a guided approach where we searched for particular words. The words we specified in the guided approach were based on input from our technical expert panel.

Topic Modeling
For notes analysis we utilized non-negative matrix factorization, a non-parametric method that works well with the bag-of-words approach. The parameters are set to use regularization to reduce topic noise. We used topic modeling, a type of statistical model for discovering the abstract "topics" that occur in a collection of documents that is frequently used in machine learning and NLP. This method is an often used text-mining tool for discovery of hidden semantic structures in a text body. Terms were combined into clusters, called "topics," and those topics were then weighted and assigned to relevant notes for analysis.
The topics were created by the following steps: • Identify a cohort of patients with OTO episodes • Extract 30 days of notes prior to the OTO fill date • Use NLP algorithm to group the terms found in those notes into "topics" which are clusters of terms or words that either frequently appear together or that occur in similar contexts, seeming to act as synonyms; no human input goes into the creation of topics (i.e., it is unsupervised machine learning) © 2020 Jeffery MM et al. JAMA Network Open.
• Identify which topics are important in each patients' notes (unsupervised) • Review the topics for meaning with the technical expert panel, describing what the topic represents (e.g., "palliative care" or "surgery") • Combine the NLP topics with the claims data to determine which topics discriminated between episodes with evidence of tolerance in claims data and those without evidence of tolerance in claims data The final model specified 100 topics; we also ran models of 50 topics and 200 topics. The 50topic model resulted in topics that were too general, while the 200-topic model resulted in topics that were too similar to each other. Words that appeared on less than 1% of the notes were dropped, as were topics with an excessive number of terms (top 10 th percentile of term counts).
Once topics were created they were weighted and assigned back to the relevant notes. After these adjustments, there were 66 topics in the topic model. See Error! Reference source not found. for the complete list of 100 topics.

Technical Expert Panel (TEP) Input
After initial exploratory work in NLP we presented preliminary results to our TEP and solicited input. Specific questions (see below) were asked of the TEP as a way to help us understand clinical relevance and clinician documentation habits related to OTO episodes: • What sorts of complaints, concerns, and scenarios might a physician hear from a patient or caregiver that would lead him/her to consider prescribing an OTO formulation?
• What questions might a physician ask of patients or caregivers that would probe for the OTO being a possible right fit for the individual patient's situation?
© 2020 Jeffery MM et al. JAMA Network Open.
• What documentation might exist in progress notes (including labs, clinical indicators, pain level, pain management, medication history) that would help identify the physician's rationale for prescribing?
• What are the specific phrases and terms the notes would have?
In addition to reviewing our topics and providing additional clinical terms that might serve as evidence of tolerance, the experts also suggested we looked for explanatory topics that might provide insight into the physician's rationale for why labelling instructions were not followed. The reasoning might not represent appropriate or safe clinical care. The goal was simply to assess possible explanations for the prescribing.
The TEP suggested a number of reasons for prescribing OTO medication in those without prior opioid tolerance including: •

Term Matrix
Finally, a term matrix including both clinical and whitelist terms was created within the secure NLP area and then transferred back into OLDW where it could be merged in with claims and structured EHR data for final analyses. This matrix included patient ID, episode identifier, term, and the number of times the term appeared in the note.

Notes Extraction
Clinical notes in the data set are not available from all providers for all years due to organizational shifts and variation that may occur over time in clinical documentation practices and potentially other factors. The clinical notes for each of the patients for 30 days prior to the OTO episodes were collected. Notes in our dataset are created individually in each provider's EHR, and a single day can produce dozens of notes for a patient while a single episode's 30-day history can produce hundreds of notes depending on documentation practices and EHR system variation. For inpatient stays, one note may be just an update of vital signs. We chose to merge all notes for a single episode into one document that could be used to evaluate the text that created topics, i.e. we processed text at the episode level. By aggregating the notes to their episodes there is a consistent unit of analysis, and we avoid the complexity of trying to attribute meaning to an episode from a multitude of notes.

Data Cleaning
NLP requires some basic cleaning and processing to make the notes more useful. Punctuation, white-space, proper names, and very common words (e.g., "the" "and") are removed. These things do not provide useful information, for example it is not possible to distinguish context for very common words.
Many notes in an EHR record are cut-and-pasted from prior notes for the same patient, resulting in a complete history of all prior notes with minimal new information added to each new note created. This can cause over counting of terms in the original note, as they would be repeated many times. An example of how cut-and-paste might look in a note is below: Often notes also include language from standard templates. These include standard surveys, discharge instructions, and phrases repeated for all patients at that provider, with no information specific to any individual patient. These are often form data logged into the note as well as answers. Hospitals and providers have their own standard templates that may be included for all patients seen at that facility.
Frequently seen examples of this in our data included: • Diet/Nutrition: Regular Diet: Eat a wide variety of foods including fresh fruits and vegetables, whole grains, lean meats, poultry and fish and low-fat dairy products. only reduces the volume of text per episode, but leads to having more meaningful text to evaluate.

Term Vectors and Clusters
The next stage of processing was to create term vectors from the data set. A term vector is a count of phrases in a text document. She is on oxycodone and has difficulty with sleep due to back pain. She feels that her activities of daily living are increased, but she is still on the oxycodone and methadone. His other medicines include oxycodone, an aspirin a day, Prilosec, Dilantin and Flagyl.

Creating a Matrix from Vectors
Fentanyl gives him at least three hours pain relief. He has significant problems with fractured sleep. Current medications include enalapril, low dose enoxaparin, Fentanyl patches. He is no longer on fluconazole.
He has chronic back pain and a fentanyl patch. He denies any constipation, diarrhea, abdominal pain. He is receiving fentanyl Since he has been here for back pain, leg pain, abdominal pain, and pain in the feet. He states that he is currently in pain and the fentanyl only helps for about an hour or so before the pain resumes.
Dr. Smith has maintained him on opioid medications consisting of Norco 10/325 mg for breakthrough pain and oxycodone. The patient has not tolerated morphine in the past. We will start oxycodone 5 mg q.2h. as needed. At this point, the episode data is contained in a term-frequency matrix. Each row of this matrix is an episode, each column is a term and the values in the matrix are the frequency counts. These counts are then multiplied by the inverse document frequency (how many episodes contain the term) to adjust the matrix so terms that are specific to only a few documents (i.e., notes) have more relevance. (Salton & Buckley, 1988).
Clustering patients or other data is usually an assignment of the observation to a single cluster that is performed to maximize similarity within each cluster while maximizing separation of the clusters. With NLP data, and term-frequency, inverse document frequency (TF-IDF) matrices, clustering is more often a weighted assignment of patients to many clusters. In this case, a patient has a weight that indicates the importance of that cluster in the patient's data. The methods for this type of clustering come from analyzing text documents where the clusters are called topics, each episode's TF-IDF vector is a weighted collection of topics like back pain, prescription orders, physical therapy etc.
The approach chosen for topic creation is non-negative matrix factorization, where the TF-IDF matrix is factored into two matrices, an episode-topic and topic-term matrix. Because the TF-IDF is non-negative, each matrix element can be approximated as a non-negative linear combination of topic weights and term weights.
Given the original matrix A, we can obtain two matrices W and H, such that A= WH. Nonnegative matrix factorization has an inherent clustering property, such that W and H represent the following information about A: A (Document-word matrix) -input that contains which words appear in which documents.
W (Basis vectors) -the topics (clusters) discovered from the documents. The number of topics is pre-selected. The project computed 25, 50 and 100 topics. After review by experts, 100 topics were chosen as sufficiently many to have precise topics (e.g., sickle cell disease, diabetes, lung cancer, and knee surgeries appear in separate topics without other conditions) without having duplicative topics (separate topics for 'oral' vs 'orally').

Free text analysis Results
Starting with a corpus of almost a billion notes from 2007-2016, we identified opioid episodes by opioid type and then applied inclusion/exclusion criteria. This left us with a cohort of 168,916 OTO episodes which occurred in 149,408 individuals. This cohort was further broken into two groups: 99,761 episodes contributed by 89,163 individuals had no record of an opioid prescribed or administered in the preceding 30 days, which were labeled as unexposed, or "controls"; 69,155 episodes contributed by 65,526 individuals had evidence of an IR opioid in the preceding 30 days and were labeled as exposed, or "cases." Tolerance: Evidence of ≥30mg of oxycodone equivalents on each day of the 7 days prior to OTO episode using claims data, exclusive of start date)

NLP Analysis Summary
Although we started with a large number of OTO episodes based on claims data and a large number of clinical notes, the overlap of notes preceding specific OTO episodes was relatively small. We used NLP to extract information from clinical notes in an attempt to identify evidence of previous opioid tolerance that may not be available in claims data alone. Using topic modeling we evaluated numerous topic cut-points and configurations and were unable to find topics that were indicative of opioid tolerance. Based on feedback from our TEP, we went a step further to determine if we could identify explanatory reasons why clinicians may prescribe in a manner inconsistent with product labeling. We were unable to identify any statistically significant explanatory topics. Poisoning by other opioids, intentional self-harm T402X2A Poisoning by other opioids, intentional self-harm, initial encounter T402X2D Poisoning by other opioids, intentional self-harm, subsequent encounter T402X2S Poisoning by other opioids, intentional self-harm, sequela T402X4 Poisoning by other opioids, undetermined T402X4A Poisoning by other opioids, undetermined, initial encounter ICD-9 ICD-10 Code Description T402X4D Poisoning by other opioids, undetermined, subsequent encounter T402X4S Poisoning by other opioids, undetermined, sequela T403 Poisoning by, adverse effect of and underdosing of methadone T403X Poisoning by, adverse effect of and underdosing of methadone T403X1 Poisoning by methadone, accidental (unintentional) T403X1A Poisoning by methadone, accidental (unintentional), initial encounter T403X1D Poisoning by methadone, accidental (unintentional), subsequent encounter T403X1S Poisoning by methadone, accidental (unintentional), sequela T403X2 Poisoning by methadone, intentional self-harm T403X2A Poisoning by methadone, intentional self-harm, initial encounter T403X2D Poisoning by methadone, intentional self-harm, subsequent encounter T403X2S Poisoning by methadone, intentional self-harm, sequela T403X4 Poisoning by methadone, undetermined T403X4A Poisoning by methadone, undetermined, initial encounter T403X4D Poisoning by methadone, undetermined, subsequent encounter T403X4S Poisoning by methadone, undetermined, sequela T404X1 Poisoning by other synthetic narcotics, accidental (unintentional) T404X1A Poisoning by other synthetic narcotics, accidental (unintentional), initial encounter T404X1D Poisoning by other synthetic narcotics, accidental (unintentional), subsequent encounter T404X1S Poisoning by other synthetic narcotics, accidental (unintentional), sequela T404X2 Poisoning by other synthetic narcotics, intentional self-harm T404X2A Poisoning by other synthetic narcotics, intentional self-harm, initial encounter T404X2D Poisoning by other synthetic narcotics, intentional self-harm, subsequent encounter T404X2S Poisoning by other synthetic narcotics, intentional self-harm, sequela T404X4 Poisoning by other synthetic narcotics, undetermined T404X4A Poisoning by other synthetic narcotics, undetermined, initial encounter T404X4D Poisoning by other synthetic narcotics, undetermined, subsequent encounter T404X4S Poisoning by other synthetic narcotics, undetermined, sequela