Are state-level opioid-reduction policies associated with a decrease in opioid poisoning in children and adolescents?
In this interrupted time series analysis of 338 476 opioid poisonings among children younger than 20 years, implementation of a prescription drug monitoring program and a pain clinic legislation was associated with immediate and sustained reductions in rates of opioid poisoning.
This study found that state-level opioid-reduction policies may reduce pediatric opioid poisoning.
Opioid-reduction policies have been enacted by US states to address the opioid epidemic. Evidence of an association between policy implementation and decreased rates of pediatric opioid poisoning provides further justification for expanded implementation of these policies.
To examine the association of 3 state-level opioid-reduction policies with the rate of opioid poisoning in children and adolescents.
Design, Setting, and Participants
This interrupted time series analysis used data from the National Poison Data System (NPDS), a database of poisoning information reported to poison control centers across the US. Individuals younger than 20 years who experienced poisoning associated with 1 or more prescription opioids from January 1, 2005, to November 30, 2017, were included. The analysis focused on 3 widespread policy interventions: the prescription drug monitoring program (PDMP), pain clinic legislation, and opioid prescribing guidelines. Data analysis was performed from January 30, 2020, to March 30, 2020.
Any opioid poisoning in individuals younger than 20 years that was reported to the NPDS.
Main Outcomes and Measures
Opioid poisoning rates per million person-months before and after implementation of each of the 3 policies, overall and stratified by age (≤4 years, 5-9 years, 10-14 years, and 15-19 years).
A total of 338 476 opioid poisoning incidences in children and young adults were reported to the NPDS within the study period. Of this study population, the mean (SD) age was 9.74 (7.15) years, and 179 011 (52.9%) were female. The implementation of a PDMP was associated with a reduction in the monthly rate of opioid poisoning in children and adolescents (–0.07 per million person-months; 95% CI, –0.09 to –0.04) in the postimplementation period. This reduction was observed for all age groups except for the 10- to 14-year age group (−0.03 per million person-months; 95% CI, −0.05 to 0.00). Pain clinic legislation was associated with an immediate reduction in opioid poisoning (–6.22 per million person-months; 95% CI, –8.98 to –3.47). This association was statistically significant across all ages except for the 4 years or younger group. Analysis of the association of implementation of opioid prescribing guidelines was limited because of insufficient follow-up data and did not show an immediate or monthly change in the rate of opioid poisoning.
Conclusions and Relevance
Results of this study suggest that certain state-level opioid-reduction policies were associated with decreases in pediatric opioid exposures across age groups. Further examination of the underlying mechanisms of these associations, including age group–specific outcomes, may expand and strengthen policies that reduce opioid poisoning, misuse, and overdoses in children and adolescents.
Children and adolescents have been harmed by the opioid epidemic in the United States. Nearly 200 000 individuals younger than 20 years were poisoned by opioids between 2000 and 2015.1 Approximately 9000 individuals younger than 20 years died from prescription and illicit opioid use between 1999 and 2016, with increasing mortality rates in all age groups.2 Among adolescents aged 15 to 19 years, the death rates associated with opioid use were 3 times higher in 2015 than in 1999.3 Consistent with these patterns, opioid-associated emergency department visits, hospital admissions, and intensive care admissions among youth have all increased in recent years.4-6
To address the opioid epidemic, US states have enacted a number of laws and regulations to decrease the amount of opioid medications prescribed and distributed (Box). Three such policies have achieved widespread adoption: a prescription drug monitoring program (PDMP), pain clinic legislation, and opioid prescribing guidelines. A PDMP is a database that digitally stores controlled-substance dispensing information across pharmacies for individual patients at the state level. During patient care, clinicians use this database to guide opioid prescribing, querying for a patient’s recent treatment with opioids or other controlled substances. Pain clinic legislation limits opioid prescribing in pain clinics, in which a disproportionately large number of prescriptions for opioids and other controlled substances have historically been written.7 Opioid prescribing guidelines restrict the prescribing and distribution of opioids for the treatment of acute pain. All 3 of these policies have been associated with decreased opioid prescribing and lower rates of opioid-associated overdoses and deaths in adults.7-15
Box Section Ref ID
State-Level Opioid-Reduction Policies
Prescription Drug Monitoring Program
A prescription drug monitoring program (PDMP) is a state-run database of the prescription and distribution of controlled substances. States vary in their required types of drugs monitored (eg, US Drug Enforcement Administration schedule II-IV, II-V, and ≥II-V), registration requirements (eg, whether physicians need to register with the PDMP), and query requirements (eg, whether physicians need to query the PDMP before prescribing or distributing controlled substances).
Pain Clinic Legislation
Pain clinics are specialty clinics that serve primarily patients with chronic pain and have been the source of disproportionately large volumes of opioid prescriptions. The specifications of pain clinic legislation vary across states and include requirements for physician owners, the presence of a medical director, limits on maximal supply dispensed, and requirements for controlled substance training.
Opioid Prescribing Guidelines
Opioid prescribing guidelines were developed to limit the amount and duration of opioid prescriptions for the treatment of acute and chronic pain. These guidelines have been adopted in the form of legislation, nonlegislative regulations, and recommendations issued by governmental agencies and professional organizations.
National guidelines and monitoring programs have focused primarily on adult populations, and information is limited on the association of state-level opioid-reduction policies with poisoning among youth. To further inform efforts to reduce opioid-associated harm in children and adolescents, we investigated the association of state-level opioid-reduction policies with opioid poisoning in this population.
This interrupted time series analysis received approval from the institutional review board of Boston Children’s Hospital. Informed consent was waived by this institutional review board because the study analyzed only deidentified patient data and did not represent human participant research.
We identified opioid poisoning incidences using the National Poison Data System (NPDS), a national database of poisoning information reported to regional poison control centers. Currently, 55 poison control centers serve all areas of the United States. Poison control specialists receive calls from individuals and health care facilities and record the details on each poison exposure, including demographic information, drug implicated in exposure, intentionality, and patient disposition.16 Given the comprehensive reach of these centers, NPDS data are routinely used to monitor national patterns of intentional and unintentional poisoning by various substances across patient populations.1,17 We queried a deidentified extract of NPDS data for all poisoning incidences associated with 1 or more prescription opioids (eTable in the Supplement) in individuals younger than 20 years from January 1, 2005, to November 30, 2017. Details for each report were manually reviewed to exclude any misclassified cases (eg, cases associated with opioid withdrawal). Patient age was categorized as 4 years or younger, 5 to 9 years, 10 to 14 years, and 15 to 19 years to correspond with the US Census Bureau categories used by the American Community Survey 1-year estimates.18
State-Level Opioid-Reduction Policies
We focused the present analysis on the PDMP, pain clinic legislation, and opioid prescribing guidelines because they are the most widespread policy interventions for reducing opioid prescribing and overdose. Information on the implementation of state-specific PDMPs and the date of online availability to prescribers was obtained from publicly available resources such as the Prescription Drug Monitoring Program Training and Technical Assistance Center at Brandeis University, the National Alliance for Model State Drug Laws, and the Prescription Drug Abuse Policy System.19-21 The implementation date for PDMPs was defined as the date the program became available online to prescribers to reflect the time point at which clinical care would be expected to change.13 One instance of conflicting data between sources was found, and it was resolved through communication with the data specialists at the Prescription Drug Monitoring Program Training and Technical Assistance Center. By November 30, 2017, a total of 49 states had enacted a PDMP, and 40 states implemented the program during the study period (eFigure in the Supplement).
Information on implementation of pain clinic legislation was obtained from the Prescription Drug Monitoring Program Training and Technical Assistance Center and the Prescription Drug Abuse Policy System.19,20 Given the heterogeneity in requirements between states, we dichotomized states according to the presence or absence of pain clinic legislation, with the date the legislation went into effect chosen as the implementation date. Eleven states had enacted such legislation by November 30, 2017, and all implemented the policy during the study period (eFigure in the Supplement).
Data on state-level legislation establishing opioid prescribing guidelines were obtained from a published systematic review that performed a multisource search to identify all state laws related to the prescription and distribution of opioids to treat acute pain.22 We included only opioid prescription guidelines in statutes and did not analyze policies promulgated by other authorized entities, such as state health officials or clinician regulatory boards. In this case, the date that the opioid guidelines went into effect was chosen as the implementation date. Twenty-five states had enacted statutory opioid prescribing guidelines by November 30, 2017, with 24 of the 25 states implementing during the study period and 17 of these 24 states implementing in 2017 (eFigure in the Supplement).
We obtained age-specific population data from the US Census Bureau using the July 1 intercensal and postcensal population estimates for individuals younger than 20 years for each year from 2005 through 2017.18 To adjust for covariates potentially associated with increased risk of opioid exposure, we collected from the US Census Bureau annual state-level data on unemployment status, median household income, race/ethnicity (White, Black, Native American, Asian, Native Hawaiian, or other), and educational achievement (proportion of individuals who graduated high school).9,15,18
We conducted state-level interrupted time series analyses with monthly repeated measures to examine the association of each opioid-reduction policy with the rate of opioid poisoning in children and adolescents. We created binary variables to indicate the presence of a PDMP, pain clinic legislation, and opioid prescribing guidelines for each month of the study period for all 50 states. Thus, for each state and opioid-reduction policy, preimplementation and postimplementation periods were defined. To account for baseline trends in pediatric poisoning, we included all states in the analysis of preimplementation and postimplementation trajectories. For the immediate change analysis of the association at the end of the first month after implementation, only states that implemented a policy during the study period were included because the analysis required both preimplementation and postimplementation periods.9,13 For each policy, we estimated Prais-Winsten regression models with the Cochrane-Orcutt transformation and robust (ie, sandwich) SEs, in which the dependent variable was the monthly rate of pediatric opioid exposure at the state level (ie, number of poisoning per million person-months).14 To verify that these models adequately accounted for first-order autocorrelation, we examined the Durbin-Watson statistic (ie, values approximate or equal to 2, the expected value under the null hypothesis of no serial correlation, indicate adequate correction). The Durbin-Watson statistic was 1.851 before serial correlation and 1.997 after serial correlation for PDMPs, 1.842 before and 1.997 after for pain clinic legislation, and 1.842 before and 1.997 after for opioid prescribing guidelines.
Each model included as independent variables the monthly state-level policy indicator, time (measured in months compared with the month of policy adoption), and a policy-by-time interaction term. The interaction term compared the preimplementation and postimplementation slopes, corresponding to changes in the monthly rate of opioid poisoning associated with the reduction policy. To measure the immediate change of each policy in opioid exposure rates, we performed a Wald test comparing the estimated exposure rate as derived from the fitted model at the end of the preimplementation period to the actual exposure rate at the first month of the postimplementation period. We accounted for the correlation of data within states by including state-level fixed effects (modeled as dummy variables) as covariates. Each model included as covariates the binary variables indicating the monthly state-level presence of the other 2 policies (eg, the model evaluating the PDMP, including covariates for the pain clinic legislation and the opioid prescribing guidelines). All models also included covariates for state-level unemployment status, median income, race/ethnicity, educational achievement, and calendar year. All model effect estimates were presented with the corresponding 95% CIs.
Changes in the yearly rate of opioid poisoning per million persons were calculated according to annual US census data (American Community Survey 1-year estimates for 2005 to 2017). Analyses were conducted with Stata, version 14.2 (StataCorp). All tests were 2-tailed, and α = .05 was used to indicate statistical significance. Figures were created with R, version 3.6.1 (R Foundation for Statistical Computing) and the ggplot2 package (Hadley Wickham). We performed data analysis from January 30, 2020, to March 30, 2020.
A total of 338 476 opioid poisoning incidences in children and adolescents were reported to the NPDS between 2005 and 2017. Of this study population, the mean (SD) age was 9.74 [7.15] years, and 179 011 (52.9%) were female individuals. We observed a bimodal age distribution with peaks among children 4 years or younger and adolescents between 15 and 19 years (Figure 1). Most opioid poisoning in children 4 years or younger was unintentional (n = 139 590 [99.2%]), whereas poisoning in those 15 to 19 years of age was mostly intentional (n = 24 150 [88.8%]).
The total number of poisoning peaked at 30 434 in 2009, with a subsequent decrease to 19 487 in 2017. Rates of poisoning were highest for children 4 years or younger with a mean annual rate of 53 per million person-months and for adolescents between 15 and 19 years with a mean annual rate of 51 per million person-months throughout the study period. Mean annual poisoning rates for children 5 to 9 years were 8.7 per million person-months and for 10 to 14 years were 12.0 per million person-months. Unadjusted state-level pediatric opioid exposure rates over the study period in states with and without opioid-reduction policies are shown in Figure 2. The mean unadjusted preimplementation incidence was 25.3 poisoning per million person-months in states that implemented a PDMP, 31.1 poisoning per million person-months in states that implemented a pain clinic legislation, and 26.0 poisoning per million person-months in states that implemented opioid prescribing guidelines. The mean postimplementation incidence was 26.6 poisoning per million person-months in states that implemented a PDMP, 23.8 poisoning per million person-months in states that implemented a pain clinic legislation, and 25.0 poisoning per million person-months in states that implemented opioid prescribing guidelines.
The implementation of PDMPs was not associated with an immediate change in the opioid poisoning rate (difference between estimated and actual rates at 1 month: 0.16 poisoning per million person-months; 95% CI, –1.05 to 1.36) (Table). However, a statistically significant long-term decrease in the rate of monthly opioid poisoning was found in the postimplementation period (–0.07 per million person-months; 95% CI, –0.09 to –0.04). The decrease in the rate of monthly opioid poisoning was statistically significant for all age groups except for the 10- to 14-year age group (−0.03 per million person-months; 95% CI, −0.05 to 0.00).
Pain clinic legislation was associated with a statistically significant immediate change in the opioid poisoning rate (–6.22 per million person-months; 95% CI, –8.98 to –3.47) (Table). No decrease in the rate of monthly opioid poisoning was found after implementation (0.00 per million person-months; 95% CI, –0.03 to 0.03). Stratifying by age group, the association of the pain clinic legislation was statistically significant across all age groups except for the 4 years or younger age group.
Opioid Prescribing Guidelines
Implementation of opioid prescribing guidelines was assessed using data from 25 states, with 21 states (84.0%) providing less than 2 years of postimplementation data and 3 states (12.0%; South Carolina, Illinois, and Tennessee) reporting 2 or more years of data. Judging from these limited data, implementation of opioid prescribing guidelines did not appear to be associated with an immediate or monthly change in opioid poisoning (Table). For example, in adolescents 15 to 19 years, the rate of monthly opioid poisoning was –0.36 per million person-months (95% CI, –0.61 to –0.12) in the preimplementation period and –0.30 per million person-months (95% CI, –0.56 to –0.05) in the postimplementation period. Overall, we found no decrease in the rate of monthly opioid poisoning (0.02 per million person-months; 95% CI, –0.01 to 0.005) and no immediate change (–1.14 per million person-months; 95% CI, –3.10 to 0.81) after implementation.
State-level opioid-reduction policies were associated with statistically significant and sustained decreases in opioid poisoning among children and adolescents. We found that implementation of a PDMP was associated with a decrease in the long-term monthly rate of opioid poisoning in this population, whereas implementation of pain clinic legislation was associated with a sustained one-time reduction in the rate of opioid poisoning. Most states implemented opioid prescribing guidelines in 2017, toward the end of the study period, limiting our ability to adequately assess the association of these guidelines with opioid poisoning rates. However, based on available data, no association was observed between the implementation of these guidelines and the rate of opioid poisoning.
Overall, the benefits of state-level opioid-reduction policies have been shown for adult populations, and the present study, to our knowledge, is the first to quantify how these policies are also benefiting children and adolescents.9,10
Specifically, PDMPs were associated with reductions in opioid poisoning among children 4 years and younger and those between 15 and 19 years, the age groups in which high rates of opioid poisoning were observed. The PDMPs enable, and in some states require, a clinician to query an online database for a patient’s opioid prescription history with the goal of identifying whether the patient has obtained or may be obtaining opioids from multiple sources (known as doctor shopping). Furthermore, PDMPs have been shown to reduce the prescribing of Drug Enforcement Administration schedule II opioids and decrease opioid-related deaths in adults.9,13 These programs have also been associated with decreases in prescribed morphine milligram equivalents per dose per month and in total opioid volume among adults.11,12 The observed reduction in opioid poisoning associated with PDMPs in the youngest children is potentially secondary to a decrease in the general availability of opioids, a mechanism supported by findings from a previous study that reported an association between the volume of opioids prescribed to adults and opioid poisoning in young children.23 Similarly, another report demonstrated that young children of women with an opioid prescription were at an increased risk of opioid overdose.24 Among adolescents and young adults, the decrease in opioid poisoning is likely associated with a decrease in nonmedical access to these medications, although a reduction in the number of prescriptions written specifically for this group may also be a factor. Based on national survey data, adolescents and young adults who misused prescription opioids received these medications often through diversion from friends and family members.25 Other studies suggested that adolescents who experienced a nonfatal opioid overdose were less likely to have received an opioid prescription in the previous year compared with adults and estimated that the odds of an opioid overdose among adolescents increased by more than 3-fold if a prescription opioid were dispensed to a family member.26,27
The implementation of pain clinic legislation was associated with an immediate decrease in pediatric opioid poisoning, and this decrease was sustained over time. This association is likely attributable to mechanisms similar to those observed with PDMPs. In this case, the legislation was associated with the removal of high-frequency prescribers from the supply chain, leading to a decrease in overall opioid prescribing and subsequent drug diversion.7,11,28 No continued monthly reduction in the rate of opioid poisoning was found, which is consistent with the 1-time association of the policy with specific types of prescribers.
The analysis of the association of opioid prescribing guidelines was limited because only 3 states provided more than 2 years of postimplementation data. Among adult populations, these guidelines have been associated with reduced opioid prescribing,8,14 which may decrease the availability of opioids to children and young adults. The large number of states enacting these guidelines may provide data for future analyses of the potential association between prescribing guidelines and pediatric poisoning.14,29
In adolescents, the deaths associated with heroin and synthetic opioids (other than methadone) overdose have increased in recent years, but the death rate associated with prescription opioids has also been substantial and is roughly the same as the rate for heroin and synthetic opioids (1.06 deaths per 100 000 persons associated with heroin, 1.21 deaths per 100 000 persons associated with synthetic opioids, and 1.11 deaths per 100 000 persons associated with prescription opioids).2 Prescription opioids also remain the most common initiating opioid for individuals who go on to develop any type of opioid-use disorder, but in certain health care settings these drugs continue to be prescribed at high rates to adolescents and young adults.30,31 Thus, establishing and implementing policies associated with reductions in prescription opioid misuse among adolescents and young adults remains critical, because of the direct and downstream consequences of misusing and overdosing on other substances.
This study has several limitations. First, although the findings show the trends in exposure rates associated with opioid-reduction policies, we did not report absolute decreases in the number of opioid poisoning. The exposure data were obtained from the NPDS, a database that does not capture comprehensive exposure data because it is based on voluntary reporting. This approach likely led to an underestimation of the total number of opioid poisonings, although the national distribution and consistent use of NPDS over time make the system conducive to assessment of trends and patterns in exposure rates.1,23,32 A recent analysis estimated that the NPDS captured 1 fatal opioid overdose for every 61 opioid-associated deaths reported in the Drug-Involved Mortality database, which uses information extracted from death certificates.33 Comprehensive data sources of pediatric opioid poisoning are needed to further quantify the absolute association of opioid-reduction policies. Second, we did not collect information on clinical outcomes; it is possible that the effectiveness of the opioid-reduction policies we studied varied widely. Third, inherent to any interrupted time series analysis is the concern for residual confounding. Although we controlled for a number of factors, additional unmeasured variables were likely present that contributed to changes in opioid poisoning in children and adolescents (eg, the PROTECT [Prevention of Overdoses and Treatment Errors in Children Taskforce] Initiative to reduce unintended medication overdoses in this population).34 To control for patterns occurring at the national level, we included calendar year as a variable in the models. The staggered implementation of reduction policies over time by individual states also helped control for temporal trends. Fourth, we considered only the presence or absence of policies, although these policies may differ across states in specific features that could be associated with poisoning in the study population. Fifth, postimplementation data for opioid prescribing guidelines were limited given that most states did not implement guidelines until 2017, restricting our ability to assess the association of this policy.
In this study, both the PDMP and pain clinic legislation appeared to be associated with significant reductions in opioid poisoning among children 4 years and younger and adolescents between 15 and 19 years, the age groups in whom high rates of opioid poisoning were observed. The mechanism by which these opioid-reduction policies are associated with opioid poisoning likely varies, with PDMPs associated with long-term reductions in poisoning rates and pain clinic legislation associated with a 1-time immediate decrease. Further examination of the underlying mechanisms of these associations, including age group–specific outcomes, may yield insights to help expand and strengthen policies aimed at reducing opioid poisoning, misuse, and overdoses in children and adolescents.
Accepted for Publication: April 25, 2020.
Corresponding Author: Michael S. Toce, MD, MS, Division of Emergency Medicine, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115 (Michael.firstname.lastname@example.org).
Published Online: July 13, 2020. doi:10.1001/jamapediatrics.2020.1980
Author Contributions: Dr Toce had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Toce, Hudgins, Bourgeois.
Acquisition, analysis, or interpretation of data: Toce, Michelson, Burns, Monuteaux, Bourgeois.
Drafting of the manuscript: Toce, Michelson.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Toce, Michelson, Monuteaux.
Administrative, technical, or material support: Toce, Michelson, Burns.
Supervision: Hudgins, Bourgeois.
Conflict of Interest Disclosures: Dr Bourgeois reported receiving the Burroughs Wellcome Fund Innovation in Regulatory Science Award and support from the Harvard-MIT Center for Regulatory Science. No other disclosures were reported.
Funding/Support: Dr Bourgeois was supported by a grant from the Burroughs Wellcome Fund and the Harvard-MIT Center for Regulatory Science.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
GA. Prescription opioid exposures among children and adolescents in the United States: 2000-2015. Pediatrics
. 2017;139(4):e20163382. doi:10.1542/peds.2016-3382
M. Drug overdose deaths among adolescents aged 15-19 in the United States: 1999-2015. NCHS Data Brief
. 2017;(282):1-8.PubMedGoogle Scholar
et al. Vital signs: trends in emergency department visits for suspected opioid overdoses - United States, July 2016-September 2017. MMWR Morb Mortal Wkly Rep
. 2018;67(9):279-285. doi:10.15585/mmwr.mm6709e1
B; Hal Johnson Consulting and Division of Disease Control and Health Promotion, Florida Department of Health. Decline in drug overdose deaths after state policy changes - Florida, 2010-2012. MMWR Morb Mortal Wkly Rep
. 2014;63(26):569-574.PubMedGoogle Scholar
JL. Opioid prescribing in the United States before and after the Centers for Disease Control and Prevention’s 2016 opioid guideline. Ann Intern Med
. 2018;169(6):367-375. doi:10.7326/M18-1243
W. 2016 Annual report of the American Association of Poison Control Centers’ National Poison Data System (NPDS): 34th annual report. Clin Toxicol (Phila)
. 2017;55(10):1072-1252. doi:10.1080/15563650.2017.1388087
PDAPS. Prescription Drug Abuse Policy System: a source of rigorous legal data for researchers and detailed policy information for the public. Accessed December 1, 2017. http://pdaps.org/
DN; Canadian Drug Safety And Effectiveness Research Network (CDSERN). Overdose risk in young children of women prescribed opioids. Pediatrics
. 2017;139(3):e20162887. doi:10.1542/peds.2016-2887
FT. Prescription opioid use and misuse among adolescents and young adults in the United States: a national survey study. PLoS Med
. 2019;16(11):e1002922. doi:10.1371/journal.pmed.1002922PubMed
et al. Reductions in prescription opioid diversion following recent legislative interventions in Florida. Pharmacoepidemiol Drug Saf
. 2014;23(3):314-320. doi:10.1002/pds.3553
MJ. State-level and system-level opioid prescribing policies: the impact on provider practices and overdose deaths, a systematic review. J Opioid Manag
. 2016;12(2):109-118. doi:10.5055/jom.2016.0322
FT. Trends in opioid prescribing for adolescents and young adults in ambulatory care settings. Pediatrics
. 2019;143(6):e20181578. doi:10.1542/peds.2018-1578
JK. A comparison of opioid-involved fatalities captured in the National Poison Data System to data derived from US death certificate literal text. Pharmacoepidemiol Drug Saf
. 2019;28(10):1377-1385. doi:10.1002/pds.4869