A, The most recent questionnaire available used to determine questions. Alaska and Hawaii not shown; Alaska used the national marijuana module, and Hawaii did not have cannabis-related questions. Ohio added the national marijuana module in 2018. B, Other includes the Massachusetts questionnaire, which includes a question related to 30-day use of marijuana, but does not include 30-day frequency of use; 7 states (ie, Alabama, Georgia, Indiana, Mississippi, New Jersey, Ohio, South Dakota) did not achieve adequate response rates to report population weighted estimates in 2017.
eFigure. Cannabis Questions Available in the Behavioral Risk Factor Surveillance System and Youth Risk Behavior Surveillance Survey and State Cannabis Legalization Status
eTable 1. Cannabis Use Questions in Behavioral Risk Factor Surveillance System and Youth Risk Behavior Surveillance Survey
eTable 2. Cannabis Use Questions in Massachusetts Behavioral Risk Factor Surveillance System from 2011-2017
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Geissler KH, Kaizer K, Johnson JK, Doonan SM, Whitehill JM. Evaluation of Availability of Survey Data About Cannabis Use. JAMA Netw Open. 2020;3(6):e206039. doi:10.1001/jamanetworkopen.2020.6039
Are US national data measuring aspects of cannabis use consistently available across states and over time?
This case series study examining 7 state and US nationwide surveys found that limited data are available at the state or national level to monitor 8 key dimensions of cannabis use and perceptions.
These findings suggest that despite growing public support for cannabis legalization, monitoring public health effects of state and national policies may be difficult owing to a lack of available data in existing population surveys.
In response to increasing public support for cannabis legalization, understanding associations of state and federal policy changes related to cannabis legalization with patterns of cannabis use is important. A challenge for public health monitoring and research is significant variation in data availability related to cannabis use behaviors and perceptions across and within states and over time, including the availability of prelegalization vs postlegalization data.
To review data available on cannabis use and related behaviors over time in Massachusetts and the US.
Design, Setting, and Participants
This case series examined state and national surveys on public health and related behaviors and outcomes to review availability of cannabis-related data for Massachusetts for 8 key indicators over time. Additionally, the Behavioral Risk Factor Surveillance System (BRFSS) and Youth Risk Behavior Surveillance System for all states were reviewed. The analysis was conducted between February 1, 2019, and March 18, 2020.
Surveys that enable state-level estimation of cannabis use and related behaviors.
Main Outcomes and Measures
Eight key indicators related to cannabis use behaviors and perceptions: lifetime cannabis use, age of initiation, frequency of use, location of use, method of use, source of cannabis, perceptions of cannabis, and reason for use (ie, medical vs nonmedical).
There were 7 surveys that monitored cannabis use and related behaviors in Massachusetts for adolescents and adults. No surveys monitored all 8 indicators of interest, and availability over time was limited. In the most recent BRFSS, 24 states asked cannabis-related questions, meaning BRFSS data on cannabis use was lacking for more than half of the US adult population. In the Youth Risk Behavior Surveillance System, 36 states asked standard cannabis questions; most other states had at least 1 question related to cannabis use and frequency.
Conclusions and Relevance
These findings of limitations of existing surveys, and particularly the lack of national questions in the BRFSS and Youth Risk Behavior Surveillance System, suggest that available data have substantial limitations for monitoring cannabis use. As cannabis policy changes continue, there is a need to remain focused on the availability of high-quality data sources that allow for critical public health research.
As of December 2019, 11 states and the District of Columbia (DC) have legalized cannabis for adult nonmedical use1 and 33 states and DC have legalized cannabis for medical purposes.2 In response to growing public support3 for cannabis legalization, understanding the effects of state and federal policy changes related to cannabis legalization on patterns of cannabis use is important.4 There is intense policy interest, including by regulators, in monitoring a number of indicators associated with cannabis use behaviors and perceptions among adolescents and adults to detect, and if necessary reduce, any negative public health effects of cannabis legalization.5-7 A challenge for monitoring and research is significant variation in data availability across and within states and over time,4,7-9 including availability of prelegalization vs postlegalization data.
Variation in available data across states highlights the need to better understand the extent of existing data in measuring effects of state and federal policy changes associated with cannabis legalization. Given ongoing policy and regulatory changes (eg, retail markets, packaging, labeling),5,7,10 it is critical to understand whether data are available to monitor a number of cannabis use indicators associated with specific legislative or regulatory provisions. These provisions continue to evolve as states implement legalization with varying policy and regulatory structures. Specific cannabis-related behaviors of importance include frequency, location, and methods of use; these behaviors may be risk factors for problematic cannabis use and associated outcomes.11-15 Understanding data availability to monitor cannabis use in nationally comparable surveys is critical to be able to compare findings from different states,7,9,16-18
Many states with legal cannabis have statutorily required research agendas to monitor outcomes, including public health.9,19,20 Negative consequences associated with cannabis use among youth are well documented and include neurocognitive defects, mental health issues, risk for dependence, lower educational attainment,21-23 and e-cigarette or vaping product use–associated lung injury.24-26 Methods of cannabis use, including among youth, are changing. A decreasing number of individuals who use cannabis report smoking, and an increasing number report use of nonsmoking modes, such as edibles, vaping, and dabbing,27-31 for which little is known about possible adverse health effects.
There is conflicting evidence about whether the prevalence of recent (ie, within the past 30 days) cannabis use increases after legalization,16,29,32,33 but it is clear that prevalence and frequency of use have increased significantly over the past 2 decades.34,35 In 2018, 45% of individuals older than 12 years had ever used marijuana and 16% used marijuana in the past year.36 Perceptions of risk associated with cannabis use have decreased, particularly among youth, which has been associated with a higher likelihood of future use.3,33,37,38
To better understand available data to monitor cannabis use among adolescents and adults, we conducted a scoping review and evaluation of survey data over time to monitor 8 indicators related to cannabis use and perceptions in Massachusetts.39 We also sought to inform future research and monitoring efforts by evaluating the utility of national data sources for generating state-level estimates of cannabis use and perceptions that can be compared across US states. We hypothesized that state-level data would better capture the 8 indicators over time than national data.
We used a case series approach to identify and examine data sources available to measure cannabis use. For Massachusetts, our in-depth case study site, we analyzed data availability over time and assessed specific cannabis use questions for the general adult population as well as specific at-risk populations, including adolescents. We focused on Massachusetts because the legislation authorizing the legal retail cannabis market outlined annual monitoring requirements, but the extent of data availability to meet those requirements was unknown.39 We then complemented this in-depth assessment of Massachusetts data with a review of cannabis use questions in major national data sources as they apply to all 50 states and Washington, DC. The analysis was conducted between February 1, 2019, and March 18, 2020. This study was determined to be not human participants research by the University of Massachusetts Human Research Protection Office. We followed the reporting guideline for case series.40
We used publicly available documents describing national and state-specific data sources to determine cannabis use data availability. We first conducted a scoping review in Massachusetts to determine available data sources to measure cannabis use at the state level for the general population and specifically for adolescents. We identified data sources using published literature, national and state government surveys, and key informant interviews with state government agencies’ staff. We used website and document review to determine specific survey questions and inclusion of specific questions over time. We used the most recent questionnaire available online (as of April 15, 2019), unless the most recent version available was administered before 2015, in which case we contacted the data provider for more information. We limited this analysis to survey data that could identify prevalence of cannabis use and/or related behaviors of interest at the state level. We excluded national surveys that could not produce state-level estimates. We also excluded surveys that captured only local (ie, within-state) estimates.
We analyzed data sets available at the national level in more detail. The Behavioral Risk Factor Surveillance System (BRFSS) is jointly conducted by the Centers for Disease Control and Prevention and state agencies annually and is designed to be representative of the noninstitutionalized adult population in a state. We first determined which states used the national marijuana module containing questions related to cannabis use, frequency of use, method of use (eg, smoking, edibles), and medical vs nonmedical use. For states that did not use the national marijuana module in the most recent year for which this information is systematically available from the Centers for Disease Control and Prevention (ie, 2017), we examined state BRFSS websites to review the most recent questionnaire (as of September 3, 2019). If the questionnaire was older than 2015, we contacted the state agency responsible for the most recent version. For the Youth Risk Behavior Surveillance Survey (YRBSS), we examined state YRBSS websites to review the most recent questionnaire (as of March 15, 2020); if the questionnaire was not available, we used the most recent results available from the state YRBSS website or the Centers for Disease Control and Prevention website.
We assessed which data were available in Massachusetts and recorded years available, frequency of collection, population representation, age range, number of individuals surveyed, and access information. For each data source that met the inclusion criteria, we used survey questionnaires from 2011 to the most recent year available to determine cannabis-related questions over time. We analyze 8 key indicators of cannabis use patterns and perception of interest to the Massachusetts legislature, as codified in the statute that legalized cannabis for adult use in 201719; this is a robust set of indicators of interest to public health researchers and policy makers. These 8 indicators include lifetime cannabis use, age of initiation, frequency of use, location of use, method of use, source of cannabis, perceptions of cannabis, and reason for use (ie, medical vs nonmedical).
For the BRFSS and YRBSS (including high school students only), after the collection of the questionnaires, we analyzed inclusion of cannabis-related questions across states. For BRFSS, we categorized states into 4 categories reflecting whether the state (1) used the national marijuana module, (2) included state-specific questions that were at least equivalent to the national marijuana module, (3) included state-specific questions related to cannabis but did not include all questions from the national module, or (4) did not include any cannabis-related questions. We used state population estimates from the 2018 bridged-race population estimates41 of adults aged 20 years and older to determine national BRFSS coverage with questions related to cannabis use. For the YRBSS, we categorized states into 4 categories reflecting whether the state (1) used the marijuana questions included in the national or standard questionnaire, (2) asked at a minimum about marijuana use and frequency of use in the past 30 days, (3) other, or (4) did not conduct the YRBSS. Questions from the BRFSS marijuana module and national or standard questions from the YRBSS questionnaire are presented in eTable 1 in the Supplement. To provide further context, we examined state policies and categorized them on the basis of the legal status of cannabis (ie, legalized general adult use, legalized medical use, decriminalized, or fully illegal) as of December 2019. Maps were created using ArcMap software version 10.7.1 (Esri).
We found 7 surveys containing cannabis use information for Massachusetts residents, including national and state surveys (Table 1). We included the national BRFSS for comparison purposes, although it cannot produce estimates for cannabis use in Massachusetts. Of these 7 surveys, 3 are national surveys that can produce state-level estimates and 4 are Massachusetts-specific surveys. There is some overlap in these surveys, as the National YRBSS and Massachusetts Youth Risk Behavior Survey (MYRBS) are conducted simultaneously. Overall, these surveys represent a comprehensive age range, with several adult-specific surveys, several adolescent-specific surveys, plus the National Survey on Drug Use and Health (NSDUH), which includes information on individuals 12 years and older, and the National Health and Nutrition Examination Survey (NHANES), which includes information on all ages.
Our review of detailed survey information to monitor cannabis use over time found that these surveys capture a wide variety of items and time periods, but no survey captures all 8 indicators of interest simultaneously (Table 2). The NHANES and the NSDUH include the most information for adolescents and adults, as they capture time series information about lifetime use, age of initiation, and frequency of use using standard questionnaires administered nationally. The NSDUH also captures information over time related to the source of marijuana, perceptions of marijuana, and reason for use. Although the national BRFSS marijuana module captures some of this information systematically, Massachusetts and 38 other states do not use this module.
State-specific survey data are available to fill some information gaps, although these surveys are not available over a long period, particularly in the prelegalization or premedical cannabis periods. The Massachusetts BRFSS includes questions related to problematic cannabis use; however, there were a number of changes in the questions related to cannabis over time, which makes it difficult to examine changes in behavior over time (eTable 2 in the Supplement). The Massachusetts Marijuana Baseline Health Study, conducted in 2017, captured baseline frequency of use, method of use, perceptions of marijuana, and reason for use,42 similar to the information captured by the national BRFSS module; however, it is unknown to what extent these measures will be assessed over time, as Massachusetts state agencies have not publicly disclosed plans to repeat this survey.
The BRFSS is a powerful national survey owing to its size, time scale, and the frequency of administration, which may allow for within- and across-state estimates; we analyzed this survey to determine which states consistently capture cannabis use data (Figure; eFigure in the Supplement). Of 50 states and DC, we found that 11 states used the national marijuana module as of 2017 (plus 1 additional state in 2018), 4 states did not use the national marijuana module but asked questions that were at least equivalent to the national module, 9 states asked any state-specific marijuana questions, and 27 states did not ask any marijuana questions (Table 3). Using state population estimates, we found that 32.2% of the US population is covered by the national marijuana module or equivalent state-specific questions.
For adolescents, the YRBSS, MYRBS, and Massachusetts Youth Health Survey are conducted among Massachusetts high school students (and middle school students for the Massachusetts Youth Health Survey). These are conducted in public schools every 2 years and representative of a large but select group of adolescents. These surveys capture information about lifetime use, age of initiation, frequency of use (not among middle school students), perceptions of marijuana, and location of use specific to school property. Intermittent information is available on method of use, but this has not been captured systematically over time. We analyzed questions in YRBSS state questionnaires and found that 36 states ask questions consistent with the standard or national questions in the most recent year (Figure, B; eFigure in the Supplement). Substantial variation exists in state level questionnaires, with 6 states asking questions related to mode of use, 6 states asking questions related to perceptions of use, and 3 states asking about use on school property, among others. In 2017, 7 states did not have adequate response rates to report population-weighted estimates.
This case series study found a number of sources that could be used to monitor cannabis use over time, but many of these data sources have substantial limitations that restrict their usefulness in understanding effects of state and federal policy changes. Our results highlight gaps in state data collection and how these gaps affect knowledge surrounding the public health effects of cannabis legalization.
Contrary to our hypothesis, we found that Massachusetts—which legalized medical cannabis in 2012 and general adult use of cannabis in 2016—does not have consistently available state data sources to monitor all 8 key cannabis use indicators of interest and that state-level data were not as robust as national data sources. Despite the public health importance of maintaining nationally comparable survey data over time to allow for outcomes analyses of anticipated and unanticipated policy changes,9 our results show this is not happening consistently within Massachusetts or using the BRFSS and YRBSS nationally. We found that although 2 major national surveys, the NSDUH and NHANES, can provide substantial information about multiple dimensions of cannabis use over time, these surveys are limited by sample size in their ability to make across- and within-state comparisons, particularly at the annual level. This is an important limitation when one considers the rapid pace at which legalization and related policies are changing and the need to detect potential changes in key indicators in as timely a manner as possible.
Monitoring cannabis use among adolescents is of particular public health importance owing to adverse effects associated with cannabis use on brain development, potential substance use dependence later in life, and changes in perceptions around cannabis use and changing methods of use.3,27-31,33,37,38 In Massachusetts, we found that there are national and state-specific surveys available (under restricted access policies from data providers) that monitor cannabis use and frequency; however, these surveys have not consistently collected information over time about method of use, location of use, and/or perceptions of cannabis, all of which are important to fully quantify changing patterns of use over time. As cannabis is legalized for adult use, monitoring diversion of legal cannabis into the adolescent population is particularly important. These youth surveys, available in Massachusetts, are not available in all states with medical or nonmedical cannabis.9 Colorado is a leader in carefully tracking these measures. Colorado was one of the first states to legalize adult use of cannabis and has carefully tracked the public health implications of legalization, including through the addition of questions over time to the Healthy Kids Colorado Survey, such that the most recent version included questions on 6 of the 8 dimensions we examine (ie, lifetime use, age of initiation, location of use, method of use, source of marijuana, and perceptions of marijuana).43
Understanding the availability and comparability of BRFSS data across states with respect to cannabis-related questions is a novel contribution of our analysis. The NSDUH is able to make state-specific estimates at the annual level using restricted data or using 2-year periods using public use data; additional national data sources are necessary to research the associations of time-sensitive state policy changes and make within-state comparisons. The BRFSS has a much larger sample size at the state level, but we found that a large number of states, with more than 56.3% of the national population, do not include any cannabis use questions. The national marijuana module became available for use in 2016; in early legalization states (ie, states that legalized cannabis before 2016), the national module does not contribute substantial information for understanding changes in cannabis use during longer periods or across the prelegalization to postlegalization transition. However, if implemented in states prior to legalization for adult use, the national BRFSS marijuana module would likely provide useful data.3,9 This shows the importance of changes to BRFSS core questions, including adding questions on cannabis use, in a rapidly changing policy environment.3,4 This is particularly important to monitor outcomes among at-risk population groups, such as adolescents, persons with substance use or mental health disorders, and pregnant women, who may be disproportionately affected by cannabis legalization and for whom there is likely not adequate sample size to examine carefully at the state level in national surveys. Additionally, to our knowledge, no data are available to measure cannabis use among incarcerated populations, who have high rates of substance use disorders.44
Our study has several limitations. The first limitation is that our primary analysis of available surveys was limited to Massachusetts; as we show with variation in availability in the BRFSS and YRBSS, substantial differences in data availability exist across states, so data available in Massachusetts may not be generalizable to the rest of the country. However, this in-depth analysis of data available for Massachusetts is an important case study to understand what may be available in other states and reveals issues likely to be common across states. The second limitation is that we limited our analysis to survey data, as this is the most commonly used way of obtaining prevalence estimates for cannabis use.9 Administrative or other data sources, such as health insurance claims data or substance use treatment facility discharge data,45 may also capture changes in cannabis use associated with cannabis use disorder. Owing to the very small number of surveys measuring polysubstance use, we were not able to consider concurrent use of cannabis and other substances, such as alcohol. This is an important extension in considering future data collection, particularly since little is known about polysubstance use for adults and adolescents.46,47 Third, because we included only data sources that support estimation at the state level, we excluded a number of data sources with potential relevance to research on cannabis policy (eg, the American College Health Survey and Monitoring the Future). Fourth, our focus on the general population omitted some important subgroups, such as pregnant women, for whom there is growing interest and concern regarding cannabis use.
In this case series study, we report 2 large, high quality, nationally representative surveys—NHANES and NSDUH—that can be used to monitor cannabis use and the outcomes associated with state and national policy changes; however, substantial challenges remain in the availability of data needed to fully monitor the public health effects associated with cannabis legalization. Existing surveys, and particularly the lack of national core questions in the BRFSS or standardized required questions in the YRBSS, suggest that available data have substantial limitations for monitoring cannabis use along the 8 dimensions we examined. As research into the effects of cannabis legalization and other changes in cannabis policy continue, remaining focused on the availability of high-quality data sources will allow critical public health research.
Accepted for Publication: March 23, 2020.
Published: June 10, 2020. doi:10.1001/jamanetworkopen.2020.6039
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Geissler KH et al. JAMA Network Open.
Corresponding Author: Kimberley H. Geissler, PhD, University of Massachusetts Amherst School of Public Health and Health Sciences, 715 N Pleasant St, 337 Arnold House, Amherst, MA 01003 (firstname.lastname@example.org).
Author Contributions: Dr Geissler had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Geissler, Johnson, Whitehill.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Geissler, Kaizer, Johnson.
Critical revision of the manuscript for important intellectual content: Geissler, Johnson, Doonan, Whitehill.
Obtained funding: Whitehill.
Administrative, technical, or material support: Geissler, Kaizer, Doonan.
Supervision: Johnson, Whitehill.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was funded in part by the Massachusetts Cannabis Control Commission.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Coauthors from the funding agency participated in interpretation of data and, consistent with authorship standards, were provided an opportunity for review, comment, and approval of the manuscript.
Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the views of Massachusetts Cannabis Control Commission.
Additional Contributions: Faith English, MPH, Isabel Albinger, and Joyce Libbin (all from Department of Health Promotion and Policy, University of Massachusetts, Amherst) provided research assistance and were compensated through their normal salaries.
Create a personal account or sign in to: