[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.  Page Volume of National Comprehensive Cancer Network Clinical Practice Guidelines by Disease Site, 1996-2019
Page Volume of National Comprehensive Cancer Network Clinical Practice Guidelines by Disease Site, 1996-2019

NSCLC indicates non–small cell lung cancer.

Figure 2.  References Cited in National Comprehensive Cancer Network Clinical Practice Guidelines by Disease Site, 1996-2019
References Cited in National Comprehensive Cancer Network Clinical Practice Guidelines by Disease Site, 1996-2019

NSCLC indicates non–small cell lung cancer.

1.
Weinstein  IB, Case  K.  The history of cancer research: introducing an AACR Centennial series.  Cancer Res. 2008;68(17):6861-6862. doi:10.1158/0008-5472.CAN-08-2827PubMedGoogle ScholarCrossref
2.
National Comprehensive Cancer Network. NCCN history. Accessed September 6, 2019. https://www.nccn.org/about/history.aspx
3.
Internet Archive. Wayback machine. Accessed September 9, 2019. https://archive.org/web/
4.
NEJM Catalyst. Cognitive load theory and its implications for health care complexity. Published March 14, 2018. Accessed September 6, 2019. https://catalyst.nejm.org/cognitive-load-theory-implications-health-care/
5.
Abernethy  AP, Etheredge  LM, Ganz  PA,  et al.  Rapid-learning system for cancer care.  J Clin Oncol. 2010;28(27):4268-4274. doi:10.1200/JCO.2010.28.5478PubMedGoogle ScholarCrossref
6.
National Comprehensive Cancer Network. Library of NCCN compendia. Accessed September 9, 2019. https://www.nccn.org/professionals/compendia.aspx
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Research Letter
    Oncology
    March 13, 2020

    Changes in Length and Complexity of Clinical Practice Guidelines in Oncology, 1996-2019

    Author Affiliations
    • 1Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
    • 2Artificial Intelligence in Medicine Program, Brigham and Women’s Hospital, Boston, Massachusetts
    • 3Huntsman Cancer Institute, The University of Utah, Salt Lake City
    JAMA Netw Open. 2020;3(3):e200841. doi:10.1001/jamanetworkopen.2020.0841
    Introduction

    The last century has seen substantial increases in the generation of knowledge pertaining to clinical cancer care.1 In 1995, the National Comprehensive Cancer Network (NCCN), a not-for-profit alliance of National Cancer Institute–designated cancer centers, was created with the goal of developing standards of care for cancer treatment. Since its inception, the NCCN has published regularly updated Clinical Practice Guidelines in Oncology (NCCN Guidelines) that cover recommended management for 97% of patients with cancer. The guidelines are used by as many as 95% of oncologists in the US in clinical practice and are relied on by public and private insurers to determine oncology coverage policies.2 Since the establishment of the NCCN Guidelines, cancer management options have continued to increase in quantity and complexity because of advances in personalized medicine, novel therapeutics, and technology. The extent to which cancer data and knowledge are growing has implications for policy making, medical education, and the health care system. Thus, we sought to quantify the increase in clinical cancer knowledge represented in the NCCN Guidelines since the turn of the 21st century.

    Methods

    We analyzed current and archived NCCN Guidelines from 1996 to 2019 using the NCCN website, the Internet Archive,3 and the Harvard Medical School library. The study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies and was conducted with permission from the NCCN. Because the study does not involve human participants, it was exempt from institutional review board review per the Common Rule. We accessed NCCN Guidelines in original electronic portable document format and performed the data analysis during September 2019. The most common cancers (ie, breast, prostate, non–small cell lung, and colon) were selected for analysis.

    Document page count was evaluated as a surrogate for guideline knowledge volume. To characterize document complexity, we evaluated total references cited in the discussion section of each guideline document as well as the number of decision paths present in the flowchart sections of guideline documents from 1996 and 2019. A decision path was defined as a unique guideline path resulting from forks in the guideline flowchart for a particular subtype of malignant neoplasm. We modeled PC and RC trends via linear, polynomial, and exponential regressions for best fit. Analysis was conducted by disease site and averaged. For each year, standard deviation among disease sites was calculated. Statistical analyses were conducted using Stata version 13 (StataCorp). Given that statistics were mainly descriptive, no threshold for statistical significance was prespecified.

    Results

    Between 1996 and 2019, the mean (SD) page count of NCCN Guidelines increased from 26 (4.2) to 198 (30.0) pages, a 762% absolute increase overall and a mean increase of 7.5 pages annually (Figure 1). Mean (SD) references cited increased from 28 (16.8) to 856 (146.3), a 3057% increase overall and a mean increase of 36 references annually (Figure 2). Similar increases were seen across all cancer types studied. The mean (SD) number of decision paths increased from 30 (8.5) to 111 (49.5), a 370% absolute increase. Trends in page count and references cited were best fit by exponential regression (R2 = 0.99 for both). Using the best-fit models, projections for mean page count and references cited in 2025 would be 355 pages and 1954 references per disease site guideline.

    Discussion

    Clinical practice guidelines in oncology have increased dramatically in quantity and complexity over the past 2 decades and follow an exponential growth pattern. Recent and projected guideline growth, which is associated with both the accumulation of new data and expert opinion, has implications for oncology trainees, policy makers, payers, and patients. Study limitations include unavailable archived guidelines from certain years. Furthermore, while the metrics used represent interpretable surrogates, they may not be directly proportional to document complexity. If increases continue exponentially, the amount of data assimilation required to deliver optimal care may be unsustainable for individual practitioners.4,5 Strategies to organize oncologic knowledge, such as the creation of Drugs and Biologics and Imaging Compendia,6 have been implemented. Further approaches, including guideline stratification by evidence level and the use of artificial intelligence for decision support, should be investigated as ways to synthesize data and improve cancer decision-making.

    Back to top
    Article Information

    Accepted for Publication: January 21, 2020.

    Published: March 13, 2020. doi:10.1001/jamanetworkopen.2020.0841

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Kann BH et al. JAMA Network Open.

    Corresponding Author: Benjamin H. Kann, MD, Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Cancer Center, Harvard Medical School, 75 Francis St, Boston, MA 02115 (benjamin_kann@dfci.harvard.edu).

    Author Contributions: Dr Kann had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Kann, Johnson, Aerts.

    Acquisition, analysis, or interpretation of data: Kann, Johnson, Mak, Nguyen.

    Drafting of the manuscript: Kann, Johnson, Aerts, Mak.

    Critical revision of the manuscript for important intellectual content: Kann, Johnson, Mak, Nguyen.

    Statistical analysis: Kann.

    Administrative, technical, or material support: Kann, Aerts.

    Supervision: Kann, Nguyen.

    Conflict of Interest Disclosures: Dr Kann reported having a patent to a computed tomography–based method for analysis of malignant neoplasms pending. Dr Aerts reported receiving personal fees from GenoSpace outside the submitted work. Dr Mak reported receiving personal fees from ViewRay, AstraZeneca, and NewRT Medical Systems outside the submitted work. Dr Nguyen reported receiving grants and personal fees from Janssen Pharmaceuticals, Astellas, and Bayer and receiving personal fees from Dendreon, Boston Scientific, Augmenix, Blue Earth Diagnostics, Ferring Pharmaceuticals, and Cota Healthcare outside the submitted work. No other disclosures were reported.

    Disclaimer: The National Comprehensive Cancer Network makes no warranties of any kind whatsoever regarding their content, use, or application and disclaims any responsibility for their application or use in any way.

    Additional Information: This article was referenced with permission from the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines), copyright National Comprehensive Cancer Network Inc, 2019. All rights reserved. Accessed September 6th, 2019. To view the most recent and complete version of the guidelines, go online to https://www.nccn.org/.

    References
    1.
    Weinstein  IB, Case  K.  The history of cancer research: introducing an AACR Centennial series.  Cancer Res. 2008;68(17):6861-6862. doi:10.1158/0008-5472.CAN-08-2827PubMedGoogle ScholarCrossref
    2.
    National Comprehensive Cancer Network. NCCN history. Accessed September 6, 2019. https://www.nccn.org/about/history.aspx
    3.
    Internet Archive. Wayback machine. Accessed September 9, 2019. https://archive.org/web/
    4.
    NEJM Catalyst. Cognitive load theory and its implications for health care complexity. Published March 14, 2018. Accessed September 6, 2019. https://catalyst.nejm.org/cognitive-load-theory-implications-health-care/
    5.
    Abernethy  AP, Etheredge  LM, Ganz  PA,  et al.  Rapid-learning system for cancer care.  J Clin Oncol. 2010;28(27):4268-4274. doi:10.1200/JCO.2010.28.5478PubMedGoogle ScholarCrossref
    6.
    National Comprehensive Cancer Network. Library of NCCN compendia. Accessed September 9, 2019. https://www.nccn.org/professionals/compendia.aspx
    ×