Ethan Basch, Cindy Geoghegan, Stephen Joel Coons, Ari Gnanasakthy, Ashley F. Slagle, Elektra J. Papadopoulos, Paul G. Kluetz. Patient-Reported Outcomes in Cancer Drug Development and US Regulatory ReviewPerspectives From Industry, the Food and Drug Administration, and the Patient. JAMA Oncol. 2015;1(3):375–379. doi:10.1001/jamaoncol.2015.0530
Data reported directly by patients about how they feel and function are rarely included in oncology drug labeling in the United States, in contrast to Europe and to nononcology labeling in the United States, where this practice is more common. Multiple barriers exist, including challenges unique to oncology trials, and industry’s concerns regarding cost, logistical complexities, and the Food and Drug Administration’s (FDA’s) rigorous application of its 2009 guidance on the use of patient-reported outcome (PRO) measures. A panel consisting of representatives of industry, FDA, the PRO Consortium, clinicians, and patients was assembled at a 2014 workshop cosponsored by FDA to identify practical recommendations for overcoming these barriers. Key recommendations included increasing proactive encouragement by FDA to clinical trial sponsors for including PROs in drug development programs; provision of comprehensive PRO plans by sponsors to FDA early in drug development; promotion of an oncology-specific PRO research agenda; development of an approach to existing (“legacy”) PRO measures, when appropriate (focused initially on symptoms and functional status); and increased FDA and industry training in PRO methodology. FDA has begun implementing several of these recommendations.
In May 2014, a panel was assembled for the plenary session of the annual Accelerating Anticancer Agent Development and Validation (AAADV) Workshop in Bethesda, Maryland, to focus on “Patient-reported outcomes (PROs) and patient-focused drug development.” The members of that panel constitute the authors of this piece, and each provided his or her perspective as a representative of the pharmaceutical industry, the Food and Drug Administration (FDA), the PRO Consortium (a collaborative effort between FDA, industry, and the Critical Path Institute to fill PRO measurement gaps),1 clinicians, and patients. Assembly of this panel by FDA was prompted by the observation that data reported directly by patients about their symptoms and functioning are rarely included in oncology drug labeling in the United States.2 This stands in contrast to oncology labeling in Europe and to nononcology labeling in the United States, where PROs are more commonly included.3
Although FDA released a guidance document in 2009 specifying the types of evidence and documentation required for PRO measures to support regulatory approval or labeling claims in the United States,4 these principles have rarely been successfully applied in oncology drug development programs. Patient-reported outcome end points are frequently an afterthought in the design of industry-sponsored clinical trials in oncology and are deemphasized relative to end points related to survival, imaging, and biomarkers. Yet survival advantages of cancer drugs are frequently modest, on the order of weeks or months, and radiographic or biomarker-related end points can be of questionable clinical relevance. Cancer drugs often carry substantial toxicities that may affect how people feel and function. It would therefore be valuable to include information on patient symptoms and function in clinical trials more frequently, and it seems counter-intuitive that PRO end points are not central in the evaluation of cancer drugs.
In this article, perspectives are provided regarding the rationale, barriers, and strategies for integrating PRO end points and guidance principles into cancer drug development and US labeling. The goal is to demonstrate differences and common ground between stakeholders, and suggest steps forward. It is acknowledged that this article reflects individual perspectives of the authors as representatives of their stakeholder categories and may not reflect the overall or complete perspectives of each stakeholder group.
Data reported directly by patients about how they feel are rarely included in cancer drug labeling.
Barriers include methodological challenges, cost and logistics, FDA guidance on patient-reported outcome (PRO) measures, and communication.
An expert panel identified practical recommendations for overcoming these barriers.
Proactive role for FDA in encouraging PRO end points
Provision of comprehensive PRO measurement plans by FDA
Characterization and mitigation of the effect of open-label trial designs
Research on patterns and cases of missing data
Development of best practices for statistical analyses of PRO data
Increased FDA and industry training in PRO methodology
Whereas the recommendations in the 2009 FDA guidance4 on the use of PROs in drug development have increased the overall rigor of PRO measurement in clinical trials, what many believe to be unattainable standards, inconsistency in implementation across the various FDA reviewing divisions, and the time taken to review the evidence (thus delaying the development of new therapies) have become a cause for frustration within the industry.5,6
Data suggest that FDA, and the Office of Hematology and Oncology Products (OHOP) in particular, includes PROs less commonly in their labels than their European oncology counterparts, or US nononcology review divisions.3,7 Although PRO-based labeling claims are granted to approximately one-quarter of all new drugs in the United States, it is a rare occurrence for oncology-related therapies.7 For new oncology-related compounds approved between 2006 and 2013 by FDA, PRO-based labeling claims were granted to only 1 of 43 compounds. This compares to 14 of 42 oncology products approved by the European regulators.7
Within this context, the industry faces major challenges when it comes to integrating PRO measurement in oncology clinical trials:
Despite small improvements in overall survival rates, both regulators and the industry continue to prioritize survival-based end points rather than end points based on patient experience.2
Whereas developing new disease-specific PRO measures to satisfy US regulators can take years, the oncology drug development programs often go through an accelerated drug approval process and generally have compressed development timelines. Of the 17 cancer-related new molecular entities approved by FDA in 2013 and 2014, less than one-third were based on double-blind randomized clinical trials and more than half of approvals went through the accelerated approvals program. Such hurdles, along with the challenges of integrating PRO measures in multinational clinical trials8 and the lack of precedents by US regulators to recognize PRO data to be worthy of labeling claims for oncology compounds,7 as well as the perceived indifference shown by the oncology reviewing divisions toward PRO data, discourage sponsors from integrating PRO end points as part of their evidence package to satisfy US regulators.
Sponsors grapple with small patient numbers, very sick or terminally ill patients, high failure rates, and single-arm studies unique to the development of new therapies in oncology.
Faced with such challenges, it is not a surprise that sponsors include the minimum evidence required to satisfy regulators. Given the relatively high probability of label claims from European regulators, and given the challenges of including disease-specific PRO measures in clinical trials to satisfy FDA, sponsors may include off-the-shelf but well-tested measures such as the European Organization for Research and Treatment of Cancer QLQ-C309 in their confirmatory clinical trials. Data from such measures, although they may not meet current FDA expectations for labeling, are methodologically rigorous and routinely satisfy regulators and payers in Europe, provide data for publication, and are informative for other stakeholders including advisory committees, patients, payers, patient advocacy groups, and clinicians.10,11
It is worth noting that whereas many FDA review divisions routinely expect sponsors to include PRO data as part of the evidence package to assess the efficacy of new therapies, the oncology reviewing divisions rarely do. In the future, ideally medical reviewers within these divisions would encourage sponsors to assess PROs in drug development programs and work collaboratively with sponsors to design a PRO measurement plan that is robust and feasible.
The 2012 Food and Drug Administration Safety and Innovation Act makes it clear that patient-focused drug development is a priority for Congress, FDA, and the American people.12 Whereas there is little argument that measuring a patient’s symptoms or function is important and can be a direct measure of clinical benefit, the challenge lies in how the patient perspective is most accurately captured. FDA requires that substantial evidence be presented to support a labeling or marketing claim of treatment benefit and this necessitates adequate and well-controlled studies using well-defined and reliable assessments.13 When evaluating whether a PRO instrument is adequate to support labeling, the agency places a strong emphasis on content validity (ie, that the instrument measures what it is intended or purported to measure). Content validity is critical in the regulatory setting, where regulators have a duty to ensure that drug labeling is not false or misleading. Unfortunately, many existing PRO measures and trial designs proposed to FDA have had substantial flaws that have precluded their inclusion in the product label.
To facilitate and standardize best practices for PRO instrument development and trial conduct, FDA finalized a 2009 guidance on the use of PROs for medical product development to support labeling claims.4 In addition to content validity, the guidance describes measurement properties (eg, reliability, construct validity, ability to detect change) that should be examined to ensure that an instrument can provide a quantitative assessment of a drug’s effects. Important trial design considerations are also discussed, including the frequency and timing of assessments, administration mode (eg, paper, electronic), and handling of missing data. All of these issues contribute to the adequacy and interpretability of PRO data to support labeling claims, and as with any other end point, if the intent is to make an efficacy claim (eg, drug X improves pain), PRO end points should be prespecified in the statistical analysis plan, with appropriate adjustments for multiplicity. Given the complexity of PRO measure development, FDA encourages sponsors to engage FDA in discussions about these end points as early as possible.
Many concerns voiced by our fellow panelists relate to the perceived strict adherence to FDA PRO guidance. The PRO guidance is a framework for optimal instrument development, trial design, and analysis of PRO data and was written to address all therapeutic areas. We realize that although incorporating all of the principles of the PRO guidance would be ideal, there are cases in which regulatory flexibility might be exerted, particularly in the oncology setting. For instance, we recognize that many oncology trials are performed in an open-label fashion when blinding is considered infeasible. Lack of blinding is a substantial limitation; however, PRO data might still be convincing in the context of a large magnitude of effect demonstrated in the setting of thoughtful trial design and conduct with very little missing data. Limitations of PRO data might also be described in FDA labeling (eg, extent of missing data, open-label nature of a study) to better inform physicians and patients when interpreting results.
Looking forward, it is clear that the successful integration of PROs into oncology product labels will not be accomplished by any one single entity. FDA does not itself develop PRO instruments or conduct clinical trials, and therefore collaborative work is needed to identify a clear research agenda supported by all relevant stakeholders. We must characterize and mitigate the challenges that we face in the oncology setting, including adequacy of instruments, open-label trials, single-arm trials, missing data, and best practices for statistical analyses. It all starts with well-defined and reliable PRO instruments; and with appropriate supportive data, the path forward may include the modification of existing legacy measures that are not considered fit for purpose in their present form. Another underutilized opportunity is the use of the FDA Center for Drug Evaluation and Research (CDER) drug development tool Clinical Outcome Assessment Qualification program, which provides a forum for FDA to work with external instrument developers to provide consultation and advice in order to develop and qualify publicly available outcome assessments for use across multiple clinical trials. FDA encourages all parties interested in PRO instrument development to engage with CDER through this program to increase the number of publicly available tools that are available to support product labeling.
FDA’s OHOP and CDER’s Study Endpoints and Labeling Development staff acknowledge the frustration among patients, drug developers, and outcomes researchers and are themselves disappointed by the low level of attention paid to PRO integration in many oncology clinical trial designs. For our part, we have put substantial effort into optimizing communication both between Study Endpoints and Labeling Development and OHOP and between FDA and stakeholders to improve consistency of PRO advice. There is a renewed focus on identifying key elements of the patient perspective that can be described in all oncology trials, and it is our expectation that we will start to see the quality of data necessary to allow for inclusion in the FDA label. There is reason to be optimistic that we can all do better, and FDA is dedicated to working with all stakeholders to address these issues.
When I (C.G.) received a diagnosis of breast cancer 19 years ago, I struggled with my role in deciding on optimal treatment, with very limited relevant information and no access to the Internet or peer support. Two decades later, patients with cancer continue to struggle in their broadened decision-making roles.
As we move to invoke a “patient-centered” health care system, patients who already struggle to navigate life-saving treatment decisions will soon be expected to drive their own care. Patient-reported outcomes data aggregated from prior trials are necessary to enable people to make rational decisions about treatments that balance impact on survival, as well as symptoms and overall quality of life and financial impact. As a community of patients and researchers, we can no longer tolerate the lack of routinely and thoughtfully incorporated PRO evidence from cancer clinical trials.
Patients will always strive for survival as their goal and as a clinical end point. When faced with a decision to choose between treatments that have shown very modest benefit without extending life, they need to know what they can expect to feel and how they can expect to function.
Without systematically collected PRO data, patients are forced to fill information gaps by searching the Internet, where they can connect instantly with individuals—patient peers and clinical trial participants—who are eager to help with real-time “peer-reported outcomes” and personal experiences. With little effort, patients already access disease-focused websites, discussion boards, Facebook posts, blogs, and Twitter feeds to find anecdotes about specific therapies, adverse events, quality of life improvements, treatment failures, and extraordinary benefits. But the patients posting and consuming these anecdotal data cannot assess accuracy or generalizability.
As a patient, I find it hard to understand how FDA can continue to evaluate the potential benefits and harms of cancer drugs that are toxic, with survival benefits measured only in days or weeks, without a comprehensive understanding of patient experiences elicited through PROs. Patients are being encouraged to demand more from clinical trials,14 and in turn we need to demand more from FDA, industry, and PRO measure developers. FDA should work with trial sponsors to ensure that they routinely collect and provide PRO information as the default expected approach for every new drug application—so that this information can be used not only as part of the application but also so that subsequent patients can understand the potential impact on their lives. Drug developers need to collaborate with patients and with FDA to ensure that they are capturing information that is meaningful to understanding how patients feel and function while receiving new therapies. Patient-centered care cannot be delivered without patient-centered outcomes information.
Although there is considerable frustration among a variety of stakeholders with the lack of PRO-based end points in cancer trials, FDA’s CDER has taken steps in the right direction. CDER took a leadership role in forming the Patient-Reported Outcome Consortium in 2008 in conjunction with the Critical Path Institute and the pharmaceutical industry.1 The mission of the PRO Consortium is to develop and achieve FDA qualification15 of PRO measures for use in clinical trials in which PRO-based efficacy end points are used to support product labeling claims. Currently, 27 pharmaceutical firms are working collaboratively within the PRO Consortium to identify and fill PRO measurement gaps. The goals of the PRO Consortium are to
Enable precompetitive collaboration that includes FDA input and expertise
Avoid development of multiple PRO measures for the same purpose
Share costs of developing new PRO measures
Develop qualified, publicly available PRO measures
Facilitate FDA review of medical products by standardizing PRO end points
In conjunction with FDA, the PRO Consortium member firms identified non–small-cell lung cancer (NSCLC), breast cancer, advanced ovarian cancer, and colorectal cancer as diseases for which there was sufficient need and interest in pursuing qualification of a PRO-based end point measure. At present, the NSCLC Working Group is the only active cancer-related working group within the PRO Consortium. The Breast Cancer Working Group was established but put on hold, and working groups for advanced ovarian cancer and colorectal cancer were not established pending the outcome of the NSCLC Working Group’s efforts. Among other impediments (eg, challenges associated with achieving consensus within a multicompany collaboration), the NSCLC Working Group and FDA have expended an inordinate amount of time and effort interacting in regard to the target population (ie, stage, treatment status/pathways, and performance status) for the measure’s context of use. Members of the PRO Consortium want to be sure that qualification of a PRO measure for NSCLC trials is possible before investing the considerable resources necessary to develop measures for the other cancer sites.
A number of obstacles have led to concern that the hurdle for PRO end point measures for oncology trials is too high. For one, even though blinding in oncology trials is often impossible to achieve, FDA communicated to the NSCLC Working Group that “even well-defined and reliable symptom assessments are rarely credible in a study where patients are unblinded to study treatment” (written communication, September 2010). In addition, FDA has been resistant to the measurement of fatigue, a common and potentially debilitating symptom of cancer and its treatment. In these cases, it seems that FDA is allowing the perfect to be the enemy of the good (enough).
Patient-reported outcome instruments measure how patients feel and/or function, which are more direct assessments of treatment benefit than many other end points. Hence, it is critical that obstacles to the use of PRO-based end points in oncology trials be addressed and minimized through more effective and constructive dialog between the primary stakeholders (ie, patients, clinicians, industry, FDA, contract research organizations, and PRO instrument developers).
This article presents relatively uncensored perspectives, and in some cases frustrations, of different stakeholder representatives striving to incorporate information in oncology drug development about the impact of treatments on the patient experience. A cycle of negative feedback exists between FDA and industry, in which FDA asserts that industry puts little effort into developing rigorous PRO end points and industry asserts that it is not worth the effort because FDA’s bar is too high. All of us note the importance of breaking this cycle to ensure progress in making oncology drug development more patient centered.
The Box provides specific recommendations derived from the common threads of stakeholder perspectives. Areas of particular emphasis include improving communication between the industry, FDA, and patient groups; early consideration of integrating PROs in drug development programs to facilitate any needed development work; and expansion of PRO expertise in the industry and FDA. It is often expected that a given PRO measure will be developed or evaluated in the planned context of use in order to be fit for purpose in a given drug development program or pivotal trial. For example, if a measure was developed for measuring symptoms in breast cancer and now is of interest in prostate cancer, some additional development work might be expected, for example, qualitative work to ensure that the concepts or questions are meaningful and relevant. The extent of additional work will depend on how similar or different the new population is from the prior population. If there are substantial differences, a new or different measure may need to be developed and/or tested. Because it may require some lead time to do such work, early discussion between sponsors and FDA is important to avoid any delays associated with such work and to facilitate potential inclusion of such work within clinical studies (eg, during phase 2, but can be considered as early as pre–Investigational New Drug Application).
Medical reviewers within the Food and Drug Administration’s (FDA’s) Office of Hematology and Oncology Products should be more proactive in encouraging sponsors to include the assessment of PROs in their drug development programs.
Trial sponsors should provide thoughtful and comprehensive proposals for PRO measures and data collection methods and should describe deviations from the PRO guidance with a rationale for why these deviations may be acceptable in their trial context. Early discussions with FDA can facilitate this process.
Patient-reported outcomes stakeholders should identify a clear research agenda to characterize and mitigate known limitations in the oncology setting including adequacy of existing instruments, the effect of open-label trial designs, patterns and causes of missing data, and best practices for statistical analyses.
A systematic approach to analyzing existing (“legacy”) questionnaires should be developed as a basis for FDA to assess and publicly endorse those that meet acceptable criteria for use in oncology registration trials toward labeling, with the understanding that there may not be perfect adherence to FDA guidance principles.
Stakeholders from industry, academia, patient groups, and FDA should work together to identify or develop PRO measures of outcomes that are important to patients (eg, physical function and fatigue) that could be endorsed as acceptable by FDA for cancer drug product labeling.
The FDA should support training of additional FDA personnel to accelerate dialog with product developers and other stakeholders about the development, qualification, implementation, and analysis of PRO measures. Inclusion of individuals with PRO expertise within FDA clinical review divisions will facilitate balancing methodological concerns with a realistic understanding of clinical issues and logistical challenges associated with oncology clinical trials.
Industry and academia should work together to develop training programs for those interested in PRO assessments for use in the regulatory setting and should consider including patient advocates in these initiatives.
As a part of FDA’s Patient-Focused Drug Development program, a number of workshops are being sponsored to openly discuss barriers to bringing the patient perspective into drug development more broadly, and to consider ways to implement solutions. Standardization of best practices across the PRO research continuum—from identification of concepts to measure, to instrument selection, to trial design and conduct, to data analysis and presentation—will be critical to success. It is our hope that the frequency of important and interpretable PRO data in oncology drug labels will increase in the coming years as a result of these efforts.
Corresponding Author: Ethan Basch, MD, MSc, Cancer Outcomes Research Program, Lineberger Comprehensive Cancer Center, University of North Carolina, 170 Manning Dr, Chapel Hill, NC 27599 (firstname.lastname@example.org).
Published Online: April 16, 2015. doi:10.1001/jamaoncol.2015.0530.
Conflict of Interest Disclosures: Dr Basch is an employee of the University of North Carolina with grant funding from the National Cancer Institute and membership on the Methodology Committee of the Patient-Centered Outcomes Research Institute. Dr Coons is an employee of the Critical Path Institute, which is supported, in part, by grant 1U18FD005320 from the US Food and Drug Administration (FDA). During preparation of this manuscript, Mr Gnanasakthy was an employee of Novartis, and he is currently an employee of RTI Health Solutions. Drs Slagle, Papadopoulos, and Kluetz are employees of FDA. No other disclosures are reported.
Disclaimer: The views expressed in this article do not necessarily represent an official position of FDA, the Patient-Centered Outcomes Research Institute, or the Critical Path Institute.