[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
Views 6,675
Citations 0
Viewpoint
December 2, 2019

Clarifying the Meaning of Clinically Meaningful Benefit in Clinical Research: Noticeable Change vs Valuable Change

Author Affiliations
  • 1Center for Health Measurement, Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina
JAMA. Published online December 2, 2019. doi:https://doi.org/10.1001/jama.2019.18496

Recent years have witnessed an increasing focus on the input of patients, their caregivers, or both to inform the selection, measurement, and interpretation of outcomes in clinical trials. One important issue for which their participation is sought is the question of what constitutes a clinically meaningful benefit, especially when patient-reported outcome measures (PROMs) are used to assess how patients feel and function. There is an extensive literature describing and appraising methods for defining minimally important differences and related terms and concepts.1-5 Several advances have been made in this field, including the recognition that the meaningfulness of differences for a person over time is distinct from differences between groups over time.6 Still, among researchers and manufacturers of drugs and devices, uncertainty remains about how best to determine clinically meaningful change in clinical research.

Uncertainty about the concept of clinical meaningfulness makes it challenging to incorporate the perspectives of patients and caregivers to inform clinical trial objectives, design, analysis, and interpretation. Patients and caregivers can contribute only when researchers and clinicians have clarity about the type of input they are seeking at any given time. Among the complexities surrounding the notion of clinically meaningful benefit, a fundamental confusion is the conflation of 2 different and equally important types of meaningful change. One type denotes change that is noticeable to the patient or caregiver and the other denotes change that is judged to be valuable by the patient or caregiver.

Noticeable Change

For assessments of how people feel or function, noticeable change refers to a change that is perceptible to the person or their caregiver. Most day-to-day communications about health status use words and expressions that convey noticeable change in ways that are easily understood by others in the community. These include communications about the presence or absence of a symptom (eg, “Yesterday, I started to get stomach cramps”) or whether a person can perform adequately in some role (eg, “After the crash, he’s been unable to drive his truck route”). Verbal expressions are used that reflect ordered feeling states, eg, after adjusting the fit of a corrective shoe, the patient says, “Yes, that feels a little better.” There are also descriptions of changes in the level of assistance needed to do something, eg, “With this new medication, she doesn’t need the wheelchair anymore and can walk with a cane.” All of these cases reflect a clear change that is noticed and easily communicated. Thus, it is difficult to imagine raising questions about whether such changes are “meaningful” in the sense of being discernable.

In contrast, there are often questions about whether a change in some PROM would be noticed by the person. Such questions might be raised when individual items have many ordered response options. For example, would people notice if an intervention could improve their “distress thermometer” responses by 2 points on a 100-point response scale? Questions are also raised when a PROM is scored by combining responses from multiple items to create some mathematical composite (eg, a total score). It is then important to know, for example, whether a 5-point change on a 0- to 50-point scale corresponds to a change in health state that is detectable by the person. In cases such as these, additional methodological work is required to determine what size change is noticeable and whether the change required to be noticeable is the same size regardless of an individual’s starting point along the scale (eg, moving from moderate to mild disease vs moving from severe to moderate disease). This methodological work can require significant resources and present conceptual and technical challenges.

Valuable Change

After having established that some noticeable change has occurred, the next question about meaningfulness should be whether the amount of change is viewed as valuable by the patient or caregiver. That is, is the change considered to be worthwhile? The answer to this question will depend critically on the context. A patient might regard a small improvement as worthwhile as long as the personal costs and inconveniences do not outweigh the benefit. But the same size improvement might be judged as less valuable if the treatment produces toxic adverse effects. These examples underscore how determinations of the value of a change is a judgment made in a particular context.

Recommendations

This preceding discussion suggests a number of recommendations for enhancing the incorporation of patient and caregiver input into the use of patient-centered end points in clinical research:

  1. Recognize that the term meaningful as it is currently used in relation to clinical benefit is ambiguous. Replace or supplement use of the term with a specification of either noticeable and/or valuable.

  2. When arbitrary metrics are used in PROMs, there is a burden on the researcher to determine and convince others about what constitutes noticeable change. Therefore, whenever possible, develop PROMs that use items and response options that hew closely to the meaningful distinctions already available in everyday language. This is consistent with the notion that PROMs essentially act as standardized versions of conversations with patients.7 For example, a question in a PROM that asks about the frequency of some symptom could be answered with an actual frequency (eg, “3 times during the day”) rather than with vague quantifiers (eg, never, rarely, sometimes, often, or always). Most would agree that a change from 3 times a day to once a day is noticeable, but it is less clear whether a change from often to sometimes is something that would be consistently noticeable.

  3. What constitutes noticeable change for a particular end point might depend on the person’s starting point (eg, less vs more severe symptoms or functional restrictions), but noticeable change is generally unlikely to vary across contexts. In contrast, definitions of valuable change might be highly influenced by contextual factors such as cost of the intervention, adverse effects, convenience, and the availability and cost-benefits of alternative interventions. Thus, assessments of the value of a given change must include specification of important contextual factors.

More specific methodological issues must be resolved to pursue some of these recommendations. There is also a need to develop best practices and methodological standards for identifying noticeable change and eliciting judgments of valuable change. Better appreciation of these 2 aspects of clinically meaningful benefit should lead to a more effective incorporation of patients and caregivers in the design and interpretation of clinical research.

Back to top
Article Information

Corresponding Author: Kevin P. Weinfurt, PhD, Department of Population Health Sciences, Duke University School of Medicine, 215 Morris St, Ste 210, Durham, NC 27701 (Kevin.weinfurt@duke.edu).

Published Online: December 2, 2019. doi:10.1001/jama.2019.18496

Conflict of Interest Disclosures: Dr Weinfurt reported receiving personal fees from Regeneron and Pacific Business Group on Health outside the submitted work.

Additional Contributions: I am grateful to Ari Gnanasakthy, MBA, MSc (RTI Health Solutions), Juan Marcos Gonzalez Sepulveda, PhD, Dan Mark, MD, MHS, and Bryce Reeve, PhD (Duke University School of Medicine), for review and comments on an earlier draft of this article. They did not receive compensation.

References
1.
Coon  CD, Cook  KF.  Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores.  Qual Life Res. 2018;27(1):33-40. doi:10.1007/s11136-017-1616-3PubMedGoogle ScholarCrossref
2.
Jayadevappa  R, Cook  R, Chhatre  S.  Minimal important difference to infer changes in health-related quality of life-a systematic review.  J Clin Epidemiol. 2017;89:188-198. doi:10.1016/j.jclinepi.2017.06.009PubMedGoogle ScholarCrossref
3.
Revicki  D, Hays  RD, Cella  D, Sloan  J.  Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes.  J Clin Epidemiol. 2008;61(2):102-109. doi:10.1016/j.jclinepi.2007.03.012PubMedGoogle ScholarCrossref
4.
Wyrwich  KW, Bullinger  M, Aaronson  N, Hays  RD, Patrick  DL, Symonds  T; Clinical Significance Consensus Meeting Group.  Estimating clinically significant differences in quality of life outcomes.  Qual Life Res. 2005;14(2):285-295. doi:10.1007/s11136-004-0705-2PubMedGoogle ScholarCrossref
5.
McGlothlin  AE, Lewis  RJ.  Minimal clinically important difference: defining what really matters to patients.  JAMA. 2014;312(13):1342-1343. doi:10.1001/jama.2014.13128PubMedGoogle ScholarCrossref
6.
Coon  CD, Cappelleri  JC.  Interpreting change in scores on patient-reported outcome instruments.  Ther Innov Regul Sci. 2016;50(1):22-29. doi:10.1177/2168479015622667PubMedGoogle Scholar
7.
Weinfurt  KP.  Viewing assessments of patient-reported heath status as conversations: implications for developing and evaluating patient-reported outcome measures.  Qual Life Res. 2019;13(1):1-7. doi:10.1007/s11136-019-02285-8PubMedGoogle Scholar
×