Addressing Risks to Advance Mental Health Research

Importance  Risk communication and management are essential to the ethical conduct of research, yet addressing risks may be time consuming for investigators and institutional review boards may reject study designs that seem too risky. This can discourage needed research, particularly in higher-risk protocols or those enrolling potentially vulnerable individuals, such as those with some level of suicidality. Improved mechanisms for addressing research risks may facilitate much needed psychiatric research.

Objective  To provide mental health researchers with practical approaches to (1) identify and define various intrinsic research risks, (2) communicate these risks to others (eg, potential participants, regulatory bodies, and society), (3) manage these risks during the course of a study, and (4) justify the risks.

Evidence Review  As part of a National Institute of Mental Health–funded scientific meeting series, a public conference and a closed-session expert panel meeting were held on managing and disclosing risks in mental health clinical trials. The expert panel reviewed the literature with a focus on empirical studies and developed recommendations for best practices and further research on managing and disclosing risks in mental health clinical trials. No institutional review board–review was required because there were no human subjects.

Findings  Challenges, current data, practical strategies, and topics for future research are addressed for each of 4 key areas pertaining to management and disclosure of risks in clinical trials: identifying and defining risks, communicating risks, managing risks during studies, and justifying research risks.

Conclusions and Relevance  Empirical data on risk communication, managing risks, and the benefits of research can support the ethical conduct of mental health research and may help investigators better conceptualize and confront risks and to gain institutional review board–approval.

With 26.2% of US adults afflicted with a psychiatric disorder, improvements in psychiatric treatments are needed.¹ Despite advances for psychiatric disorders, there has been increased public scrutiny of how mental health research is conducted and the risks to participants, particularly in the wake of concerns regarding unpublished data on selective serotonin reuptake inhibitors (SSRIs) and suicide,^2,3 studies on schizophrenia in which medication is withheld,⁴ and increased litigation.⁵ Researchers must proactively address risks—from minimizing risks inherent to study design (eg, placebo-controlled studies), to mitigating potential participant misunderstanding of the research purpose, and to addressing risks of worsening symptoms during research participation (eg, emergence of suicidal ideation or psychosis). Yet, the means of addressing risks are not always readily apparent, particularly when those risks may be inherent in the overall research endeavor or associated with the illness itself, not the research. Institutional review boards (IRBs) and sponsors may halt or delay studies over concerns about participant risks. Increased investigator burdens because of real and perceived risks may discourage much-needed research.

Quiz Ref IDFor example, suicide is clearly associated with a history of depression and is the most serious potential outcome of depression, but only 10% of clinical trials examining SSRI efficacy between 1984 and 2001 included individuals with some level of suicidality; there are no SSRI efficacy trials in actively suicidal individuals.⁶ Because the presence of suicidal ideation is a typical exclusion criterion in research, the community is left with “evidence-based” guidance that is generalized to these excluded individuals. This pattern of exclusion has resulted in a dearth of evidence regarding treatment for individuals with varying degrees of suicidality. To facilitate research in this area, suicide should be treated as a negative outcome comparable to negative outcomes of other diseases—an undesirable yet to-be-expected event.^7,8 To ignore this is to continue the unwarranted and unjust exclusion of these individuals.

Mental health researchers must strive to design ethical and scientifically sound research that does not ignore populations or kinds of research merely because of the difficulties involved. If mental health researchers continue to allow risks to discourage research, certain groups that could benefit from research will continue to be harmed by being understudied. Groups perceived as “high risk” deserve scientifically rigorous study as well. Researchers must be able to identify and communicate risks and potential benefits of this research to the public and regulatory bodies, demonstrate that they will manage risks effectively, and provide strong ethical justification for such research.

This article provides mental health researchers with practical approaches to (1) identify and define various intrinsic research risks, (2) communicate risks to others (eg, potential participants, regulatory bodies, and society), (3) manage risks during the course of a study, and (4) justify the risks. These recommendations are the result of a systematic literature review, a public conference during which authors participated as speakers or panelists, a closed-door working session that included all of the authors, and subsequent e-mail and telephone conversations to formulate consensus recommendations. All authors participated in the writing and editing process.

The Belmont Report⁹ provides the ethical framework for US research regulations and identifies 5 primary kinds of harm relevant to research review and oversight. These are summarized in the Table.

In addition to recognizing the kinds of harms that study participation may pose, investigators and regulatory bodies must determine the studies’ overall risk level. Although risk is an inherent aspect of human life and familiar to all, it is difficult to define operationally. Quiz Ref IDMost regulatory and philosophical definitions of risk include 3 primary concepts: harm and its probability and magnitude. However, there is no agreed-on formula for determining risk level. The Common Rule recognizes 2 risk categories for adult research: minimal and greater than minimal. Minimal risk is defined as “the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests.”¹⁰ Whose daily life and which examinations are routine are not specified and instead left to IRB judgment. The Department of Health and Human Services identifies 3 categories of risk for research involving children: minimal risk, greater than minimal risk with prospect of direct benefit, and minor increase over minimal risk with no direct benefit. Studies that do not fit these categories require special federal review.¹⁰

To foresee and address research risks, investigators must identify the types, probability, and magnitude of harm that their study poses and describe risk information in protocols and informed consent discussions and forms clearly. Regulatory bodies, such as IRBs, ultimately define the study risks relevant to review and determine whether the “[r]isks to subjects are reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result.”¹⁰ However, guidance about defining and evaluating risks and benefits is limited and often focused on determining the extent to which additional protections are needed for studies deemed to be higher risk. Researchers may educate IRB members about study risks, including a comparison of research risks across different types of studies, when they have data that might inform IRB determinations.

There is no consensus on how best to evaluate and rate research risk levels, and IRBs are inconsistent in how they evaluate risk. Shah et al¹¹ found significant variability among IRB chairs’ rating of risk level. Similar studies of IRB practices have demonstrated inconsistencies in rating protocols as eligible for expedited review vs requiring full review, a determination based largely on assessment of risk level since expedited studies must be minimal risk.^12-14 This variability results in uneven participant protections and delays in start dates for funded research.^14,15 These issues are magnified in large-scale multisite studies, which can provide greater statistical power with more meaningful and generalizable outcomes yet require review by multiple IRBs.¹⁴

Quiz Ref IDDespite data indicating that mental health research is as safe as most other medical research and that participants typically possess the capacity to give consent for themselves,^16-21 mental health researchers may find that IRBs, which are not required to and may not have members with substantive expertise in mental health research,¹⁰ judge their studies as posing higher risks than comparably risky non–mental health studies and believe that potential participants are less likely to be able to understand risks and given informed consent.^22,23

Strategies for appropriately identifying research risks are summarized in Box 1 and include the following:

Box Section Ref ID

1. Risks should be identified and defined early in study development. The Research Protocol Ethics Assessment Tool (RePEAT), developed by Roberts and Roberts,²⁴ is essentially a checklist that covers multiple ethical domains, including scientific merit of the protocol; risks and benefits; the expertise, commitment, and integrity of the research team; informed consent and decisional capacity issues; incentives for participation; and confidentiality. Researchers can apply this tool (or variants of it) to their protocols to ensure that ethically important research elements are proactively addressed.

2. Study proposals should explicitly identify and address foreseen potential risks, how these risks will be communicated to potential participants, and how emerging risks will be managed, depending on risk level, once the study begins, including, but not limited to, the use of data and safety monitoring boards (DSMBs) or external monitors. Strategies for communicating and managing risks are explored below.

3. To conduct research with groups that IRBs often exclude owing to their “vulnerability,” mental health investigators may need to work harder than other investigators to document for IRBs evidence of research risks and the abilities of potential participants (who are otherwise likely to be perceived as vulnerable) to understand risk information and provide informed consent. For example, researchers may collaborate with community advisory boards, review the literature, and conduct focus groups to support inclusion.

A better understanding of participant and IRB views and decision making regarding risk would assist researchers in designing studies. Future research should aim to identify and clarify the risks that matter most to potential participants, including understanding their concerns about privacy violations, legal risks, social harms at the individual level, and the burdens of participating in studies. Finally, more research on how IRBs evaluate risk, perceive institutional risk, compare risks and potential benefits, and make decisions could help investigators explain studies better when applying for IRB approval. Such research might also identify areas for IRB education.

For example, one study compared IRB member judgments of the consent capacity of potential participants in oncology, chronic pain, and major depressive disorder research studies. Persons with major depressive disorder were judged as having significantly less consent capacity than those with non–mental health diagnoses even when IRB members were told of the high incidence of depression as a comorbid condition in patients with cancer and chronic pain. The IRB members also judged the psychiatric study as posing greater legal risk to the institution than the other studies.^22,23 If these apparently inconsistent analyses of capacity and risk by IRBs are typical, it would be appropriate to develop educational interventions to help IRBs develop new strategies for evaluating studies. Knowing whether there is a significant difference between perceived and actual institutional risk might help facilitate research. Future research needs are summarized in Box 2.

Box Section Ref ID

The regulatory¹⁰ and ethical⁹ obligation to obtain informed consent from research participants (or their legally authorized representatives) requires effective risk communication.²⁵ Risk communication is an information-sharing process in which investigators disclose research risks and benefits in language potential participants can understand and elicit their need for additional information and in which participants understand and appreciate the information relevant to their research participation decision.

Numerous factors can impede effective risk communication for research participants. Individuals with certain mental health conditions may face additional barriers, some of which can be overcome with appropriate interventions.^26-30 Many barriers identified here are exacerbated by heavy reliance on informed consent forms as the primary means for disclosing risks and by expectations that potential participants will spontaneously ask for further information or clarification. The factors to consider are as follows.

Risk information has 5 dimensions: identity, permanence, timing, probability, and value or seriousness.³¹ Effective risk communication requires disclosure, understanding, and appreciation of all 5 dimensions. Providing too much information at once, however, particularly without categorizing it or distinguishing between less and more significant and probable risks, impedes effective risk communication. Informed consent documents that incorporate required information from different sources can become long and difficult to read.³² Risk communication involves objective, factual, and subjective information whose relevance depends on individual values, goals, and priorities. There may be a tendency to approach this process with a more legalistic or formalistic vision, whereby disclosure is seen as a standardized process without attention to the specific information particular participants might need and value.³³

Adults with average cognitive capacities do not always understand and use basic mathematical and statistical concepts well.^34,35 Understanding and appreciating risks requires a conceptual understanding of probability and basic mathematical skills. Poor numeracy interferes with this ability. If the person disclosing risk information does not understand these concepts well, disclosure may be inadequate.

Tversky and Kahneman^36,37 and others have identified ways in which cognitive biases affect the understanding and interpretation of numerical information. Cognitive biases include framing (information about the chance of a risk materializing and not materializing is understood differently), compression (small risks seem greater than they are and large risks seem smaller), availability (culturally significant or otherwise well-known risks are overestimated), anchoring (risks are estimated relative to a familiar risk), and comparison bias (risk perception changes when given comparative risk information).^38,39

Quiz Ref IDPoor health literacy is well documented in the United States⁴⁰ and can interfere with the ability to understand and appreciate research risk information. Risk information often is communicated in writing. Given the estimated numbers of functionally illiterate (40 million) and marginally literate (50 million) adults in the United States,⁴¹ written communication may be unreliable, particularly when written at a high reading level, as often is the case with research consent documents.^42,43 Poor literacy may pose special concerns in psychiatric research because adults with certain disorders may read several grades below their educational level.^43-46

The decision to participate in a study is a series of decisions to enroll and remain in a study. Individuals must be able to recall information about the study, including risk information, to make these decisions. Individuals often have difficulty retaining and recalling risk information.^47-49 Working memory limitations also can impede understanding and appreciation of risk information, which requires processing and manipulating significant amounts of information simultaneously. This poses special concern for potential participants with mental health conditions that adversely affect working memory, such as schizophrenia, mild cognitive impairment, Alzheimer disease, and other forms of dementia.

Risk evaluation is value laden, and different individuals may evaluate the same risks differently. Knowing which risks individuals would want disclosed can improve risk communication. This envisions a more subjective standard of disclosure over a purely formalistic approach (akin to an “information dump”), as discussed in clinical decision making.⁵⁰

Potential participants not only must understand and appreciate research risks but must also evaluate them in light of the potential benefits of participation. Exaggerating risks, using vague information regarding benefits, or allowing potential participants to think that research participation poses greater personal benefit than it does can impede effective risk communication.⁵¹

Existing research suggests that specific practices can foster effective risk communication. These recommended practices are summarized in Box 1 and include the following:

1. Use plain language (see www.plainlanguage.gov).

2. Use interactive informed consent processes that can foster decisional capacity, including understanding and appreciation of risks, among persons with different learning styles and capacities.^{27,30,50,52-54}

3. Use graphs, charts, or other means of presenting information, particularly numerical information, to promote understanding among persons with different learning styles and capacities.^55-57

4. Assess the capacity for consent, not only to determine whether a person is ready to give valid informed consent but also because specific processes designed to assess capacity involve educational interventions that can foster understanding and appreciation.^58-60

5. Avoid unnecessary vagueness, and provide necessary information regarding research risks and benefits, distinguishing between probable and remote risks or benefits.^51,57,61-63

6. Review research information periodically to address retention and recall.⁶¹

There are few studies on risk communication involving actual participants or persons contemplating enrollment in a real study. Studies about participation in hypothetical trials may not be fully generalizable to real-world decision-making contexts.

As summarized in Box 2, for each barrier identified, additional research is needed to better understand the nature of the barrier and identify cost-effective mechanisms that can overcome it. There also remain important normative questions, such as what level of understanding and appreciation among potential participants is necessary for valid informed consent.⁶⁴

High-risk studies are essential to improving knowledge of psychiatric disorders and effective treatment. Effective risk management during these studies is critical. However, IRBs may pay little attention to the investigator’s obligation to minimize risks⁶⁵ and may either disapprove studies perceived as higher risk or approve studies with insufficient risk management plans. This makes it particularly important for mental health researchers to provide evidence-based risk information and propose appropriate risk management plans when seeking IRB approval. Studies may be considered high risk because the intervention is novel, the study population is deemed high risk, the study targets risky behavior (eg, suicidality or psychosis), or the study targets disorders in which risky behaviors are features of the illnesses (eg, depression or bipolar disorder). Automatic exclusion of participants considered high risk or removal of participants exhibiting risky behavior limits generalizability.⁶⁶ Risk management strategies instead focus on trying to initiate and maintain study participation safely.

Data and safety monitoring boards may monitor studies, although they can be costly and there has been controversy about their independence from study sponsors. A July 2010 New England Journal of Medicine editorial proposed that DSMBs should be chosen and convened “under the aegis of an independent public body, such as… the Foundation for the National Institutes of Health,” and stated that the journal will be examining the independence of DSMBs for future manuscript submissions when appropriate.^67(p478)

For longitudinal studies in populations at risk of fluctuating levels of decisional abilities, there is concern about diminishing capacity and the possibility that participants will be unable to protect their interests if new risks arise during a study. Use of third-party participant advocates who ensure that a research participant’s interests remain protected during a long-term study should the participant exhibit diminished capacity may help researchers allow individuals to remain in a study despite fluctuating or diminishing decisional capacity without adding significant costs.

Quiz Ref IDFor example, the large-scale, 18-month-plus, Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia study examining long-term outcomes used such advocates.⁶⁸ At enrollment, participants designated an advocate. Although advocates participated in the initial informed consent discussions, all participants had the capacity to give consent. Advocates received training to determine whether the risk-benefit ratio acknowledged by the participant at time of enrollment had changed “substantially and adversely,”⁶⁸ thus no longer reflecting the participant’s expressed interests. If so, participants were withdrawn. However, if the risk-benefit ratio was not substantially altered, then the advocate could permit the participant to continue participation. Stroup and Appelbaum⁶⁹ surveyed research personnel and a subset of participants and advocates when participants left or completed the study. A majority in all groups favorably viewed the advocate process; 12% of sampled participants believed that the process negatively affected their autonomy.⁶⁹

Managing risk during a study may also involve altering standard study designs to address safety concerns and accounting for these changes in data analysis.^70,71 Consider the National Institute of Mental Health–funded randomized controlled Prevention of Suicide in Primary Care Elderly–Collaborative Trial (PROSPECT).⁷² This study randomly assigned depression health specialists in primary care clinics (the study intervention) and measured depression and suicidal ideation rates in elderly primary care patients. These outcomes were compared with those in clinics that were assigned to an enhanced treatment-as-usual (TAU) protocol. Individuals with suicidal ideation were not excluded from participation, and rates of suicidal ideation were examined over time. Unlike a true TAU clinical setting, all participants assigned to TAU received enhanced care in the form of increased psychiatric surveillance (screening and assessment). This enhanced TAU design created potential study limitations, but it was necessary for study design purposes (to initially identify cases and evaluate at follow-up study visits) and to meet stringent standards for assuring participants’ safety.⁷³ Similar designs with high-risk participants have been used in other clinical trials^74,75 and support the notion that such approaches facilitate important research that otherwise would be constrained.

Training research staff how to assess and manage risks and anxiety-provoking situations (eg, using role play) has been shown to be effective.^76,77

Investigators may implement a number of strategies to manage research risks. There are 3 principal strategies:

1. Conduct research ethics consultations, which involve “interaction between researchers and other stakeholders in the research enterprise” and a person (or persons) knowledgeable about the ethical considerations in research, regarding an ethical question related to any aspect of planning, conducting, interpreting, or disseminating results of research related to human health and well-being. The purpose of the interaction is to provide information; identify, analyze, and/or deliberate about ethical issues; and recommend a course of action.”^78(p3)McCormick et al⁷⁹ reported that psychiatric researchers were more likely than other researchers to find such services useful.

2. Plan for ongoing research staff training on the use of management plans and ways of responding to different situations. The principal investigator or a qualified coinvestigator should be readily available to research staff during the study.

3. Identify situations that necessitate a written risk management plan. Such plans may include tools such as checklists for easy and systematic implementation of specific plan elements. Studies examining specific risky behavior require explicit procedures that include frequent assessment and crisis intervention when needed. For example, if investigators anticipate psychiatric emergencies, the plan should address how these will be handled, who will be involved, and how and to what extent confidentiality will be protected. Management plans may include a protocol for handling dropouts from the study and for obtaining follow-up information or maintaining an up-to-date list of multiple ways to contact participants. Risk management plans should clearly delineate the roles of research team members and the relationship between researchers and clinicians in managing safety. These plans should be detailed in the protocol, and elements relevant to participant rights or willingness to participate should be disclosed during the informed consent process.

Further research is needed to create evidence-based risk management plans and train investigators on risk management. This is especially important given the need for more research in high-risk areas, such as suicide. Research to develop empirically based guidance about risk management in mental health research would benefit researchers and regulatory bodies alike. This research agenda should include further examination of the specific types of risky situations necessitating intervention and the specific interventions that are effective; the roles and responsibilities of researchers, clinicians, and clinician-researchers with respect to risk management; and the perspectives of participants, families, and regulatory bodies about the use of risk management strategies in research that carries greater than minimal risk.

The Belmont Report and current regulations consistently require that risks be evaluated in light of potential benefits to participants and society; research that is scientifically invalid or otherwise lacks value is unethical insofar as it wastes resources and unnecessarily exposes participants to risk.^64,80 Protecting the scientific validity of a study requires management of conflicts of interest that may bias the conduct or reporting of a study.⁸¹ Trust in psychiatric research has been harmed by reports of investigators who conducted research of questionable validity with inadequately disclosed and unmanaged conflicts of interest.⁸²

One frequently heard criticism of mental health research is that, despite years of investigation, the field has made few significant advances.⁸³ However, at least 2 federally sponsored studies have shown that available treatments for psychiatric disorders are comparable in efficacy to treatments in other medical specialties.^84,85 The lack of research on some types of patients may explain the lack of progress in some areas. Moreover, the development of beneficial treatments is often extremely complex, involving false starts, competing interests, and stigma.

These factors are illustrated well in the research history of lithium, which has been proved effective as an acute and maintenance treatment^86,87 in bipolar disorder and is approved by the Food and Drug Administration for these indications. Several studies show that lithium significantly reduces rates of suicide,^88,89 yet despite 60 years of clinical use, it has not been systematically and thoroughly studied in either pediatric⁹⁰ or geriatric⁹¹ populations. Furthermore, there have been few controlled studies of lithium in “dual-diagnosis” populations (eg, patients with both bipolar disorder and substance abuse or dependence).⁹² Ironically, suicide rates are highest in elderly populations, with affective illness a potent risk factor,⁹³ suggesting that the potential benefit of research in this population is reasonable relative to the risks. Similarly, dual-diagnosis status (bipolar and substance use disorder) is associated with poor prognosis and treatment resistance.^92,94 Thus, 2 populations at significant risk for morbidity and mortality are understudied with respect to lithium treatment.

Reluctance to undertake research in these groups may stem from legitimate concerns regarding lithium’s adverse effects and potential toxicity⁹⁵; however, other factors may be at work. For example, unlike many anticonvulsants and “atypical” antipsychotics, lithium receives little marketing support from pharmaceutical companies.⁹⁶ The role of stigma surrounding lithium must also be considered. Scott and Tacchi describe engaging in cognitive therapy with clients with bipolar disorder to address inaccurate perceptions that “if they took lithium prior to important meetings then they would be like a zombie, namely they believed their cognitive functioning would be impaired.”^97(p388) Clearly, if IRB members, researchers, or potential study participants buy into the “zombie” myth, research on lithium is likely to be impeded—notwithstanding a World Health Organization study estimating that lithium treatment saved more than $145 billion in hospitalization costs in the United States between 1970 and 1994.⁹⁸

It is also important to determine which benefits matter most to participants and when and why they matter most. Some data suggest that the interests of researchers and funding agencies do not always align with those of participants, the primary stakeholders in mental health research. For example, mental health consumers tend to prioritize more than researchers regarding alternative treatment modalities (eg, nutritional or self-help modalities), iatrogenic harms, and qualitative research.^99,100

The ethical conduct of research requires investigators to identify, communicate, and manage risks effectively. Investigators’ efforts to identify and communicate research risks and potential benefits clearly, demonstrate knowledge of participants’ perceptions of risks and the kinds of potential harms that most concern them, document potential participants’ capacity to give informed consent, and develop plans to monitor risks during the study and identify new risks may help researchers improve their studies and collaborations with IRBs. For individual investigators, providing IRBs with tables that explain risk information and clear management plans may be useful. Continued research is required to advance researchers’ and IRB understanding of how best to identify, define, effectively communicate, and manage risks. Such efforts are essential to facilitating much needed mental health research.

Submitted for Publication: October 10, 2012; final revision received March 12, 2013; accepted March 13, 2013.

Corresponding Author: James M. DuBois, PhD, DSc, Albert Gnaegi Center for Health Care Ethics, St Louis University, Salus Center, 3545 Lafayette, 5th Floor, St Louis, MO 63104 (jdubois@wustl.edu).

Published Online: October 30, 2013. doi:10.1001/jamapsychiatry.2013.2105.

Author Contributions: Drs Iltis and DuBois had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition of data: Iltis, Misra, Glowinski, DuBois.

Analysis and interpretation of data: Iltis, Misra, Glowinski, DuBois.

Drafting of the manuscript: Iltis, Misra, Dunn, Glowinski, Hadley, DuBois.

Critical revision of the manuscript for important intellectual content: Iltis, Misra, Dunn, Brown, Campbell, Earll, Glowinski, Pies, DuBois.

Obtained funding: DuBois.

Administrative, technical, and material support: Iltis, Misra, Brown, Hadley, DuBois.

Study supervision: Iltis, Misra, DuBois.

Conflict of Interest Disclosures: Dr Dunn reports serving as a consultant in 2012 to Eli Lilly Inc. No other disclosures were reported.

Funding/Support: This work was supported by the National Institute of Mental Health (grant 1R13MH079690) and the National Center for Advancing Translational Science (grant UL1 TR000448), National Institutes of Health.

Role of the Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.