Collaboration With People With Lived Experience of Mental Illness to Reduce Stigma and Improve Primary Care Services: A Pilot Cluster Randomized Clinical Trial | Psychiatry and Behavioral Health | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Visual Abstract. Collaboration With People With Lived Experience of Mental Illness to Reduce Stigma and Improve Care
Collaboration With People With Lived Experience of Mental Illness to Reduce Stigma and Improve Care
Figure 1.  Training as Usual (TAU) vs Reducing Stigma Among Healthcare Providers (RESHAPE) Training Strategy
Training as Usual (TAU) vs Reducing Stigma Among Healthcare Providers (RESHAPE) Training Strategy

Differences in strategies for PCPs being trained to deliver mental health services in primary care settings. In TAU, PCPs are trained by mental health specialists for 9 days on the mhGAP-IG vs the RESHAPE training strategy in which people with lived experience of mental illness and aspirational figures cofacilitate sections of the training alongside mental health professionals. mhGAP-IG indicates Mental Health Gap Action Programme-Intervention Guide; PCPs, primary care practitioners.

Figure 2.  Flowchart for RESHAPE Pilot Cluster Randomized Clinical Trial
Flowchart for RESHAPE Pilot Cluster Randomized Clinical Trial

CONSORT flowchart for pilot cluster randomized clinical trial comparing training as usual (standard Mental Health Gap Action Programme-Intervention Guide implementation) with the experimental condition (Reducing Stigma Among Healthcare Providers [RESHAPE] version of Mental Health Gap Action Programme-Intervention Guide training which includes people with lived experience of mental illness and aspirational figures as cofacilitators). PCP indicates primary care practitioner; RESHAPE, Reducing Stigma Among Healthcare Providers.

Figure 3.  Stigma, Knowledge, Competency, and Diagnostic Accuracy Outcomes for Primary Care Practitioners in TAU vs RESHAPE
Stigma, Knowledge, Competency, and Diagnostic Accuracy Outcomes for Primary Care Practitioners in TAU vs RESHAPE

A, Primary care practitioners’ (PCPs’) absolute change and 95% CI for baseline (pretraining) to midline (4 months after training) and baseline to end line (16 months after training) for training as usual (TAU) and Reducing Stigma Among Healthcare Providers (RESHAPE). B, Diagnostic accuracy in standardized role-plays is presented for 4- and 16-months posttraining, and diagnostic accuracy with actual patients is presented for the period of 14 to 22 months after training. Sample size: midline primary care practitioners (n = 40 TAU; n = 38 RESHAPE); end line primary care practitioners (n = 33 TAU; n = 33 RESHAPE); actual patients (n = 31 TAU; n = 51 RESHAPE). ENACT indicates Enhancing Assessment of Common Therapeutic Factors; IAT, Implicit Association Test; mhGAP, mental health Gap Action Programme-Intervention Guide.

Table 1.  Demographics of PCPs Reported at Enrollment, and Group Differences for End Line Analysis Participants and Those Lost to Follow-up
Demographics of PCPs Reported at Enrollment, and Group Differences for End Line Analysis Participants and Those Lost to Follow-up
Table 2.  PCPs’ Training Outcomes and Clinical Accuracy by Study Group
PCPs’ Training Outcomes and Clinical Accuracy by Study Group
1.
Semrau  M, Alem  A, Abdulmalik  J,  et al.  Developing capacity-building activities for mental health system strengthening in low- and middle-income countries for service users and caregivers, service planners, and researchers.   Epidemiol Psychiatr Sci. 2018;27(1):11-21. doi:10.1017/S2045796017000452 PubMedGoogle ScholarCrossref
2.
Kilbourne  AM, Beck  K, Spaeth-Rublee  B,  et al.  Measuring and improving the quality of mental health care: a global perspective.   World Psychiatry. 2018;17(1):30-38. doi:10.1002/wps.20482 PubMedGoogle ScholarCrossref
3.
Jørgensen  K, Rendtorff  JD.  Patient participation in mental health care—perspectives of healthcare professionals: an integrative review.   Scand J Caring Sci. 2018;32(2):490-501. doi:10.1111/scs.12531 PubMedGoogle ScholarCrossref
4.
World Health Organization.  Mental Health Action Plan 2013-2020. World Health Organization; 2013.
5.
Ssebunnya  J, Kigozi  F, Ndyanabangi  S.  Developing a national mental health policy: a case study from Uganda.   PLoS Med. 2012;9(10):e1001319. doi:10.1371/journal.pmed.1001319 PubMedGoogle Scholar
6.
Marais  DL, Quayle  M, Petersen  I.  Making consultation meaningful: insights from a case study of the South African mental health policy consultation process.   PLoS One. 2020;15(1):e0228281. doi:10.1371/journal.pone.0228281 PubMedGoogle Scholar
7.
Ministry of Health Republic of Liberia.  Mental Health Policy and Strategic Plan for Liberia, 2016-2021. WHO Africa; 2016.
8.
National Institute for Health and Care Excellence (NICE). Patient and public involvement policy. Published November 2013. Accessed October 12, 2021. https://www.nice.org.uk/about/nice-communities/nice-and-the-public/public-involvement/public-involvement-programme/patient-public-involvement-policy
9.
World Psychiatric Association.  Handbook of Service User Involvement in Mental Health Research. Wiley-Blackwell; 2009.
10.
Stomski  NJ, Morrison  P.  Participation in mental healthcare: a qualitative meta-synthesis.   Int J Ment Health Syst. 2017;11(1):67. doi:10.1186/s13033-017-0174-y PubMedGoogle ScholarCrossref
11.
Semrau  M, Lempp  H, Keynejad  R,  et al.  Service user and caregiver involvement in mental health system strengthening in low- and middle-income countries: systematic review.   BMC Health Serv Res. 2016;16(1):79. doi:10.1186/s12913-016-1323-8 PubMedGoogle ScholarCrossref
12.
Heim  E, Kohrt  BA, Koschorke  M, Milenova  M, Thornicroft  G.  Reducing mental health-related stigma in primary health care settings in low- and middle-income countries: a systematic review.   Epidemiol Psychiatr Sci. 2018;29:e3. doi:10.1017/S2045796018000458 PubMedGoogle Scholar
13.
Vistorte  AOR, Ribeiro  WS, Jaen  D, Jorge  MR, Evans-Lacko  S, Mari  JJ.  Stigmatizing attitudes of primary care professionals towards people with mental disorders: a systematic review.   Int J Psychiatry Med. 2018;53(4):317-338. doi:10.1177/0091217418778620 PubMedGoogle ScholarCrossref
14.
Henderson  C, Noblett  J, Parke  H,  et al.  Mental health-related stigma in health care and mental health-care settings.   Lancet Psychiatry. 2014;1(6):467-482. doi:10.1016/S2215-0366(14)00023-6 PubMedGoogle ScholarCrossref
15.
Fekadu  A, Medhin  G, Selamu  M,  et al.  Recognition of depression by primary care clinicians in rural Ethiopia.   BMC Fam Pract. 2017;18(1):56. doi:10.1186/s12875-017-0628-y PubMedGoogle ScholarCrossref
16.
Jenkins  R, Othieno  C, Okeyo  S, Aruwa  J, Kingora  J, Jenkins  B.  Health system challenges to integration of mental health delivery in primary care in Kenya--perspectives of primary care health workers.   BMC Health Serv Res. 2013;13(1):368. doi:10.1186/1472-6963-13-368 PubMedGoogle ScholarCrossref
17.
Muga  FA, Jenkins  R.  Training, attitudes and practice of district health workers in Kenya.   Soc Psychiatry Psychiatr Epidemiol. 2008;43(6):477-482. doi:10.1007/s00127-008-0327-z PubMedGoogle ScholarCrossref
18.
Ola  B, Crabb  J, Adewuya  A, Olugbile  F, Abosede  OA.  The state of readiness of Lagos state primary health care physicians to embrace the care of depression in Nigeria.   Community Ment Health J. 2014;50(2):239-244. doi:10.1007/s10597-013-9648-9PubMedGoogle ScholarCrossref
19.
Gwaikolo  WS, Kohrt  BA, Cooper  JL.  Health system preparedness for integration of mental health services in rural Liberia.   BMC Health Serv Res. 2017;17(1):508. doi:10.1186/s12913-017-2447-1 PubMedGoogle ScholarCrossref
20.
Kohrt  BA, Harper  I.  Navigating diagnoses: understanding mind-body relations, mental health, and stigma in Nepal.   Cult Med Psychiatry. 2008;32(4):462-491. doi:10.1007/s11013-008-9110-6 PubMedGoogle ScholarCrossref
21.
Kauye  F, Jenkins  R, Rahman  A.  Training primary health care workers in mental health and its impact on diagnoses of common mental disorders in primary care of a developing country, Malawi: a cluster-randomized controlled trial.   Psychol Med. 2014;44(3):657-666. doi:10.1017/S0033291713001141 PubMedGoogle ScholarCrossref
22.
Jenkins  R, Othieno  C, Okeyo  S,  et al.  Short structured general mental health in service training programme in Kenya improves patient health and social outcomes but not detection of mental health problems - a pragmatic cluster randomised controlled trial.   Int J Ment Health Syst. 2013;7(1):25. doi:10.1186/1752-4458-7-25 PubMedGoogle ScholarCrossref
23.
Pettigrew  TF, Tropp  LR.  How does intergroup contact reduce prejudice? meta-analytic tests of three mediators.   Eur J Soc Psychol. 2008;38(6):922-934. doi:10.1002/ejsp.504 Google ScholarCrossref
24.
Pettigrew  TF, Tropp  LR, Wagner  U, Christ  O.  Recent advances in intergroup contact theory.   Int J Intercult Relat. 2011;35(3):271-280. doi:10.1016/j.ijintrel.2011.03.001 Google ScholarCrossref
25.
Jorm  AF.  Effect of contact-based interventions on stigma and discrimination: a critical examination of the evidence.   Psychiatr Serv. 2020;71(7):735-737.doi:10.1176/appi.ps.201900587PubMedGoogle ScholarCrossref
26.
Corrigan  PW, Morris  SB, Michaels  PJ, Rafacz  JD, Rüsch  N.  Challenging the public stigma of mental illness: a meta-analysis of outcome studies.   Psychiatr Serv. 2012;63(10):963-973. doi:10.1176/appi.ps.201100529 PubMedGoogle ScholarCrossref
27.
Thornicroft  G, Mehta  N, Clement  S,  et al.  Evidence for effective interventions to reduce mental-health-related stigma and discrimination.   Lancet. 2016;387(10023):1123-1132. doi:10.1016/S0140-6736(15)00298-6 PubMedGoogle ScholarCrossref
28.
Heim  E, Henderson  C, Kohrt  BA, Koschorke  M, Milenova  M, Thornicroft  G.  Reducing mental health-related stigma among medical and nursing students in low- and middle-income countries: a systematic review.   Epidemiol Psychiatr Sci. 2019;29:e28. doi:10.1017/S2045796019000167PubMedGoogle Scholar
29.
Keynejad  R, Spagnolo  J, Thornicroft  G.  WHO mental health gap action programme (mhGAP) intervention guide: updated systematic review on evidence and impact.   Evid Based Ment Health. 2021;24:124-130.PubMedGoogle ScholarCrossref
30.
World Health Organization.  mhGAP Intervention Guide for Mental, Neurological and Substance-Use Disorders in Non-Specialized Health Settings: mental health Gap Action Programme (mhGAP). WHO Press; 2010.
31.
Kohrt  BA, Turner  EL, Rai  S,  et al.  Reducing mental illness stigma in healthcare settings: proof of concept for a social contact intervention to address what matters most for primary care providers.   Soc Sci Med. 2020;250:112852. doi:10.1016/j.socscimed.2020.112852 PubMedGoogle Scholar
32.
Kohrt  BA, Jordans  MJD, Turner  EL,  et al.  Reducing stigma among healthcare providers to improve mental health services (RESHAPE): protocol for a pilot cluster randomized controlled trial of a stigma reduction intervention for training primary healthcare workers in Nepal.   Pilot Feasibility Stud. 2018;4(1):36. doi:10.1186/s40814-018-0234-3 PubMedGoogle ScholarCrossref
33.
Moher  D, Schulz  KF, Altman  DG.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.   Lancet. 2001;357(9263):1191-1194. doi:10.1016/S0140-6736(00)04337-3 PubMedGoogle ScholarCrossref
34.
Eldridge  SM, Chan  CL, Campbell  MJ,  et al; PAFS consensus group.  CONSORT 2010 statement: extension to randomised pilot and feasibility trials.   BMJ. 2016;355:i5239. doi:10.1136/bmj.i5239 PubMedGoogle Scholar
35.
Campbell  MK, Piaggio  G, Elbourne  DR, Altman  DG; CONSORT Group.  Consort 2010 statement: extension to cluster randomised trials.   BMJ. 2012;345:e5661. doi:10.1136/bmj.e5661 PubMedGoogle Scholar
36.
Lund  C, Tomlinson  M, De Silva  M,  et al.  PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries.   PLoS Med. 2012;9(12):e1001359. doi:10.1371/journal.pmed.1001359 PubMedGoogle Scholar
37.
Jordans  MJ, Luitel  NP, Pokhrel  P, Patel  V.  Development and pilot testing of a mental healthcare plan in Nepal.   Br J Psychiatry. 2016;208(suppl 56):s21-s28. doi:10.1192/bjp.bp.114.153718 PubMedGoogle ScholarCrossref
38.
Jordans  MJD, Luitel  NP, Kohrt  BA,  et al.  Community-, facility-, and individual-level outcomes of a district mental healthcare plan in a low-resource setting in Nepal: a population-based evaluation.   PLoS Med. 2019;16(2):e1002748. doi:10.1371/journal.pmed.1002748 PubMedGoogle Scholar
39.
Kohrt  BA.  Vulnerable social groups in post-conflict settings: a mixed-methods policy analysis and epidemiology study of caste and psychological morbidity in Nepal Intervention.   Intervention. 2009;7(3):239-264. doi:10.1097/WTF.0b013e3283346426 Google ScholarCrossref
40.
Kohrt  BA, Speckman  RA, Kunz  RD,  et al.  Culture in psychiatric epidemiology: using ethnography and multiple mediator models to assess the relationship of caste with depression and anxiety in Nepal.   Ann Hum Biol. 2009;36(3):261-280. doi:10.1080/03014460902839194 PubMedGoogle ScholarCrossref
41.
Bhardwaj  A, Bourey  C, Rai  S, Adhikari  RP, Worthman  CM, Kohrt  BA.  Interpersonal violence and suicidality among former child soldiers and war-exposed civilian children in Nepal.   Glob Ment Health (Camb). 2018;5:e9. doi:10.1017/gmh.2017.31 PubMedGoogle Scholar
42.
Furr  LA.  On the relationship between cultural values and preferences and affective health in Nepal.   Int J Soc Psychiatry. 2005;51(1):71-82. doi:10.1177/0020764005053283 PubMedGoogle ScholarCrossref
43.
Kohrt  BA, Carruth  L.  Syndemic effects in complex humanitarian emergencies: a framework for understanding political violence and improving multi-morbidity health outcomes.   Soc Sci Med. 2020;113378. doi:10.1016/j.socscimed.2020.113378 PubMedGoogle Scholar
44.
Rai  S, Adhikari  SB, Acharya  NR, Kaiser  BN, Kohrt  BA.  Elucidating adolescent aspirational models for the design of public mental health interventions: a mixed-method study in rural Nepal.   Child Adolesc Psychiatry Ment Health. 2017;11:65. doi:10.1186/s13034-017-0198-8 PubMedGoogle ScholarCrossref
45.
Luitel  NP, Jordans  MJD, Kohrt  BA, Rathod  SD, Komproe  IH.  Treatment gap and barriers for mental health care: a cross-sectional community survey in Nepal.   PLoS One. 2017;12(8):e0183223. doi:10.1371/journal.pone.0183223 PubMedGoogle Scholar
46.
Yang  LH, Chen  FP, Sia  KJ,  et al.  “What matters most:” a cultural mechanism moderating structural vulnerability and moral experience of mental illness stigma.   Soc Sci Med. 2014;103:84-93. doi:10.1016/j.socscimed.2013.09.009 PubMedGoogle ScholarCrossref
47.
Kohrt  BA. Social neuroscience in global mental health: case study on stigma reduction in Nepal. In: Kirmayer  LJ, Worthman  CM, Kitayama  S, Lemelson  R, Cummings  CA, eds.  Culture, Mind, and Brain: Emerging Concepts, Models, and Applications. Cambridge University Press; 2020:440-451. doi:10.1017/9781108695374.024
48.
Wang  C, Burris  MA.  Photovoice: concept, methodology, and use for participatory needs assessment.   Health Educ Behav. 1997;24(3):369-387.PubMedGoogle ScholarCrossref
49.
Catalani  C, Minkler  M.  PhotoVoice: a review of the literature in health and public health.   Health Educ Behav. 2010;37(3):424-451. doi:10.1177/1090198109342084 PubMedGoogle ScholarCrossref
50.
Angdembe  M, Kohrt  BA, Jordans  M, Rimal  D, Luitel  NP.  Situational analysis to inform development of primary care and community-based mental health services for severe mental disorders in Nepal.   Int J Ment Health Syst. 2017;11(1):69. doi:10.1186/s13033-017-0176-9 PubMedGoogle ScholarCrossref
51.
Hagaman  AK, Maharjan  U, Kohrt  BA.  Suicide surveillance and health systems in Nepal: a qualitative and social network analysis.   Int J Ment Health Syst. 2016;10:46. doi:10.1186/s13033-016-0073-7 PubMedGoogle ScholarCrossref
52.
Kaiser  BN, Varma  S, Carpenter-Song  E, Sareff  R, Rai  S, Kohrt  BA.  Eliciting recovery narratives in global mental health: benefits and potential harms in service user participation.   Psychiatr Rehabil J. 2020;43(2):111-120. doi:10.1037/prj0000384PubMedGoogle ScholarCrossref
53.
Rai  S, Gurung  D, Kaiser  BN,  et al.  A service user co-facilitated intervention to reduce mental illness stigma among primary healthcare workers: utilizing perspectives of family members and caregivers.   Fam Syst Health. 2018;36(2):198-209. doi:10.1037/fsh0000338 PubMedGoogle ScholarCrossref
54.
Bogardus  ES.  Measuring social distance.   J Appl Sociol. 1925;9:299-308. https://brocku.ca/MeadProject/Bogardus/Bogardus_1925c.htmlGoogle Scholar
55.
van Brakel  WH, Cataldo  J, Grover  S,  et al.  Out of the silos: identifying cross-cutting features of health-related stigma to advance measurement and intervention.   BMC Med. 2019;17(1):13. doi:10.1186/s12916-018-1245-x PubMedGoogle ScholarCrossref
56.
Kohrt  BA, Mutamba  BB, Luitel  NP,  et al.  How competent are non-specialists trained to integrate mental health services in primary care? global health perspectives from Uganda, Liberia, and Nepal.   Int Rev Psychiatry. 2018;30(6):182-198. doi:10.1080/09540261.2019.1566116 PubMedGoogle ScholarCrossref
57.
Pescosolido  BA, Medina  TR, Martin  JK, Long  JS.  The “backbone” of stigma: identifying the global core of public prejudice associated with mental illness.   Am J Public Health. 2013;103(5):853-860. doi:10.2105/AJPH.2012.301147 PubMedGoogle ScholarCrossref
58.
Olafsdottir  S, Pescosolido  BA.  Constructing illness: how the public in eight Western nations respond to a clinical description of “schizophrenia”.   Soc Sci Med. 2011;73(6):929-938. doi:10.1016/j.socscimed.2011.06.029 PubMedGoogle ScholarCrossref
59.
Greenwald  AG, Nosek  BA.  Health of the Implicit Association Test at age 3.   Z Exp Psychol. 2001;48(2):85-93. doi:10.1026//0949-3946.48.2.85 PubMedGoogle ScholarCrossref
60.
Tergesen  CL, Gurung  D, Dhungana  S,  et al.  Impact of service user video presentations on explicit and implicit stigma toward mental illness among medical students in Nepal: a randomized controlled trial.   Int J Environ Res Public Health. 2021;18(4):2143. doi:10.3390/ijerph18042143 PubMedGoogle ScholarCrossref
61.
Hanlon  C, Semrau  M, Alem  A,  et al.  Evaluating capacity-building for mental health system strengthening in low- and middle-income countries for service users and caregivers, service planners and researchers.   Epidemiol Psychiatr Sci. 2018;27(1):3-10. doi:10.1017/S2045796017000440 PubMedGoogle ScholarCrossref
62.
Kohrt  BA, Jordans  MJD, Rai  S,  et al.  Therapist competence in global mental health: development of the ENhancing Assessment of Common Therapeutic factors (ENACT) rating scale.   Behav Res Ther. 2015;69:11-21. doi:10.1016/j.brat.2015.03.009 PubMedGoogle ScholarCrossref
63.
Kohrt  BA, Ramaiya  MK, Rai  S, Bhardwaj  A, Jordans  MJD.  Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the Enhancing Assessment of Common Therapeutic Factors (ENACT) scale.   Glob Ment Health (Camb). 2015;2:e23. doi:10.1017/gmh.2015.21 PubMedGoogle Scholar
64.
Ghimire  DJ, Chardoul  S, Kessler  RC, Axinn  WG, Adhikari  BP.  Modifying and validating the Composite International Diagnostic Interview (CIDI) for use in Nepal.   Int J Methods Psychiatr Res. 2013;22(1):71-81. doi:10.1002/mpr.1375 PubMedGoogle ScholarCrossref
65.
Leon  AC, Davis  LL, Kraemer  HC.  The role and interpretation of pilot studies in clinical research.   J Psychiatr Res. 2011;45(5):626-629. doi:10.1016/j.jpsychires.2010.10.008 PubMedGoogle ScholarCrossref
66.
Eldridge  SM, Costelloe  CE, Kahan  BC, Lancaster  GA, Kerry  SM.  How big should the pilot study for my cluster randomised trial be?   Stat Methods Med Res. 2016;25(3):1039-1056. doi:10.1177/0962280215588242 PubMedGoogle ScholarCrossref
67.
Hayes  RJ, Moulton  LH.  Cluster Randomised Trials. Second ed. Chapman and Hall/CRC Press; 2009. doi:10.1201/9781584888178
68.
Stata Statistical Software: Release 15 [computer program]. College Station, Texas: StataCorp LLC; 2017.
69.
Stata Statistical Software: Release 16 [computer program]. College Station, Texas: StataCorp LLC; 2019.
70.
Matza  LS, Rajagopalan  KS, Thompson  CL, de Lissovoy  G.  Misdiagnosed patients with bipolar disorder: comorbidities, treatment patterns, and direct treatment costs.   J Clin Psychiatry. 2005;66(11):1432-1440. doi:10.4088/JCP.v66n1114 PubMedGoogle ScholarCrossref
71.
Lake  CR. The negative impact of the misdiagnosis of schizophrenia upon patients, their families, and their caretakers. In: Lake  CR, ed.  Schizophrenia Is a Misdiagnosis: Implications for the DSM-5 and the ICD-11. Springer US; 2012:297-351. doi:10.1007/978-1-4614-1870-2_15
72.
Forgione  FA.  Diagnostic dissent: experiences of perceived misdiagnosis and stigma in persons diagnosed with schizophrenia.   J Humanist Psychol. 2019;59(1):69-98. doi:10.1177/0022167818777151 Google ScholarCrossref
73.
Chaudhury  N, Hammer  J, Kremer  M, Muralidharan  K, Rogers  FH.  Missing in action: teacher and health worker absence in developing countries.   J Econ Perspect. 2006;20(1):91-116. doi:10.1257/089533006776526058 PubMedGoogle ScholarCrossref
74.
Zimmerman  M, Shah  S, Shakya  R,  et al.  A staff support programme for rural hospitals in Nepal.   Bull World Health Organ. 2016;94(1):65-70. doi:10.2471/BLT.15.153619 PubMedGoogle ScholarCrossref
75.
Gurung  G, Tuladhar  S.  Fostering good governance at peripheral public health facilities: an experience from Nepal.   Rural Remote Health. 2013;13(2):2042.PubMedGoogle Scholar
76.
Kohrt  BA. Reducing Stigma Among Healthcare Providers (RESHAPE) - a cluster randomized controlled trial: NCT04282915. Accessed October 4, 2021. https://clinicaltrials.gov/ct2/show/NCT04282915
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,843
    Citations 0
    Original Investigation
    Global Health
    November 3, 2021

    Collaboration With People With Lived Experience of Mental Illness to Reduce Stigma and Improve Primary Care Services: A Pilot Cluster Randomized Clinical Trial

    Author Affiliations
    • 1Department of Psychiatry and Behavioral Sciences, The George Washington University, Washington, DC
    • 2Transcultural Psychosocial Organization Nepal, Kathmandu, Nepal
    • 3Duke Global Health Institute, Duke University, Durham, North Carolina
    • 4Center for Global Mental Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
    • 5University of Amsterdam, Amsterdam, the Netherlands
    • 6Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina
    • 7Jackson School of International Studies, Department of Global Health, University of Washington, Seattle
    • 8School of Public Health, Patan Academy of Health Sciences, Lalitpur, Nepal
    • 9Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland
    • 10Juggernaut Mindset, Kathmandu, Nepal
    • 11Campbell Family Mental Health Research Institute, Centre of Addiction and Mental Health, Toronto, Ontario, Canada
    • 12Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
    • 13Lunenfeld Tanenbaum Research Institute, Toronto, Ontario, Canada
    • 14Alan J. Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
    • 15Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts
    • 16Department of Global Health and Population, Harvard T. H. Chan School of Public Health, Harvard University, Boston, Massachusetts
    • 17Sangath, Goa, India
    • 18Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, New York
    JAMA Netw Open. 2021;4(11):e2131475. doi:10.1001/jamanetworkopen.2021.31475
    Key Points

    Question  Does involvement of people with lived experience of mental illness in cofacilitation of trainings for primary care practitioners reduce stigma and improve mental health services?

    Findings  In this pilot cluster randomized clinical trial of 88 primary care practitioners, procedures were feasible and acceptable for primary care practitioners to be trained by people with mental illness. Pilot findings suggested the potential for reduced stigma and improved diagnostic accuracy in cofacilitated training compared with standard training.

    Meaning  These findings suggest that a full-scale trial should be conducted to evaluate sustained attitudinal improvements and improved quality of mental health services in primary care when mental health trainings are cofacilitated by people living with mental illness.

    Abstract

    Importance  Collaboration with people with lived experience of mental illness (PWLE), also referred to as service users, is a growing priority to reduce stigma and improve mental health care.

    Objective  To examine feasibility and acceptability of conducting an antistigma intervention in collaboration with PWLE during mental health training of primary care practitioners (PCPs).

    Design, Setting, and Participants  This pilot cluster randomized clinical trial was conducted from February 7, 2016, to August 10, 2018, with assessors, PCPs, and patients blinded to group assignment. The participants were PCPs and primary care patients diagnosed with depression, psychosis, or alcohol use disorder at primary care facilities (the cluster unit of randomization) in Nepal. Statistical analysis was performed from February 2020 to February 2021.

    Interventions  In the control group, PCPs were trained on the World Health Organization Mental Health Gap Action Programme-Intervention Guide (mhGAP-IG). In the Reducing Stigma Among Healthcare Providers (RESHAPE) group, the mhGAP-IG trainings for PCPs were cofacilitated by PWLE who presented recovery testimonials through photographic narratives.

    Main Outcomes and Measures  Prespecified feasibility and acceptability measures were adequacy of randomization, retention rates, intervention fidelity, data missingness, and safety. Outcome measures for PCPs included the Social Distance Scale (SDS), accuracy of diagnoses of mental illness in standardized role-plays using the Enhancing Assessment of Common Therapeutic factors tool (ENACT), and accuracy of diagnosis with actual patients. The primary end point was 16 months posttraining.

    Results  Among the overall sample of 88 PCPs, 75 (85.2%) were men and 67 (76.1%) were upper caste Hindus; the mean (SD) age was 36.2 (8.8) years. Nine of the PCPs (10.2%) were physicians, whereas the remaining 79 PCPs (89.8%) were health assistants or auxiliary health workers. Thirty-four facilities were randomized to RESHAPE or the control group. All eligible PCPs participated: 43 in RESHAPE and 45 in the control group, with 76.7% (n = 33) and 73.3% (n = 33) retention at end line, respectively. Due to PCP dropout, 29 facilities (85.3%) were included in end line analysis. Of 15 PWLE trained as cofacilitators, 11 (73.3%) participated throughout the 3 months of PCP trainings. Among PCPs, mean SDS changes from pretraining to 16 months were −10.6 points (95% CI, −14.5 to −6.74 points) in RESHAPE and −2.79 points (−8.29 to 2.70 points) in the control group. Role-play–based diagnoses with ENACT were 78.1% (25 of 32) accurate in RESHAPE and 66.7% (22 of 33) in the control group. Patient diagnoses were 72.5% (29 of 40) accurate by PCPs in RESHAPE compared with 34.5% (10 of 29) by PCPs in the control group. There were no serious adverse events.

    Conclusions and Relevance  This pilot cluster randomized clinical trial found that procedures were feasible and acceptable for PCPs to be trained by PWLE. These pilot results will help inform a full trial to evaluate benefits of collaboration with PWLE during training of PCPs to reduce stigma and improve diagnostic accuracy.

    Trial Registration  ClinicalTrials.gov Identifier: NCT02793271

    Introduction

    Collaboration with people with lived experience of mental illness (PWLE), also referred to as service users, is increasingly recognized as an integral strategy to improve mental health care.1-3 The commitment to collaboration has been endorsed by the World Health Organization (WHO),4 national governments in their mental health policies,5-8 and mental health professional organizations.9 However, there is a scarcity of evidence-based approaches with demonstrated safety for PWLE and effectiveness in improving care.10 This gap in evidence for collaboration with PWLE is especially pronounced in low- and middle-income countries (LMIC) where efforts are under way to rapidly expand access to mental health services in primary care.11

    Collaboration with PWLE can be particularly important to reduce stigma among primary care practitioners (PCPs), which is a barrier to effective integration of mental health care.12-14 Stigma among PCPs is one contributor to low rates of detection of mental illness,15-20 which is a common shortcoming in primary care mental health programs in LMICs.15,21,22 One avenue to reduce stigma is through social contact interventions between PWLE and stigmatizing groups, such as PCPs. In social contact interventions, stigmatized and stigmatizing groups interact through sharing personal stories, engaging in collaborative activities, and having structured and unstructured social interactions.23-25 Unfortunately, most research on social contact and mental illness stigma is limited to high-income countries, and few studies in any setting have demonstrated long-term attitudinal change (eg, only 1 study in LMICs included a 12-month follow-up12); nor have they routinely evaluated behavioral changes, such as the association between stigma reduction and improved clinical care.12,25-28 There has recently been a call for more methodological rigor in social contact intervention trails.25 Moreover, despite expanding research29 on the WHO Mental Health Gap Action Programme–Intervention Guide (mhGAP-IG)30—the global training initiative for primary care–based mental health services in low-resource settings—there is a lack of studies on structured involvement of PWLE and potential benefits of social contact during these trainings.

    Therefore, we conducted a pilot cluster randomized clinical trial (cRCT) of Reducing Stigma Among Healthcare Providers (RESHAPE) in Nepal. RESHAPE is a stigma-reduction intervention conducted in collaboration with PWLE to change attitudes of PCPs participating in mhGAP-IG training.31 This pilot cRCT was deemed necessary before proceeding to a full-scale trial to address the specific objectives of assuring that PWLE could safely participate, that PCPs would be willing to attend trainings with PWLE, and to establish parameters for recruitment, randomization, and retention. We also sought preliminary estimates of benefit among PCPs measured as reduction in stigma and improvements in clinical competency, operationalized as accuracy of mental illness diagnoses. A cluster design, with primary care facilities being the unit of clustering, was used because of the shared mental health care responsibilities among PCPs working at the same facility.

    Methods
    Design

    The study protocol is available in Supplement 1 and has been published.32 This report follows the Consolidated Standards of Reporting Trials Extension (CONSORT Extension) reporting guideline for randomized studies,33 including extensions for pilot and feasibility trials34 and for cluster trials.35 This pilot cRCT was conducted in Chitwan, Nepal, using a 1:1 allocation ratio of primary care facilities (the unit of clustering). Nepal was selected because it exemplifies low-resource conditions in LMICs, and there was an existing research infrastructure evaluating mental health care integration into primary care through the Programme for Improving Mental Health Care (PRIME).36-38 No methodological changes were made after trial commencement.

    Ethical Review of the Study

    The study was granted ethical approval by the Nepal Health Research Council, Duke University institutional review board, and George Washington University institutional review board. All participants completed a signed consent form in Nepali. Before the start of PhotoVoice training, PWLE were evaluated by psychiatrists to appraise ability to safely participate in the program. The psychiatrist was available if PWLE had symptom relapse during the weeks of the PhotoVoice trainings. A psychosocial counselor was present to support PWLE for all PhotoVoice sessions and the PCP trainings. For the diagnostic accuracy component of the study, any patients found to have an incorrect diagnosis had their medical records corrected by the study psychiatrist, and they were started on the appropriate treatment for the corrected diagnosis.

    Participants and Setting

    All primary care facilities in Chitwan district in which mental health services had not yet been integrated were eligible for inclusion as clusters. All PCPs who had prescribing privileges at the primary care facilities were eligible. Primary care facilities typically have only 1 or 2 PCPs. Therefore, although each cluster included all eligible staff, there were few PCPs per facility. Approximately 1 year after PCPs were trained and supervised, patients whom they newly diagnosed with depression, psychosis, or alcohol use disorder were evaluated by a psychiatrist for accuracy of the PCP diagnosis. Caste and ethnicity data were recorded for all PCPs and patients participating in the study. Caste and ethnicity data were documented by research assistants based on participants’ last names, which indicated caste and ethnic background. In cases of last names that could be categorized into multiple groups, research assistants asked the participant to clarify the caste and ethnic identification. Caste and ethnicity were recorded for this study because the social categorizations have been associated with stigmatization and discrimination, mental illness risk factors, and differential treatment within the health system.39-45

    Interventions

    The RESHAPE intervention is designed based on social neuroscience, social psychology, and medical anthropology theories to create a what-matters-most approach to stigma reduction.46,47 Figure 1 depicts the components of RESHAPE. Full details on the content development and proof of concept testing have been published.31 The design of the RESHAPE intervention and implementation of this trial were conducted in collaboration with PWLE.

    The RESHAPE intervention engages PWLE to participate as cofacilitators in a 40-hour mhGAP-IG training30 adapted for Nepal.37 Within the mhGAP-IG training, PWLE present recovery testimonials through photographic narratives, using a technique known as PhotoVoice.48 PhotoVoice is a commonly used participatory methodology in global health.49 PWLE are eligible for the PhotoVoice skill-building if they have been treated at primary care settings for 1 of 4 priority disorders (depression, psychosis, alcohol use disorder, and epilepsy) and are now in a state of recovery, based on evaluations conducted by a Nepali psychiatrist. Selected PWLE were trained through 12 PhotoVoice sessions over 3 months in which they were taught to develop a photographic recovery narrative. PWLE constructed narratives that were approximately 7 minutes in duration and included 3 components: life before treatment, the experience of treatment, and life after treatment. On days 2 through 8 of the 9-day mhGAP-IG training, PLWE participated by providing their narratives followed by question-and-answer sessions, totaling approximately 45 minutes of direct facilitation per day. PWLE also participated in structured and unstructured social activities with PCPs (eg, ice-breakers, energizers, and meals).

    In addition, the RESHAPE model included presentations from aspirational figures. Aspirational figures were PCPs who had previously been trained to provide mental health services and who were recognized by supervisors as enthusiastic about treating patients with mental illness. These PCPs were referred to as aspirational figures because of the hope that PCP trainees will aspire to similar commitment to caring for patients with mental health concerns. Aspirational figures presented a myth-busting session and a recovery story from the perspective of a health care clinician. PWLE and aspirational figures participated over approximately 3 months of PCP trainings.

    The key themes addressed in RESHAPE were identified through the what-matters-most framework for understanding origins of stigma.31 The 3 stigma domains were survival threats, social threats, and professional threats. Survival threats included beliefs that people with mental illness are violent, including lay ethnopsychology understandings that the brain-mind (Nepali: dimaag) controls social behavior and inhibits violence, but a damaged brain-mind in mental illness leads to loss of behavioral control.20 Social threats referred to beliefs that interacting with people with mental illness could cause mental illness in health care clinicians and result in loss of social status,50 as captured in the saying “the doctor of mad patients is also mad” (Nepali: “paagal ko daktar pani paagal ho”).20 Professional threats included beliefs that providing health care for people with mental illness is ineffective, burdensome, and ultimately would jeopardize other patient care responsibilities.20,50,51 In addition, there was intersectional stigma resulting from the dual-burden of discrimination experienced by low-caste and ethnic minorities and women, who are disenfranchised in society and also considered more likely to experience mental illnesses and alcohol use disorders.39-42 The PhotoVoice narratives of PWLE and the discussion led by aspirational figures were structured to address these 3 stigma domains of survival, social, and professional threats.31

    The control condition—training as usual (TAU)—was the Nepali adaptation of mhGAP-IG without the structured participation of PWLE. mhGAP-IG included flow-charts on clinical decision-making with basic information on diagnosis and treatment.30 The mhGAP-IG was adapted for Nepal as part of PRIME.36 Through PRIME in Nepal, a 9-day PCP training was developed including approximately 40 hours of learning with 24 hours dedicated to mhGAP modules, 12 hours on psychosocial basics, and 4 hours on logistical implementation processes.37

    The RESHAPE version of the PCP training and the TAU control were time matched at 9 days of training such that some sections in the TAU group that would be covered by a psychiatrist were covered by PWLE in the RESHAPE approach. Group supervision was held separately by groups for approximately 4 to 6 hours in a 1-day session conducted once every 3 months following the training. The training and supervision were conducted by Nepali psychiatrists and psychosocial counselors.

    Outcomes

    The prespecified outcomes for determining feasibility and acceptability and progression to a full trial were identification of qualitative themes related to recovery; 75% fidelity of PWLE and aspirational figures to the items on the RESHAPE fidelity checklist; comparable PCP baseline characteristics for the groups; retention of 50% of service users trained in PhotoVoice; retention of 66% of PCPs at the end point; fewer than 15% missing items on outcome measures; and fewer than 10% adverse events. The current analysis focused on the quantitative outcomes. Qualitative outcomes have been previously presented.31,52,53 Fidelity on the RESHAPE fidelity checklist was recorded by a research assistant who observed all of the trainings and noted what activities were done for each section of the training related to RESHAPE components. For example, (1) did PhotoVoice narratives include the 3 components of pretreatment experience of mental illness, experience of treatment, and life after starting treatment?; (2) was there a question-and-answer session after the PhotoVoice presentation in which PWLE responded to PCP trainees?; and (3) did the myth-busting section by aspirational figures include all 8 myths? For PWLE, adverse events were measured at each PhotoVoice training session and after each PCP training by a psychosocial counselor asking about adverse experiences, including both specific concerns (eg, suicidality, symptom relapse) as well as giving PWLE an opportunity to raise any other concerns.52 Family members of PLWE were also given an opportunity to discuss any adverse events.53 Adverse events among PCPs were recorded at supervision sessions, and as well as ad hoc documentation of events raised by PCPs contacting the research team or clinical supervisors.

    Quantitative outcomes at the level of individual PCPs were included to evaluate within-group trends over time. The main assessment periods for PCPs in both groups were (1) baseline, which was the first day of the training; (2) midline, which was 4 months after training; and (3) end line, which was the primary end point occurring 16 months after training.

    The primary quantitative outcome measure was PCPs’ level of stigma as measured with the Social Distance Scale (SDS).54,55 The SDS consists of 12 questions about willingness to participate in different activities with PWLE (eg, how willing would you be to spend time with, work with, or have a meal with a person with mental illness). The SDS was previously used in Nepal56 and adapted from sections of the Stigma in Global Context—Mental Health Study.57,58 A number of secondary PCP outcomes were also included. The mhGAP Attitudes Assessment examines stigmatizing beliefs and stereotypes (eg, people with mental illness are violent, contagious, or to blame for their illness). The Implicit Association Test (IAT)59 is a computer-based implicit measure of stigma adapted for use with stimuli appropriate for Nepali health care clinicians.60 The mhGAP Knowledge Assessment is a 26-item true-false and multiple-choice test.61 Clinical competency in common factors of mental health care was evaluated with the Enhancing Assessment of Common Therapeutic Factors (ENACT) tool.62 The ENACT tool, developed in Nepal,63 is used by raters observing standardized role-plays of PCP trainees. Actors presented 1 of 3 vignettes (depression, psychosis, or alcohol use disorder). At the end of the role-play, PCPs were asked to provide a provisional diagnosis.

    The evaluation of diagnostic accuracy of actual patients was done approximately 14 to 22 months after training to give PCPs at least 1 year of supervised practice to establish their diagnostic skills. After PCPs made a new diagnosis of mental illness for a primary care patient, the accuracy of diagnosis was determined by a Nepali psychiatrist administering the Nepali-validated version of the Composite International Diagnostic Interview (CIDI)64 to the patient. The psychiatrist was blinded to the PCP’s diagnosis of the patient. The diagnoses of interest were the priority disorders from the mhGAP-IG training modules, which included depression, psychosis, alcohol use disorder, and epilepsy. For the purposes of the current analyses, we excluded epilepsy because of its nature as neurological disorder, and we focused on accuracy of depression, psychosis, and alcohol use disorder diagnoses. No changes were made to the assessment tools after the trial commenced.

    Sample Size

    Following recommendations for the design of pilot studies which discourage between group hypothesis testing with small samples,65-67 we did not use inference testing (ie, measuring between-group effect sizes) as the criteria for determining the number of clusters and participants. Instead, we used all eligible primary care facilities in Chitwan district. Based on results of this pilot study, we will be able to make inferences to inform estimation for the coefficient of intracluster correlation (k) for a fully powered trial using primary care facilities as the unit of randomization with a comparable population of PCPs and patients. No interim analyses or stopping guidelines were planned.

    Randomization and Masking

    Randomization of primary care facility clusters was performed by the study statistician using a random number generator in Stata statistical software version 14 (StataCorp),68 with no restrictions or stratification. PCPs were included in the study group to which their health facility was randomized. To address demand characteristics typical in social contact interventions,25 PCPs were informed that the study was an overall mental health training evaluation, rather than specific to stigma reduction. Research assistants, psychosocial counselors who performed ENACT, and psychiatrists who performed the CIDI were blinded to the study group. No a priori unblinding specifications were established. Potential sources of contamination across groups were the movement of PCPs from a facility in one group to a facility in another group (eg, moving from a primary care clinic in the RESHAPE group to the control group).

    Statistical Analysis

    The quantitative outcomes of interest for PCPs were summarized descriptively using appropriate summary statistics (mean and standard deviation for continuous outcomes and numbers and proportions for categorical outcomes). Using the baseline measurement for PCPs (ie, prior to beginning the training), preliminary estimates of within- and between-cluster variances and intracluster correlation coefficients were estimated using 1-way random effects analysis of variance. Within-group changes over time were estimated using separate linear mixed models for each group, with a random intercept for health facility. Regarding missing data, only participants with data available at follow-up time points were included in analyses and no imputation was conducted for missing participants. No between-group comparisons were estimated owing to the pilot nature of the study.65-67 Analyses were conducted using Stata statistical software version 16 (StataCorp) from February 2020 to February 2021.69

    Results
    Participants

    Thirty-four facilities were eligible for randomization (Figure 2). For the 17 facilities allocated to mhGAP-IG training as usual, 45 PCPs were eligible. For the 17 facilities allocated to RESHAPE, 43 PCPs were eligible. PCP demographics are shown in Table 1. Among the overall sample of 88 PCPs, 75 (85.2%) were men and 67 (76.1%) were upper caste Hindus; the mean (SD) age was 36.2 (8.8) years (range, 21-56 years). Nine of the PCPs (10.2%) were physicians, whereas the remaining 79 PCPs (89.8%) were health assistants or auxiliary health workers.

    This cluster randomized clinical trial study was conducted as planned from February 7, 2016, to August 10, 2018. Training of PCPs took place from February 7, 2016, through May 18, 2016. PCP assessments took place from February 7, 2016, through July 4, 2017. Patient enrollment occurred from July 16, 2017, through December 31, 2017.

    At the primary end point for analysis at 16 months after training, 33 PCPs (73.3%) participated in the TAU control group and 33 (76.7%) in the RESHAPE group. In the TAU control group, 6 PCPs dropped out because the government reassigned them to a primary care facility outside the study area, and 6 did not attend the end line evaluation session. In the RESHAPE group, 6 PCPs were reassigned, 3 did not attend the end line session, and 1 retired. Table 1 includes the baseline demographics of PCPs who participated in the end line vs those who were lost to follow-up. The participants lost to follow-up were more likely to be younger (aged <30 years), have a medical degree (MBBS), be stationed at a primary health care center, and have fewer than 5 years of experience in health care services. Only 4 physicians with MBBS (44.4%) enrolled were retained, compared with 54 auxiliary health workers (81.8%) and 17 health assistants (70.8%). Because many health facilities had only 1 or 2 PCPs at baseline, the PCP dropouts led to a loss of 3 clusters in the control group and 2 clusters in the RESHAPE group (ie, a loss of 14.7% of the clusters). See eTable 1 in Supplement 2 for information on missingness of data.

    We also tracked reassignment of PCPs across facilities within the study. The government reassigned 2 control PCPs to RESHAPE facilities between baseline and midline (both PCPs participated in the midline assessment and 1 participated in the end line assessment). One RESHAPE PCP was reassigned to a control TAU facility between midline and end line; this facility had 3 control PCPs in the study at the time. Taking all of these transfers into account, at end line there were 4 control PCPs working alongside RESHAPE-trained PCPs, which suggests that 12% (n = 4) of all control PCPs assessed at end line potentially experienced contamination of attitudinal and/or behavioral changes.

    Regarding PWLE who cofacilitated the RESHAPE trainings, 15 PWLE were trained in PhotoVoice, of whom 11 (73.3%) went on to cofacilitate RESHAPE trainings for PCPs. Among the 4 PWLE who dropped out during the PhotoVoice training process, reasons for dropout were family refusal (n = 1), time constraints (n = 1), symptom relapse (n = 1), and concerns for additional stigma by speaking in public (n = 1).53 Of the 2 RESHAPE-based trainings conducted, 8 PWLE participated in each training (approximately 2 PWLE for each priority disorder: depression, psychosis, alcohol use disorder, and epilepsy). Both trainings were above the 75% fidelity benchmark for antistigma components conducted by the PWLE and aspirational figures.

    Outcomes

    For the primary stigma outcome, SDS, there was a mean change from baseline to end line of −10.6 points (95% CI, −14.5 to −6.74 points) for PCPs in the RESHAPE group compared with −2.8 points (95% CI, −8.29 to 2.70 points) in the control group, where decreases in scores corresponded to decreases in reported social distance, ie, lower stigma (Figure 3 and Table 2). For mhGAP Knowledge, mhGAP Attitudes, and ENACT competencies, there was within-group improvement for both RESHAPE and control. For IAT, neither the control nor RESHAPE showed within-group improvement.

    At 4 months after training, diagnostic accuracy in standardized role-plays was 78.4% (29 of 37) in the RESHAPE group and 57.5% (23 of 40) in the control group (Figure 3 and Table 2). At 16 months after training, accuracy was 78.1% (25 of 32) in RESHAPE and 66.7% (22 of 33) in the control group. Patients newly diagnosed during the period of 14 to 22 months after training were enrolled in the study for assessment of diagnostic accuracy (patient demographics are in eTable 2 in Supplement 2). For actual patient diagnoses confirmed with the psychiatrist-administered CIDI, 72.5% (29 of 40) of patients in the RESHAPE group were correctly diagnosed and 34.5% (10 of 29) in the control group. The incorrectly diagnosed patients were false positives (ie, the PCP diagnosed them with a particular disorder and the psychiatrist did not confirm the diagnosis). Notably, 13 of 14 patients diagnosed with psychosis by the control PCPs were false positives, and 8 of 14 patients diagnosed with psychosis by RESHAPE-trained PCPs were false positives. Among the disorders, the disorder with the largest absolute difference between groups in diagnostic accuracy was depression in both standardized role-plays and actual patient evaluations. No serious adverse events were reported for PWLE, PCPs, or patients in either group.

    Discussion

    This pilot cRCT of a stigma reduction intervention for PCPs was conducted in collaboration with PLWE. The goal was to determine the feasibility and acceptability of study procedures in preparation for a full trial. All a priori benchmarks for progression to a full trial were met, including retention rates of participants, limited missingness of data, high intervention fidelity, and a lack of serious adverse events. These quantitative findings for feasibility and acceptability support our previously published qualitative findings for RESHAPE.31,52,53 The preliminary findings suggest that RESHAPE may have the potential to reduce stigma among PCPs without introducing substantial risk of harm to PWLE collaborating in trainings. Regarding generalizability to the broader field of stigma interventions, the potential trend of greater stigma reduction in the RESHAPE group is consistent with other findings for social contact.14,26,27

    Exposure to structured recovery testimonials from PWLE may help to increase accuracy of diagnosis by PCPs. This is important because the study revealed high rates of incorrectly diagnosing patients with psychosis (ie, false positives). Misdiagnosis, especially of psychotic disorders, increases exposure to medications with adverse effects, and misdiagnosis is costly and stigmatizing for patients and families.70-72 Although some misdiagnoses may be mitigated by improving attitudes of PCPs, it also draws attention to the need for greater supervision of diagnostic practices after mhGAP-IG training.

    Strengths and Limitations

    This study had some strengths and limitations. The pilot design addressed a number of the limitations raised about social contact intervention research25: use of a control group, trial registration, reducing demand characteristics by including a range of outcomes beyond stigma, and evaluating behavior change in the form of clinical skills. Examining long-term outcomes, specifically a 16-month follow-up, was also considerably longer than the 6-month follow-up of most antistigma interventions.14,27 The pilot findings suggest that RESHAPE may be beneficial across outcomes, except IAT, a few months after training, but the longer-term benefit compared with standard training (ie, at 16 months) may be limited to fewer domains (social distance and diagnostic accuracy).

    The long duration of our pilot was important to estimate actual retention rates of participants and clusters. By having a long duration and a large number of clusters, we found that 14.7% of the clusters did not have enough PCPs working on site for participation in the end point. If we had fewer clusters or a shorter duration in the pilot, we may not have been able to establish a reliable cluster dropout rate. We also found that PCPs who were physicians, younger (aged <30 years), and had fewer years of experience in health care (<5 years) were the most likely to drop out. This is likely due to government programs that place young physicians in rural areas after they complete training for brief assignments of only 1 to 2 years. In addition, physicians assigned to rural areas have been criticized for absenteeism in which they are working at the government health facilities.73-75 More than half of the physicians enrolled in the study dropped out, compared with 82% of auxiliary health workers and 71% of health assistants who were actively engaged in health services to their communities. Our pilot also revealed PCPs being transferred across study group health facilities (eg, RESHAPE to control and vice versa). Regarding contamination, 12% of control PCPs were working alongside RESHAPE-trained PCPs at the end line assessment, which may have influenced their attitudes and clinical practices. In a full trial, alternative strategies are needed for defining and retaining clusters, as well as preventing contamination.

    A study limitation was that mental health specialists conducting the trainings could not be blinded to the participation of PWLE in the trainings. The presence of PWLE may have impacted the psychiatrist trainer’s behavior in some manner. Because of this, a full trial should record the psychiatrist trainer’s fidelity to mhGAP-IG components to see if this differs between groups. Based on the lessons learned regarding strengths and limitations of the pilot trial, a full cRCT of RESHAPE is now under way in Nepal.76

    Another limitation of this study was that it focused on in-service health training of certified PCPs, which is the equivalent of continuing medical education courses. However, strategies are also needed to reduce stigma and improve mental health diagnostic skills of during preservice training (eg, in medical schools and auxiliary health worker vocational training programs). Evidence-based preservice stigma reduction programs are also lacking in LMICs.28 To effectively reduce the mental health care treatment gap in Nepal, preservice and in-service stigma reduction and mental health training are especially important for auxiliary health workers and health assistants who provide the majority of care in primary care settings.

    Conclusions

    This pilot cRCT met its prespecified feasibility and acceptability measures, and a larger cRCT is ongoing. Ultimately, the potential to collaborate with PWLE to reduce stigma and improve diagnosis is encouraging for enhancing the success of mhGAP-IG implementation and more broadly for successful integration of mental health services into primary care settings around the world.

    Back to top
    Article Information

    Accepted for Publication: August 23, 2021.

    Published: November 3, 2021. doi:10.1001/jamanetworkopen.2021.31475

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Kohrt BA et al. JAMA Network Open.

    Corresponding Author: Brandon A. Kohrt, MD, PhD, Department of Psychiatry and Behavioral Sciences, The George Washington University, 2120 L St NW, Ste 600, Washington, DC 20037 (bkohrt@gwu.edu).

    Author Contributions: Drs Kohrt and Turner had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Kohrt, Jordans, Turner, Rai, Gurung, Lamichhane, Singla, Lund, Patel, Luitel, Sikkema.

    Acquisition, analysis, or interpretation of data: Kohrt, Turner, Rai, Dhakal, Bhardwaj, Luitel, Sikkema.

    Drafting of the manuscript: Kohrt, Turner, Rai, Gurung.

    Critical revision of the manuscript for important intellectual content: Jordans, Turner, Rai, Dhakal, Bhardwaj, Lamichhane, Singla, Lund, Patel, Luitel, Sikkema.

    Statistical analysis: Kohrt, Turner.

    Obtained funding: Kohrt, Turner, Sikkema.

    Administrative, technical, or material support: Kohrt, Rai, Gurung, Dhakal, Lund, Luitel.

    Supervision: Jordans, Rai, Dhakal, Lamichhane, Patel, Luitel, Sikkema.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: This study was funded by the US National Institute of Mental Health (K01MH104310; PI: Dr Kohrt) and the United Kingdom Department for International Development funding to the Programme for Improving Mental Healthcare Consortium.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: All information and materials in the manuscript are original. The figures are original and produced by the first author. They have not been published elsewhere.

    Data Sharing Statement: See Supplement 3.

    Additional Contributions: We thank Adesewa Adelekun (UCLA, PhotoVoice training, implicit association test data collection and coordination), Anup Adhikari (Transcultural Psychosocial Organization [TPO]-Nepal, supervision), Kamal Gautam (TPO-Nepal, supervision), Ashley Hagaman (Yale University, qualitative study), Bonnie Kaiser (University of California San Diego, qualitative study), Suraj Koirala (TPO-Nepal, supervision), Parbati Pandey (TPO-Nepal, clinical team), Megan Ramaiya (University of Washington, clinical competence study), Jaya Regmi (TPO-Nepal, clinical team), Alisha Rijal (TPO-Nepal, data collection and coordination), Sunil Kumar Shah (Bharatpur Hospital Chitwan, clinical team), Pragya Shrestha (TPO-Nepal, clinical team), Jitendra Rai (TPO-Nepal, clinical team), and Cori Tergesen (DePaul University, implicit association test coordination and qualitative data analysis) for their contributions to the RESHAPE project. We acknowledge the Nepal Health Research Council (ethical approval).

    References
    1.
    Semrau  M, Alem  A, Abdulmalik  J,  et al.  Developing capacity-building activities for mental health system strengthening in low- and middle-income countries for service users and caregivers, service planners, and researchers.   Epidemiol Psychiatr Sci. 2018;27(1):11-21. doi:10.1017/S2045796017000452 PubMedGoogle ScholarCrossref
    2.
    Kilbourne  AM, Beck  K, Spaeth-Rublee  B,  et al.  Measuring and improving the quality of mental health care: a global perspective.   World Psychiatry. 2018;17(1):30-38. doi:10.1002/wps.20482 PubMedGoogle ScholarCrossref
    3.
    Jørgensen  K, Rendtorff  JD.  Patient participation in mental health care—perspectives of healthcare professionals: an integrative review.   Scand J Caring Sci. 2018;32(2):490-501. doi:10.1111/scs.12531 PubMedGoogle ScholarCrossref
    4.
    World Health Organization.  Mental Health Action Plan 2013-2020. World Health Organization; 2013.
    5.
    Ssebunnya  J, Kigozi  F, Ndyanabangi  S.  Developing a national mental health policy: a case study from Uganda.   PLoS Med. 2012;9(10):e1001319. doi:10.1371/journal.pmed.1001319 PubMedGoogle Scholar
    6.
    Marais  DL, Quayle  M, Petersen  I.  Making consultation meaningful: insights from a case study of the South African mental health policy consultation process.   PLoS One. 2020;15(1):e0228281. doi:10.1371/journal.pone.0228281 PubMedGoogle Scholar
    7.
    Ministry of Health Republic of Liberia.  Mental Health Policy and Strategic Plan for Liberia, 2016-2021. WHO Africa; 2016.
    8.
    National Institute for Health and Care Excellence (NICE). Patient and public involvement policy. Published November 2013. Accessed October 12, 2021. https://www.nice.org.uk/about/nice-communities/nice-and-the-public/public-involvement/public-involvement-programme/patient-public-involvement-policy
    9.
    World Psychiatric Association.  Handbook of Service User Involvement in Mental Health Research. Wiley-Blackwell; 2009.
    10.
    Stomski  NJ, Morrison  P.  Participation in mental healthcare: a qualitative meta-synthesis.   Int J Ment Health Syst. 2017;11(1):67. doi:10.1186/s13033-017-0174-y PubMedGoogle ScholarCrossref
    11.
    Semrau  M, Lempp  H, Keynejad  R,  et al.  Service user and caregiver involvement in mental health system strengthening in low- and middle-income countries: systematic review.   BMC Health Serv Res. 2016;16(1):79. doi:10.1186/s12913-016-1323-8 PubMedGoogle ScholarCrossref
    12.
    Heim  E, Kohrt  BA, Koschorke  M, Milenova  M, Thornicroft  G.  Reducing mental health-related stigma in primary health care settings in low- and middle-income countries: a systematic review.   Epidemiol Psychiatr Sci. 2018;29:e3. doi:10.1017/S2045796018000458 PubMedGoogle Scholar
    13.
    Vistorte  AOR, Ribeiro  WS, Jaen  D, Jorge  MR, Evans-Lacko  S, Mari  JJ.  Stigmatizing attitudes of primary care professionals towards people with mental disorders: a systematic review.   Int J Psychiatry Med. 2018;53(4):317-338. doi:10.1177/0091217418778620 PubMedGoogle ScholarCrossref
    14.
    Henderson  C, Noblett  J, Parke  H,  et al.  Mental health-related stigma in health care and mental health-care settings.   Lancet Psychiatry. 2014;1(6):467-482. doi:10.1016/S2215-0366(14)00023-6 PubMedGoogle ScholarCrossref
    15.
    Fekadu  A, Medhin  G, Selamu  M,  et al.  Recognition of depression by primary care clinicians in rural Ethiopia.   BMC Fam Pract. 2017;18(1):56. doi:10.1186/s12875-017-0628-y PubMedGoogle ScholarCrossref
    16.
    Jenkins  R, Othieno  C, Okeyo  S, Aruwa  J, Kingora  J, Jenkins  B.  Health system challenges to integration of mental health delivery in primary care in Kenya--perspectives of primary care health workers.   BMC Health Serv Res. 2013;13(1):368. doi:10.1186/1472-6963-13-368 PubMedGoogle ScholarCrossref
    17.
    Muga  FA, Jenkins  R.  Training, attitudes and practice of district health workers in Kenya.   Soc Psychiatry Psychiatr Epidemiol. 2008;43(6):477-482. doi:10.1007/s00127-008-0327-z PubMedGoogle ScholarCrossref
    18.
    Ola  B, Crabb  J, Adewuya  A, Olugbile  F, Abosede  OA.  The state of readiness of Lagos state primary health care physicians to embrace the care of depression in Nigeria.   Community Ment Health J. 2014;50(2):239-244. doi:10.1007/s10597-013-9648-9PubMedGoogle ScholarCrossref
    19.
    Gwaikolo  WS, Kohrt  BA, Cooper  JL.  Health system preparedness for integration of mental health services in rural Liberia.   BMC Health Serv Res. 2017;17(1):508. doi:10.1186/s12913-017-2447-1 PubMedGoogle ScholarCrossref
    20.
    Kohrt  BA, Harper  I.  Navigating diagnoses: understanding mind-body relations, mental health, and stigma in Nepal.   Cult Med Psychiatry. 2008;32(4):462-491. doi:10.1007/s11013-008-9110-6 PubMedGoogle ScholarCrossref
    21.
    Kauye  F, Jenkins  R, Rahman  A.  Training primary health care workers in mental health and its impact on diagnoses of common mental disorders in primary care of a developing country, Malawi: a cluster-randomized controlled trial.   Psychol Med. 2014;44(3):657-666. doi:10.1017/S0033291713001141 PubMedGoogle ScholarCrossref
    22.
    Jenkins  R, Othieno  C, Okeyo  S,  et al.  Short structured general mental health in service training programme in Kenya improves patient health and social outcomes but not detection of mental health problems - a pragmatic cluster randomised controlled trial.   Int J Ment Health Syst. 2013;7(1):25. doi:10.1186/1752-4458-7-25 PubMedGoogle ScholarCrossref
    23.
    Pettigrew  TF, Tropp  LR.  How does intergroup contact reduce prejudice? meta-analytic tests of three mediators.   Eur J Soc Psychol. 2008;38(6):922-934. doi:10.1002/ejsp.504 Google ScholarCrossref
    24.
    Pettigrew  TF, Tropp  LR, Wagner  U, Christ  O.  Recent advances in intergroup contact theory.   Int J Intercult Relat. 2011;35(3):271-280. doi:10.1016/j.ijintrel.2011.03.001 Google ScholarCrossref
    25.
    Jorm  AF.  Effect of contact-based interventions on stigma and discrimination: a critical examination of the evidence.   Psychiatr Serv. 2020;71(7):735-737.doi:10.1176/appi.ps.201900587PubMedGoogle ScholarCrossref
    26.
    Corrigan  PW, Morris  SB, Michaels  PJ, Rafacz  JD, Rüsch  N.  Challenging the public stigma of mental illness: a meta-analysis of outcome studies.   Psychiatr Serv. 2012;63(10):963-973. doi:10.1176/appi.ps.201100529 PubMedGoogle ScholarCrossref
    27.
    Thornicroft  G, Mehta  N, Clement  S,  et al.  Evidence for effective interventions to reduce mental-health-related stigma and discrimination.   Lancet. 2016;387(10023):1123-1132. doi:10.1016/S0140-6736(15)00298-6 PubMedGoogle ScholarCrossref
    28.
    Heim  E, Henderson  C, Kohrt  BA, Koschorke  M, Milenova  M, Thornicroft  G.  Reducing mental health-related stigma among medical and nursing students in low- and middle-income countries: a systematic review.   Epidemiol Psychiatr Sci. 2019;29:e28. doi:10.1017/S2045796019000167PubMedGoogle Scholar
    29.
    Keynejad  R, Spagnolo  J, Thornicroft  G.  WHO mental health gap action programme (mhGAP) intervention guide: updated systematic review on evidence and impact.   Evid Based Ment Health. 2021;24:124-130.PubMedGoogle ScholarCrossref
    30.
    World Health Organization.  mhGAP Intervention Guide for Mental, Neurological and Substance-Use Disorders in Non-Specialized Health Settings: mental health Gap Action Programme (mhGAP). WHO Press; 2010.
    31.
    Kohrt  BA, Turner  EL, Rai  S,  et al.  Reducing mental illness stigma in healthcare settings: proof of concept for a social contact intervention to address what matters most for primary care providers.   Soc Sci Med. 2020;250:112852. doi:10.1016/j.socscimed.2020.112852 PubMedGoogle Scholar
    32.
    Kohrt  BA, Jordans  MJD, Turner  EL,  et al.  Reducing stigma among healthcare providers to improve mental health services (RESHAPE): protocol for a pilot cluster randomized controlled trial of a stigma reduction intervention for training primary healthcare workers in Nepal.   Pilot Feasibility Stud. 2018;4(1):36. doi:10.1186/s40814-018-0234-3 PubMedGoogle ScholarCrossref
    33.
    Moher  D, Schulz  KF, Altman  DG.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.   Lancet. 2001;357(9263):1191-1194. doi:10.1016/S0140-6736(00)04337-3 PubMedGoogle ScholarCrossref
    34.
    Eldridge  SM, Chan  CL, Campbell  MJ,  et al; PAFS consensus group.  CONSORT 2010 statement: extension to randomised pilot and feasibility trials.   BMJ. 2016;355:i5239. doi:10.1136/bmj.i5239 PubMedGoogle Scholar
    35.
    Campbell  MK, Piaggio  G, Elbourne  DR, Altman  DG; CONSORT Group.  Consort 2010 statement: extension to cluster randomised trials.   BMJ. 2012;345:e5661. doi:10.1136/bmj.e5661 PubMedGoogle Scholar
    36.
    Lund  C, Tomlinson  M, De Silva  M,  et al.  PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries.   PLoS Med. 2012;9(12):e1001359. doi:10.1371/journal.pmed.1001359 PubMedGoogle Scholar
    37.
    Jordans  MJ, Luitel  NP, Pokhrel  P, Patel  V.  Development and pilot testing of a mental healthcare plan in Nepal.   Br J Psychiatry. 2016;208(suppl 56):s21-s28. doi:10.1192/bjp.bp.114.153718 PubMedGoogle ScholarCrossref
    38.
    Jordans  MJD, Luitel  NP, Kohrt  BA,  et al.  Community-, facility-, and individual-level outcomes of a district mental healthcare plan in a low-resource setting in Nepal: a population-based evaluation.   PLoS Med. 2019;16(2):e1002748. doi:10.1371/journal.pmed.1002748 PubMedGoogle Scholar
    39.
    Kohrt  BA.  Vulnerable social groups in post-conflict settings: a mixed-methods policy analysis and epidemiology study of caste and psychological morbidity in Nepal Intervention.   Intervention. 2009;7(3):239-264. doi:10.1097/WTF.0b013e3283346426 Google ScholarCrossref
    40.
    Kohrt  BA, Speckman  RA, Kunz  RD,  et al.  Culture in psychiatric epidemiology: using ethnography and multiple mediator models to assess the relationship of caste with depression and anxiety in Nepal.   Ann Hum Biol. 2009;36(3):261-280. doi:10.1080/03014460902839194 PubMedGoogle ScholarCrossref
    41.
    Bhardwaj  A, Bourey  C, Rai  S, Adhikari  RP, Worthman  CM, Kohrt  BA.  Interpersonal violence and suicidality among former child soldiers and war-exposed civilian children in Nepal.   Glob Ment Health (Camb). 2018;5:e9. doi:10.1017/gmh.2017.31 PubMedGoogle Scholar
    42.
    Furr  LA.  On the relationship between cultural values and preferences and affective health in Nepal.   Int J Soc Psychiatry. 2005;51(1):71-82. doi:10.1177/0020764005053283 PubMedGoogle ScholarCrossref
    43.
    Kohrt  BA, Carruth  L.  Syndemic effects in complex humanitarian emergencies: a framework for understanding political violence and improving multi-morbidity health outcomes.   Soc Sci Med. 2020;113378. doi:10.1016/j.socscimed.2020.113378 PubMedGoogle Scholar
    44.
    Rai  S, Adhikari  SB, Acharya  NR, Kaiser  BN, Kohrt  BA.  Elucidating adolescent aspirational models for the design of public mental health interventions: a mixed-method study in rural Nepal.   Child Adolesc Psychiatry Ment Health. 2017;11:65. doi:10.1186/s13034-017-0198-8 PubMedGoogle ScholarCrossref
    45.
    Luitel  NP, Jordans  MJD, Kohrt  BA, Rathod  SD, Komproe  IH.  Treatment gap and barriers for mental health care: a cross-sectional community survey in Nepal.   PLoS One. 2017;12(8):e0183223. doi:10.1371/journal.pone.0183223 PubMedGoogle Scholar
    46.
    Yang  LH, Chen  FP, Sia  KJ,  et al.  “What matters most:” a cultural mechanism moderating structural vulnerability and moral experience of mental illness stigma.   Soc Sci Med. 2014;103:84-93. doi:10.1016/j.socscimed.2013.09.009 PubMedGoogle ScholarCrossref
    47.
    Kohrt  BA. Social neuroscience in global mental health: case study on stigma reduction in Nepal. In: Kirmayer  LJ, Worthman  CM, Kitayama  S, Lemelson  R, Cummings  CA, eds.  Culture, Mind, and Brain: Emerging Concepts, Models, and Applications. Cambridge University Press; 2020:440-451. doi:10.1017/9781108695374.024
    48.
    Wang  C, Burris  MA.  Photovoice: concept, methodology, and use for participatory needs assessment.   Health Educ Behav. 1997;24(3):369-387.PubMedGoogle ScholarCrossref
    49.
    Catalani  C, Minkler  M.  PhotoVoice: a review of the literature in health and public health.   Health Educ Behav. 2010;37(3):424-451. doi:10.1177/1090198109342084 PubMedGoogle ScholarCrossref
    50.
    Angdembe  M, Kohrt  BA, Jordans  M, Rimal  D, Luitel  NP.  Situational analysis to inform development of primary care and community-based mental health services for severe mental disorders in Nepal.   Int J Ment Health Syst. 2017;11(1):69. doi:10.1186/s13033-017-0176-9 PubMedGoogle ScholarCrossref
    51.
    Hagaman  AK, Maharjan  U, Kohrt  BA.  Suicide surveillance and health systems in Nepal: a qualitative and social network analysis.   Int J Ment Health Syst. 2016;10:46. doi:10.1186/s13033-016-0073-7 PubMedGoogle ScholarCrossref
    52.
    Kaiser  BN, Varma  S, Carpenter-Song  E, Sareff  R, Rai  S, Kohrt  BA.  Eliciting recovery narratives in global mental health: benefits and potential harms in service user participation.   Psychiatr Rehabil J. 2020;43(2):111-120. doi:10.1037/prj0000384PubMedGoogle ScholarCrossref
    53.
    Rai  S, Gurung  D, Kaiser  BN,  et al.  A service user co-facilitated intervention to reduce mental illness stigma among primary healthcare workers: utilizing perspectives of family members and caregivers.   Fam Syst Health. 2018;36(2):198-209. doi:10.1037/fsh0000338 PubMedGoogle ScholarCrossref
    54.
    Bogardus  ES.  Measuring social distance.   J Appl Sociol. 1925;9:299-308. https://brocku.ca/MeadProject/Bogardus/Bogardus_1925c.htmlGoogle Scholar
    55.
    van Brakel  WH, Cataldo  J, Grover  S,  et al.  Out of the silos: identifying cross-cutting features of health-related stigma to advance measurement and intervention.   BMC Med. 2019;17(1):13. doi:10.1186/s12916-018-1245-x PubMedGoogle ScholarCrossref
    56.
    Kohrt  BA, Mutamba  BB, Luitel  NP,  et al.  How competent are non-specialists trained to integrate mental health services in primary care? global health perspectives from Uganda, Liberia, and Nepal.   Int Rev Psychiatry. 2018;30(6):182-198. doi:10.1080/09540261.2019.1566116 PubMedGoogle ScholarCrossref
    57.
    Pescosolido  BA, Medina  TR, Martin  JK, Long  JS.  The “backbone” of stigma: identifying the global core of public prejudice associated with mental illness.   Am J Public Health. 2013;103(5):853-860. doi:10.2105/AJPH.2012.301147 PubMedGoogle ScholarCrossref
    58.
    Olafsdottir  S, Pescosolido  BA.  Constructing illness: how the public in eight Western nations respond to a clinical description of “schizophrenia”.   Soc Sci Med. 2011;73(6):929-938. doi:10.1016/j.socscimed.2011.06.029 PubMedGoogle ScholarCrossref
    59.
    Greenwald  AG, Nosek  BA.  Health of the Implicit Association Test at age 3.   Z Exp Psychol. 2001;48(2):85-93. doi:10.1026//0949-3946.48.2.85 PubMedGoogle ScholarCrossref
    60.
    Tergesen  CL, Gurung  D, Dhungana  S,  et al.  Impact of service user video presentations on explicit and implicit stigma toward mental illness among medical students in Nepal: a randomized controlled trial.   Int J Environ Res Public Health. 2021;18(4):2143. doi:10.3390/ijerph18042143 PubMedGoogle ScholarCrossref
    61.
    Hanlon  C, Semrau  M, Alem  A,  et al.  Evaluating capacity-building for mental health system strengthening in low- and middle-income countries for service users and caregivers, service planners and researchers.   Epidemiol Psychiatr Sci. 2018;27(1):3-10. doi:10.1017/S2045796017000440 PubMedGoogle ScholarCrossref
    62.
    Kohrt  BA, Jordans  MJD, Rai  S,  et al.  Therapist competence in global mental health: development of the ENhancing Assessment of Common Therapeutic factors (ENACT) rating scale.   Behav Res Ther. 2015;69:11-21. doi:10.1016/j.brat.2015.03.009 PubMedGoogle ScholarCrossref
    63.
    Kohrt  BA, Ramaiya  MK, Rai  S, Bhardwaj  A, Jordans  MJD.  Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the Enhancing Assessment of Common Therapeutic Factors (ENACT) scale.   Glob Ment Health (Camb). 2015;2:e23. doi:10.1017/gmh.2015.21 PubMedGoogle Scholar
    64.
    Ghimire  DJ, Chardoul  S, Kessler  RC, Axinn  WG, Adhikari  BP.  Modifying and validating the Composite International Diagnostic Interview (CIDI) for use in Nepal.   Int J Methods Psychiatr Res. 2013;22(1):71-81. doi:10.1002/mpr.1375 PubMedGoogle ScholarCrossref
    65.
    Leon  AC, Davis  LL, Kraemer  HC.  The role and interpretation of pilot studies in clinical research.   J Psychiatr Res. 2011;45(5):626-629. doi:10.1016/j.jpsychires.2010.10.008 PubMedGoogle ScholarCrossref
    66.
    Eldridge  SM, Costelloe  CE, Kahan  BC, Lancaster  GA, Kerry  SM.  How big should the pilot study for my cluster randomised trial be?   Stat Methods Med Res. 2016;25(3):1039-1056. doi:10.1177/0962280215588242 PubMedGoogle ScholarCrossref
    67.
    Hayes  RJ, Moulton  LH.  Cluster Randomised Trials. Second ed. Chapman and Hall/CRC Press; 2009. doi:10.1201/9781584888178
    68.
    Stata Statistical Software: Release 15 [computer program]. College Station, Texas: StataCorp LLC; 2017.
    69.
    Stata Statistical Software: Release 16 [computer program]. College Station, Texas: StataCorp LLC; 2019.
    70.
    Matza  LS, Rajagopalan  KS, Thompson  CL, de Lissovoy  G.  Misdiagnosed patients with bipolar disorder: comorbidities, treatment patterns, and direct treatment costs.   J Clin Psychiatry. 2005;66(11):1432-1440. doi:10.4088/JCP.v66n1114 PubMedGoogle ScholarCrossref
    71.
    Lake  CR. The negative impact of the misdiagnosis of schizophrenia upon patients, their families, and their caretakers. In: Lake  CR, ed.  Schizophrenia Is a Misdiagnosis: Implications for the DSM-5 and the ICD-11. Springer US; 2012:297-351. doi:10.1007/978-1-4614-1870-2_15
    72.
    Forgione  FA.  Diagnostic dissent: experiences of perceived misdiagnosis and stigma in persons diagnosed with schizophrenia.   J Humanist Psychol. 2019;59(1):69-98. doi:10.1177/0022167818777151 Google ScholarCrossref
    73.
    Chaudhury  N, Hammer  J, Kremer  M, Muralidharan  K, Rogers  FH.  Missing in action: teacher and health worker absence in developing countries.   J Econ Perspect. 2006;20(1):91-116. doi:10.1257/089533006776526058 PubMedGoogle ScholarCrossref
    74.
    Zimmerman  M, Shah  S, Shakya  R,  et al.  A staff support programme for rural hospitals in Nepal.   Bull World Health Organ. 2016;94(1):65-70. doi:10.2471/BLT.15.153619 PubMedGoogle ScholarCrossref
    75.
    Gurung  G, Tuladhar  S.  Fostering good governance at peripheral public health facilities: an experience from Nepal.   Rural Remote Health. 2013;13(2):2042.PubMedGoogle Scholar
    76.
    Kohrt  BA. Reducing Stigma Among Healthcare Providers (RESHAPE) - a cluster randomized controlled trial: NCT04282915. Accessed October 4, 2021. https://clinicaltrials.gov/ct2/show/NCT04282915
    ×