[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
Figure.
Review and Selection of Articles on the Association Between Learning Environment Interventions and Medical Student Well-being
Review and Selection of Articles on the Association Between Learning Environment Interventions and Medical Student Well-being
Table 1.  
Study Comparison and Outcome Measures
Study Comparison and Outcome Measures
Table 2.  
Methods of Included Studies
Methods of Included Studies
1.
Dezee  KJ, Artino  AR, Elnicki  DM, Hemmer  PA, Durning  SJ.  Medical education in the United States of America.  Med Teach. 2012;34(7):521-525.PubMedGoogle ScholarCrossref
2.
Brazeau  CM, Shanafelt  T, Durning  SJ,  et al.  Distress among matriculating medical students relative to the general population.  Acad Med. 2014;89(11):1520-1525.PubMedGoogle ScholarCrossref
3.
Dyrbye  LN, Massie  FS  Jr, Eacker  A,  et al.  Relationship between burnout and professional conduct and attitudes among US medical students.  JAMA. 2010;304(11):1173-1180.PubMedGoogle ScholarCrossref
4.
Schwenk  TL, Davis  L, Wimsatt  LA.  Depression, stigma, and suicidal ideation in medical students.  JAMA. 2010;304(11):1181-1190.PubMedGoogle ScholarCrossref
5.
Dyrbye  LN, Thomas  MR, Massie  FS,  et al.  Burnout and suicidal ideation among US medical students.  Ann Intern Med. 2008;149(5):334-341.PubMedGoogle ScholarCrossref
6.
Association of American Medical Colleges.  Educating Doctors to Provide High Quality Medical Care: A Vision for Medical Education in the United States. Washington, DC: Association of American Medical Colleges; 2004.
7.
Gelfand  J. Grey literature: plausible applications for distance education practices. In: GL’99 Conference Proceedings: Fourth International Conference on Grey Literature: “New Frontiers in Grey Literature.” Amsterdam, the Netherlands: GreyNet; January 2000:1-11. GL Conference Series No. 4.
8.
Stacy  R, Spencer  J.  Assessing the evidence in qualitative medical education research.  Med Educ. 2000;34(7):498-500.PubMedGoogle ScholarCrossref
9.
Côté  L, Turgeon  J.  Appraising qualitative research articles in medicine and medical education.  Med Teach. 2005;27(1):71-75.PubMedGoogle ScholarCrossref
10.
Reed  DA, Cook  DA, Beckman  TJ, Levine  RB, Kern  DE, Wright  SM.  Association between funding and quality of published medical education research.  JAMA. 2007;298(9):1002-1009.PubMedGoogle ScholarCrossref
11.
Cook  DA, Reed  DA.  Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale–Education.  Acad Med. 2015;90(8):1067-1076.PubMedGoogle ScholarCrossref
12.
Lin  H, Lin  E, Auditore  S, Fanning  J.  A narrative review of high-quality literature on the effects of resident duty hours reforms.  Acad Med. 2016;91(1):140-150.PubMedGoogle ScholarCrossref
13.
Bloodgood  RA, Short  JG, Jackson  JM, Martindale  JR.  A change to pass/fail grading in the first two years at one medical school results in improved psychological well-being.  Acad Med. 2009;84(5):655-662.PubMedGoogle ScholarCrossref
14.
DuPuy  HJ. Self-representations of general psychological well-being of American adults. Presented at: American Public Health Association Meeting; Los Angeles, CA; October 15-19, 1978.
15.
Rohe  DE, Barrier  PA, Clark  MM, Cook  DA, Vickers  KS, Decker  PA.  The benefits of pass-fail grading on stress, mood, and group cohesion in medical students.  Mayo Clin Proc. 2006;81(11):1443-1448.PubMedGoogle ScholarCrossref
16.
Cohen  S, Kamarck  T, Mermelstein  R.  A global measure of perceived stress.  J Health Soc Behav. 1983;24(4):385-396.PubMedGoogle ScholarCrossref
17.
Cohen  S. Perceived stress in a probability sample of the United States. In: Oskamp  SSS, ed.  The Social Psychology of Health. Thousand Oaks, CA: Sage Publications; 1988:31-67.
18.
McNair  DM, Lorr  M, Droppleman  LF.  POMS Manual Profile of Mood States. San Diego, CA: EdiTS/Educational and Industrial Testing Service; 1992.
19.
Bollen  KA, Hoyle  RH.  Perceived cohesion: a conceptual and empirical examination.  Soc Forces. 1990;69(2):479-504.Google ScholarCrossref
20.
Reed  DA, Shanafelt  TD, Satele  DW,  et al.  Relationship of pass/fail grading and curriculum structure with well-being among preclinical medical students: a multi-institutional study.  Acad Med. 2011;86(11):1367-1373.PubMedGoogle ScholarCrossref
21.
Maslach  C, Jackson  SE.  The measurement of experienced burnout.  J Organ Behav. 1981;2(2):99-113.Google ScholarCrossref
22.
Stewart  AL, Ware  JE, eds.  Measuring Functioning and Well-being: the Medical Outcome Study Approach. Durham, NC: Duke University Press; 1992.
23.
Tarlov  AR, Ware  JE  Jr, Greenfield  S, Nelson  EC, Perrin  E, Zubkoff  M.  The Medical Outcomes Study: an application of methods for monitoring the results of medical care.  JAMA. 1989;262(7):925-930.PubMedGoogle ScholarCrossref
24.
Thompson  D, Goebert  D, Takeshita  J.  A program for reducing depressive symptoms and suicidal ideation in medical students.  Acad Med. 2010;85(10):1635-1639.PubMedGoogle ScholarCrossref
25.
Radloff  LS.  The CES-D Scale: a self-report depression scale for research in the general population.  Appl Psychol Meas. 1977;1(3):385-401.Google ScholarCrossref
26.
Downs  N, Feng  W, Kirby  B,  et al.  Listening to depression and suicide risk in medical students: the Healer Education Assessment and Referral (HEAR) Program.  Acad Psychiatry. 2014;38(5):547-553.PubMedGoogle ScholarCrossref
27.
Spitzer  RL, Kroenke  K, Williams  JB.  Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study.  JAMA. 1999;282(18):1737-1744.PubMedGoogle ScholarCrossref
28.
Kroenke  K, Spitzer  RL, Williams  JB.  The PHQ-9: validity of a brief depression severity measure.  J Gen Intern Med. 2001;16(9):606-613.PubMedGoogle ScholarCrossref
29.
Seritan  AL, Rai  G, Servis  M, Pomeroy  C.  The office of student wellness: innovating to improve student mental health.  Acad Psychiatry. 2015;39(1):80-84.PubMedGoogle ScholarCrossref
30.
American Association of Medical Colleges. Graduation Questionnaire (GQ). https://www.aamc.org/data/gq/. Accessed October 21, 2016.
31.
Moutier  C, Norcross  W, Jong  P,  et al.  The suicide prevention and depression awareness program at the University of California, San Diego School of Medicine.  Acad Med. 2012;87(3):320-326.PubMedGoogle ScholarCrossref
32.
Erogul  M, Singer  G, McIntyre  T, Stefanov  DG.  Abridged mindfulness intervention to support wellness in first-year medical students.  Teach Learn Med. 2014;26(4):350-356.PubMedGoogle ScholarCrossref
33.
Neff  KD.  The development and validation of a scale to measure self-compassion.  Self Identity. 2003;2(3):223-250.Google ScholarCrossref
34.
Holtzworth-Munroe  A, Munroe  MS, Smith  RE.  Effects of a stress-management training program on first- and second-year medical students.  J Med Educ. 1985;60(5):417-419.PubMedGoogle Scholar
35.
Spielberger  CD, Gorsuch  RL, Lushene  RE, Vagg  PR, Jacobs  GA.  Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press; 1983.
36.
Kraemer  KM, Luberto  CM, O’Bryan  EM, Mysinger  E, Cotton  S.  Mind-body skills training to improve distress tolerance in medical students: a pilot study.  Teach Learn Med. 2016;28(2):219-228.PubMedGoogle ScholarCrossref
37.
Simons  JS, Gaher  RM.  The Distress Tolerance Scale: development and validation of a self-report measure.  Motiv Emot. 2005;29(2):83-102.Google ScholarCrossref
38.
Thompson  ER.  Development and validation of an internationally reliable short-form of the Positive and Negative Affect Schedule (PANAS).  J Cross Cult Psychol. 2007;38(2):227-242.Google ScholarCrossref
39.
Rosenzweig  S, Reibel  DK, Greeson  JM, Brainard  GC, Hojat  M.  Mindfulness-based stress reduction lowers psychological distress in medical students.  Teach Learn Med. 2003;15(2):88-92.PubMedGoogle ScholarCrossref
40.
Finkelstein  C, Brownstein  A, Scott  C, Lan  YL.  Anxiety and stress reduction in medical education: an intervention.  Med Educ. 2007;41(3):258-264.PubMedGoogle ScholarCrossref
41.
Derogatis  LR.  SCL-90-R Administration, Scoring and Procedures Manual, Vol ll. Towson, MD: Clinical Psychometric Research; 1983.
42.
Vitaliano  PP, Russo  J, Carr  JE, Heerwagen  JH.  Medical school pressures and their relationship to anxiety.  J Nerv Ment Dis. 1984;172(12):730-736.PubMedGoogle ScholarCrossref
43.
Whooley  MA, Avins  AL, Miranda  J, Browner  WS.  Case-finding instruments for depression: two questions are as good as many.  J Gen Intern Med. 1997;12(7):439-445.PubMedGoogle ScholarCrossref
44.
Greeson  JM, Toohey  MJ, Pearce  MJ.  An adapted, four-week mind-body skills group for medical students: reducing stress, increasing mindfulness, and enhancing self-care.  Explore (NY). 2015;11(3):186-192.PubMedGoogle ScholarCrossref
45.
Feldman  G, Hayes  A, Kumar  S,  et al.  Mindfulness and emotion regulation: the development and initial validation of the Cognitive and Affective Mindfulness Scale-Revised (CAMS-R).  J Psychopathol Behav Assess. 2006;29(3):177.Google ScholarCrossref
46.
Bond  AR, Mason  HF, Lemaster  CM,  et al.  Embodied health: the effects of a mind-body course for medical students.  Med Educ Online. 2013;18:1-8.PubMedGoogle ScholarCrossref
47.
Dyrbye  LN, Eacker  A, Durning  SJ,  et al.  The impact of stigma and personal experiences on the help-seeking behaviors of medical students with burnout.  Acad Med. 2015;90(7):961-969.PubMedGoogle ScholarCrossref
48.
Plaut  SM, Maxwell  SA, Seng  L, O’Brien  JJ, Fairclough  GF  Jr.  Mental health services for medical students: perceptions of students, student affairs deans, and mental health providers.  Acad Med. 1993;68(5):360-365.PubMedGoogle ScholarCrossref
49.
University of New Mexico Center on Alcoholism, Substance Abuse, and Addictions. The Self-Regulation Questionnaire. http://casaa.unm.edu/inst/SelfRegulation%20Questionnaire%20(SRQ).pdf. Accessed October 26, 2016.
50.
Hojat  M, Mangione  S, Nasca  TJ,  et al.  The Jefferson Scale of Physician Empathy: development and preliminary psychometric data.  Educ Psychol Meas. 2001;61(2):349-365.Google ScholarCrossref
51.
Camp  DL, Hollingsworth  MA, Zaccaro  DJ, Cariaga-Lo  LD, Richards  BF.  Does a problem-based learning curriculum affect depression in medical students?  Acad Med. 1994;69(10)(suppl):S25-S27.PubMedGoogle ScholarCrossref
52.
Zung  WW.  A self-rating depression scale.  Arch Gen Psychiatry. 1965;12:63-70.PubMedGoogle ScholarCrossref
53.
Kornitzer  B, Ronan  E, Rifkin  MR.  Improving the adjustment of educationally disadvantaged students to medical school: the Summer Enrichment Program.  Mt Sinai J Med. 2005;72(5):317-321.PubMedGoogle Scholar
54.
Drolet  BC, Rodgers  S.  A comprehensive medical student wellness program—design and implementation at Vanderbilt School of Medicine.  Acad Med. 2010;85(1):103-110.PubMedGoogle ScholarCrossref
55.
Fleming  A, Cutrer  W, Moutsios  S,  et al.  Building learning communities: evolution of the colleges at Vanderbilt University School of Medicine.  Acad Med. 2013;88(9):1246-1251.PubMedGoogle ScholarCrossref
56.
Real  FJ, Zackoff  MW, Davidson  MA, Yakes  EA.  Medical student distress and the impact of a school-sponsored wellness initiative.  Med Sci Educ. 2015;25(4):397-406.Google ScholarCrossref
57.
Spitzer  RL, Williams  JB, Kroenke  K,  et al.  Utility of a new procedure for diagnosing mental disorders in primary care: the PRIME-MD 1000 study.  JAMA. 1994;272(22):1749-1756.PubMedGoogle ScholarCrossref
58.
Slavin  SJ, Schindler  DL, Chibnall  JT.  Medical student mental health 3.0: improving student wellness through curricular changes.  Acad Med. 2014;89(4):573-577.PubMedGoogle ScholarCrossref
59.
Strayhorn  G.  Effect of a major curriculum revision on students’ perceptions of well-being.  Acad Med. 1989;64(1):25-29.PubMedGoogle ScholarCrossref
60.
Rothman  AI, Ayoade  F.  The development of a learning environment: a questionnaire for use in curriculum evaluation.  J Med Educ. 1970;45(10):754-759.PubMedGoogle Scholar
61.
Brook  RH, Ware  JE, Davies  AR,  et al.  Conceptualization and Measurement of Health for Adults in the Health Insurance Study. Vol VIII. Santa Monica, CA: RAND Corp; 1979.
62.
Whitehouse  WG, Dinges  DF, Orne  EC,  et al.  Psychosocial and immune effects of self-hypnosis training for stress management throughout the first semester of medical school.  Psychosom Med. 1996;58(3):249-263.PubMedGoogle ScholarCrossref
63.
Goetzel  RZ, Croen  LG, Shelov  S, Boufford  JI, Levin  G.  Evaluating self-help support groups for medical students.  J Med Educ. 1984;59(4):331-340.PubMedGoogle Scholar
64.
Lee  J, Graham  AV.  Students’ perception of medical school stress and their evaluation of a wellness elective.  Med Educ. 2001;35(7):652-659.PubMedGoogle ScholarCrossref
65.
Kushner  RF, Kessler  S, McGaghie  WC.  Using behavior change plans to improve medical student self-care.  Acad Med. 2011;86(7):901-906.PubMedGoogle ScholarCrossref
66.
Sastre  EA, Burke  EE, Silverstein  E,  et al.  Improvements in medical school wellness and career counseling: a comparison of one-on-one advising to an advisory college program.  Med Teach. 2010;32(10):e429-e435.PubMedGoogle ScholarCrossref
67.
Coates  WC, Crooks  K, Slavin  SJ, Guiton  G, Wilkerson  L.  Medical school curricular reform: fourth-year colleges improve access to career mentoring and overall satisfaction.  Acad Med. 2008;83(8):754-760.PubMedGoogle ScholarCrossref
68.
Ficklin  FL, Hazelwood  JD, Carter  JE, Shellhamer  RH.  Evaluation of a small-group support program for first-year medical students.  J Med Educ. 1983;58(10):817-819.PubMedGoogle Scholar
69.
Egger  M, Smith  GD, Phillips  AN.  Meta-analysis: principles and procedures.  BMJ. 1997;315(7121):1533-1537.PubMedGoogle ScholarCrossref
70.
Derogatis  LR, Melisaratos  N.  The Brief Symptom Inventory: an introductory report.  Psychol Med. 1983;13(3):595-605.PubMedGoogle ScholarCrossref
71.
Jones  KG, Pedersen  RL, Carmichael  SW, Pawlina  W.  Effects of pass/fail grading system on academic performance of first year medical students in gross anatomy course.  FASEB J. 2003;17:A385.Google Scholar
72.
Vosti  KL, Jacobs  CD.  Outcome measurement in postgraduate year one of graduates from a medical school with a pass/fail grading system.  Acad Med. 1999;74(5):547-549.PubMedGoogle ScholarCrossref
73.
Robins  LS, Fantone  JC, Oh  MS, Alexander  GL, Shlafer  M, Davis  WK.  The effect of pass/fail grading and weekly quizzes on first-year students’ performances and satisfaction.  Acad Med. 1995;70(4):327-329.PubMedGoogle ScholarCrossref
74.
Association of American Medical Colleges. Number of Medical Schools Using Selected Grading Systems in Pre-Clerkship Courses (Excluding Physical Diagnosis/Clinical Skills). https://www.aamc.org/initiatives/cir/406418/11.html. Accessed October 16, 2016.
75.
Dyrbye  LN, Thomas  MR, Shanafelt  TD.  Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students.  Acad Med. 2006;81(4):354-373.PubMedGoogle ScholarCrossref
76.
Givens  JL, Tjia  J.  Depressed medical students’ use of mental health services and barriers to use.  Acad Med. 2002;77(9):918-921.PubMedGoogle ScholarCrossref
77.
Gentile  JP, Roman  B.  Medical student mental health services: psychiatrists treating medical students.  Psychiatry (Edgmont). 2009;6(5):38-45.PubMedGoogle Scholar
78.
Rose  GL, Rukstalis  MR, Schuckit  MA.  Informal mentoring between faculty and medical students.  Acad Med. 2005;80(4):344-348.PubMedGoogle ScholarCrossref
79.
Mosteller  F, Boruch  RF.  Evidence Matters: Randomized Trials in Education Research. Washington, DC: Brookings Institution Press; 2002.
80.
Green  JL, Camilli  G, Elmore  PB.  Handbook of Complementary Methods in Education Research. 3rd ed. Washington, DC: American Educational Research Association; 2006.
81.
US Department of Health and Human Services. Federal Policy for the Protection of Human Subjects. In: Federal Register. http://archive.hhs.gov/ohrp/documents/19881110.pdf. Accessed October 16, 2016.
82.
Johansson  AC, Durning  SJ, Gruppen  LD, Olson  ME, Schwartzstein  RM, Higgins  PA.  Perspective: medical education research and the institutional review board: reexamining the process.  Acad Med. 2011;86(7):809-817.PubMedGoogle ScholarCrossref
83.
Keune  JD, Brunsvold  ME, Hohmann  E, Korndorffer  JR  Jr, Weinstein  DF, Smink  DS.  The ethics of conducting graduate medical education research on residents.  Acad Med. 2013;88(4):449-453.PubMedGoogle ScholarCrossref
84.
Russell  DW.  UCLA Loneliness Scale (Version 3): reliability, validity, and factor structure.  J Pers Assess. 1996;66(1):20-40.PubMedGoogle ScholarCrossref
85.
Moos  RH.  The assessment of the social climate of correctional institutions.  J Res Crime Delinq. 1968;5:174-188.Google ScholarCrossref
86.
Moos  RH, Houts  PS.  Assessment of the social atmospheres of psychiatric wards.  J Abnorm Psychol. 1968;73(6):595-604.PubMedGoogle ScholarCrossref
87.
Moos  RH, Insel  PM, Humphrey  B.  Preliminary Manual for Family Environment Scale, Work Environment Scale, Group Environment Scale. Palo Alto, CA: Consulting Psychologists Press; 1974.
Original Investigation
December 6, 2016

Association Between Learning Environment Interventions and Medical Student Well-beingA Systematic Review

Author Affiliations
  • 1Center for Behavioral Cardiovascular Health, Columbia University Medical Center, New York, New York
  • 2George Washington University, Washington, DC
  • 3Association of American Medical Colleges, Washington, DC
  • 4Department of Psychology, College of Liberal Arts and Sciences, University of Colorado at Denver
  • 5NewYork–Presbyterian Hospital, New York, New York
 

Copyright 2016 American Medical Association. All Rights Reserved.

JAMA. 2016;316(21):2237-2252. doi:10.1001/jama.2016.17573
Key Points

Question  What undergraduate medical education learning environment interventions are associated with improved emotional well-being among medical students?

Findings  In a systematic review of the medical literature, only 28 articles described empirically evaluated interventions and only 3 included randomization, so methodologic rigor was limited. However, some data support preclinical pass/fail grading, mental health programs, wellness programs, mentoring programs, curricular restructuring, and multicomponent program reform.

Meaning  There is limited evidence to support learning environment interventions for improvement of emotional well-being among medical students. High-quality research is needed.

Abstract

Importance  Concerns exist about the current quality of undergraduate medical education and its effect on students’ well-being.

Objective  To identify best practices for undergraduate medical education learning environment interventions that are associated with improved emotional well-being of students.

Data Sources  Learning environment interventions were identified by searching the biomedical electronic databases Ovid MEDLINE, EMBASE, the Cochrane Library, and ERIC from database inception dates to October 2016. Studies examined any intervention designed to promote medical students’ emotional well-being in the setting of a US academic medical school, with an outcome defined as students’ reports of well-being as assessed by surveys, semistructured interviews, or other quantitative methods.

Data Extraction and Synthesis  Two investigators independently reviewed abstracts and full-text articles. Data were extracted into tables to summarize results. Study quality was assessed by the Medical Education Research Study Quality Instrument (MERQSI), which has a possible range of 5 to 18; higher scores indicate higher design and methods quality and a score of 14 or higher indicates a high-quality study.

Findings  Twenty-eight articles including at least 8224 participants met eligibility criteria. Study designs included single-group cross-sectional or posttest only (n = 10), single-group pretest/posttest (n = 2), nonrandomized 2-group (n = 13), and randomized clinical trial (n = 3); 89.2% were conducted at a single site, and the mean MERSQI score for all studies was 10.3 (SD, 2.11; range, 5-13). Studies encompassed a variety of interventions, including those focused on pass/fail grading systems (n = 3; mean MERSQI score, 12.0), mental health programs (n = 4; mean MERSQI score, 11.9), mind-body skills programs (n = 7; mean MERSQI score, 11.3), curriculum structure (n = 3; mean MERSQI score, 9.5), multicomponent program reform (n = 5; mean MERSQI score, 9.4), wellness programs (n = 4; mean MERSQI score, 9.0), and advising/mentoring programs (n = 3; mean MERSQI score, 8.2).

Conclusions and Relevance  In this systematic review, limited evidence suggested that some specific learning environment interventions were associated with improved emotional well-being among medical students. However, the overall quality of the evidence was low, highlighting the need for high-quality medical education research.

Introduction

Medical schools strive to educate knowledgeable, caring, and professional physicians and pay particular attention to opportunities for improving the undergraduate medical education (UME) learning environment as they realize its influence on the education of future physicians.1

Quiz Ref IDA critical element of the learning environment is its effect on student well-being. Although matriculating US medical students begin training with significantly lower rates of depression and burnout and report better mental and emotional quality of life than other college-educated young adults,2 their reported well-being decreases during the UME years. The reported rate of moderate to severe depression is approximately 14% and of burnout symptoms is 52%—higher than reported by other graduate students or population control samples.3,4 Studies indicate that up to 11% of medical students report suicidal ideation.5

The Association of American Medical Colleges includes in its vision for improving medical education “the health and well-being of learners.”6 This systematic review evaluated the association between UME learning environment interventions and the emotional well-being of students.

Methods
Search Strategy

Potentially relevant articles were identified (Figure) by searching the biomedical electronic databases Ovid MEDLINE, EMBASE, the Cochrane Library, and ERIC from database inception dates to October 2016 (eAppendix in the Supplement). Additional records were identified by scanning the reference lists of relevant studies and reviews published between May 2011 and October 2016 and by using the “similar articles” feature in PubMed and the “cited reference search” in Web of Science. We searched for gray literature (“that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers”)7 through ongoing trial registries, academic dissertations, and websites of relevant organizations (eg, Association of American Medical Colleges) (eAppendix in the Supplement).

Selection Criteria

Studies had to have examined the outcomes associated with any intervention aiming to promote students’ emotional well-being in the setting of an academic US medical school. The well-being outcome had to be obtained through surveys, semistructured interviews, or other quantitative methods. Open-ended response formats were excluded because their methodologic quality could not be appraised with the instrument used in this review. Medical education interventions measured with open-ended responses have been reviewed and appraised elsewhere.8,9

Methodologic Quality Rating

Study quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI), which was developed to appraise the methodologic quality of quantitative medical education research.10 MERSQI scores have been positively correlated with editorial decisions to publish and with the presence of external funding for the research conducted.10 The instrument is based on 10 design and methods criteria: study design, number of institutions studied, response rate, data type, internal structure, content validity, criterion validity, appropriateness of data analysis, complexity of analysis, and outcome level. These criteria form 6 domains, each with a maximum score of 3 and a minimum of 0 or 1, that sum to produce a total score that ranges from 5 to 18.

The MERSQI was preferred to the Newcastle-Ottawa Scale–Education (NOS-E), another assessment tool for medical education research quality, because it was found to have generally higher interrater reliability (0.68-0.89)11 than the NOS-E. This may be due to its more objective assessments of design strengths and weaknesses, although it omits items on the comparability of groups and blinding.11 Although there are no defined cutoff values differentiating high-quality from low-quality study methods, 1 study used a MERSQI score of 14.0 or higher as an a priori cutoff of high quality.12

Data Extraction

Two review authors (L.M. and L.F.) independently scanned the title or abstract of all search results to determine which studies required further assessment, investigated all potentially relevant articles as full text, selected studies to include in this review, assigned a MERSQI score for each, and calculated a mean quality score across studies. Data disagreements were resolved by consultation with the third and fourth review authors (L.T.W. and I.L.). The original intention noted in the study protocol was to conduct a meta-analysis, but because of the considerable variation in the interventions, study designs, and outcomes, we did not pool the studies quantitatively, as they were judged to not be combinable.69

Results

The literature search yielded 4207 publications, of which 28 met the eligibility criteria for this systematic review (Figure). Publications were excluded if they were irrelevant or did not meet the inclusion criteria; for example, we excluded publications that focused on medical residents rather than medical students, measured academic rather than well-being outcomes, or contained interventions not focused on the learning environment. The studies included at least 8224 student participants (1 study did not report a sample size) and encompassed a variety of designs, including single-group cross-sectional or posttest only (n = 10), single-group pretest/posttest (n = 2), nonrandomized 2-group (n = 13), and randomized clinical trial (RCT; n = 3) designs; 89.2% were conducted at a single site. They had a wide range of approaches to improving students’ well-being that are categorized and described below (pass-fail grading systems [n = 3], mental health programs [n = 4], mind-body skills education/training [n = 7], curriculum structure [n = 3], multicomponent program reform [n = 5], wellness programs [n = 4], and group-based faculty advisor/mentor programs [n = 3]). Individual study results are described below and statistical details are provided for many key findings; additional results and methods are shown in Table 1 and Table 2. The included studies’ methodologic rigor varied, with MERSQI scores ranging from 5.0 to 13.0 (mean score, 10.3; SD, 2.11 [n = 28]). The mean MERSQI score in published medical education studies, as assessed in another review, was 10.0.10 The studies with the highest-quality methods crossed all types of interventions and all types of outcome measures. The highest-scored categories tested interventions involving pass/fail grading, mental health programs, and mind-body skills education/training.

Pass/Fail Grading System (Mean MERSQI Score, 12.0)

Bloodgood et al13 (n = 281; MERSQI, 11.5) and Rohe et al15 (n = 81; MERSQI, 12.0) each described that a cohort of preclinical students graded according to a pass/fail grading system, compared with an earlier student cohort evaluated according to a 5-interval grading system (A/B/C/D/F), reported statistically significantly better well-being. They reported less anxiety, depression,13 and stress15 and better well-being13 and group cohesion scores at various study time points.15 These 2 studies differed, however, in the durability of improvements. Bloodgood et al13 found no difference at 2 years between the cohort of students with a 2-year pass/fail system compared with a cohort of students with a 5-interval system on measures of anxiety (General Well-Being Schedule [GWB]14 anxiety subscore [range, 3-28; lower scores indicate more severe distress]; mean, 14.08 vs 14.20; P = .86), depression (GWB14 depression subscore [range, 2-22; lower scores indicate more severe distress]; mean, 15.56 vs 15.35; P = .71), or well-being (GWB14 well-being subscore [range, 3-18; lower scores indicate more severe distress]; mean, 10.59 vs 10.40; P = .67). Rohe et al15 reported a persistent difference at 2 years between grading cohorts on a measure of stress (Perceived Stress Scale16 [range, 0-40; higher score indicates more stress]; mean, 15.8 [SD, 6.8] vs 20.5 [SD, 7.8]; P = .01) and speculated that this difference was due to continuing reports of elevated group cohesion (Perceived Cohesion Scale19 [range, 0-36; higher scores indicate more cohesion]; mean, 33.8 [SD, 8.0] vs 29.0 [SD, 9.9]; P = .02).

Reed et al20 (n = 2056; MERSQI, 12.5) compared well-being among students at different medical schools with grading systems that were categorized as either having 3 or more intervals (eg, honors/pass/fail) or pass/fail and found that systems with 3 or more intervals were associated with statistically significantly more stress (β = 1.91; 95% CI, 1.05-2.78; P < .001) and burnout (odds ratio, 1.58; 95% CI, 1.24 to 2.01; P < .001), and a higher likelihood of considering withdrawing from medical school (odds ratio, 1.91; 95% CI, 1.30-2.80; P = .001).

Mental Health Programs (Mean MERSQI Score, 11.9)

Thompson et al24 (n = 120; MERSQI, 11.5) evaluated a multipronged program aimed at reducing mental health stigma and making services more accessible. The study found that significantly smaller proportions of the student cohort exposed to the program compared with the prior student cohort reported symptoms of mild or probable depression (14/58 [24.1%] vs 26/44 [59.1%]; P < .01) and suicidal ideation (1/33 [3.0%] vs 13/43 [30.2%]; P < .001).26 Seritan et al29 (number of participants not reported; MERSQI, 11.5) examined a different multipronged mental health/wellness program offering prevention, support, and enhanced clinical services, which was associated with improved student ratings of personal counseling, mental health, and stress management services.29 Percentages of self-referral to mental health services increased from a baseline rate of 50% to a postintervention rate of 91%. For both findings, statistical significance was not reported.29

Two studies evaluated programs consisting of education and a web-based mental health screening survey to facilitate students’ use of mental health services. Downs et al26 (n = 1008; MERSQI, 13.0) described a program that was associated with an increase in mental health service utilization and a non–statistically significant decrease in assessed suicide risk during the 4 years, perhaps due to low screening rates (34%). Moutier et al31 (n = 498; MERSQI, 11.5) reported that that 11% of medical students exposed to another educational program were referred to a mental health care professional, although no comparison was provided and the screening rate was also low (27%).

Mind-Body Skills Education/Training Programs (Mean MERSQI Score, 11.3)

Two RCTs evaluated mind-body programs. Erogul et al32 (n = 58; MERSQI, 12.0) found that students randomized to attend a mindfulness program reported a significant reduction in stress after intervention (Perceived Stress Scale16; mean change, 3.63; 95% CI, 0.37-6.89; P = .03) but not at 6-month follow-up (mean change, 2.91; 95% CI, −0.37 to 6.19; P = .08). However, students in the mind-body program reported a significant increase in self-compassion that persisted at 6-month follow-up (Self-Compassion Scale33 [range, 0-5; higher score indicates more self-compassion]; mean change, 0.56; 95% CI, 0.25-0.87; P = .001).32 In the study by Holtzworth-Munroe et al34 (n = 40; MERSQI, 10.0), students randomized to a mind-body program were reported to have significantly more awareness of tension (F5,18 = 37.16; P < .001), better ability to deal with school stress (F5,18 = 5.05; P < .04), and less test anxiety at 10-week follow-up (F1,22 = 10.42; P < .005).

Three studies evaluated mind-body programs using a pretest/posttest design with nonrandomized control groups. Kraemer et al36 (n = 52; MERSQI, 12.0) found that students undergoing mind-body skills training reported significantly improved distress tolerance (Distress Tolerance Scale G37 [range, 1-5; higher scores indicate higher levels of distress tolerance]; mean change, 0.53; 95% CI, 0.92-0.14; P = .01); no difference was found for the control group. Rosenzweig et al39 (n = 302; MERSQI, 11.0) described a mindfulness-based stress reduction program associated with significant improvements in total mood disturbance (Profile of Mood States18 [range, 0-200; higher scores indicate higher mood disturbance]; intervention group pretest mean, 38.7 [SD, 33.3] vs posttest mean, 31.8 [SD, 33.89]; P = .05; control group pretest mean, 28.0 [SD, 31.2] vs posttest mean, 38.6 [SD, 32.8]; P < .001; interaction P < .001). Finkelstein et al40 (n = 72; MERSQI, 11.0) found a significant group × time interaction association with improved anxiety (F1,2 = 3.95; P < .05).

Two studies evaluating medical student mind-body programs with a pretest/posttest design without a control group also reported associations with significant improvements in well-being. Greeson et al44 (n = 44; MERSQI, 11.5) reported improved stress (Perceived Stress Scale16; pretest mean, 29.73 [SD, 9.61]; posttest mean, 20.25 [SD, 9.03]; P < .001; d = 1.38) and mindfulness (pretest mean, 29.24 [SD, 5.54]; posttest mean, 33.88 [SD, 6.13]; P < .001; d = 0.92). Bond et al46 (n = 27; MERSQI, 11.5) reported improved self-regulation (Self-Regulation Questionnaire49 [range, 1-5; higher score indicates more self-regulation]; mean change, 0.13 [SD, 0.20]; P = .003; d = −0.41) and self-compassion (Self-Compassion Scale33; mean change, 0.28 [SD, 0.61]; P = .04; d = −0.55).

Curriculum Structure (Mean MERSQI Score, 9.5)

Elements of curriculum structure targeted by studies identified in this review were varied. Reed et al20 (n = 2056; MERSQI, 12.5) compared elements of curriculum structure at different medical schools. Students who reported more clinical contact hours were significantly less likely to report serious thoughts of dropping out (odds ratio, 0.96; 95% CI, 0.93-1.00; P = .03). Although the number of tests was not associated with any difference in well-being, spending more time taking tests was associated with significantly higher perceived stress (β = 0.29; 95% CI, 0.10-0.84; P = .003) and lower mental quality of life (β = 2.79; 95% CI, 4.09-1.50; P < .001).20

Camp et al51 (n = 275; MERSQI, 12.0) found that students in a new problem-based learning curriculum, compared with a lecture-based one, had similar reports of depression with covariate adjustment. A prematriculation summer enrichment program for medicine and nonscience undergraduate majors from underrepresented groups described reports of gaining confidence, making friends, and perceiving an easier transition to medical school (n = 92; MERSQI, 7.0).53

Multicomponent Program Reform (Mean MERSQI Score, 9.4)

Vanderbilt University restructured its medical school learning environment, which, after multiple iterations, ultimately took the form of “learning communities” or colleges within the school. These intentionally developed groups of faculty and students work together longitudinally, with functions that include mentoring, wellness programming (including mind-body skill training, career advising, and personal and professional development), and formal medical humanities coursework. Several different studies evaluated the multicomponent program at various stages of its development and implementation. Drolet and Rodgers54 (n = 116; MERSQI, 6.5) evaluated the faculty advisor/mentor program after the addition of several components and found that 95% of students reported a positive experience with the wellness program. Fleming et al55 (n = 245; MERSQI, 6.0) assessed the association of the most recent program iteration, including colleges, and found that more than 91% of students reported that colleges contributed at least somewhat meaningfully to their medical school experience. Real et al56 (n = 450; MERSQI, 10.5) reported that students credited the program in general (and more specifically, faculty mentors), the student-led programming committee, and annual retreats with lowering reported rates of burnout.

The St Louis University School of Medicine also undertook multicomponent program reform that was introduced in phases to preclinical students: (1 component) pass/fail grading for preclinical courses, reduced preclinical contact hours, extended electives, and learning communities; (2 components) addition of mind-body skills training; and (3 components) addition of anatomy course reform. As reported in a study by Slavin et al58 (n = 890; MERSQI, 12.0), phase 1 was significantly associated with improved depression, stress, and cohesion by the end of the second year of UME. The 2-component phase was associated with significantly improved anxiety, stress, and cohesion by the end of the first year of UME; depression was reported to be improved by the end of the second year of UME.58 The 3-component phase was associated with statistically significant improvements in all measures of well-being by the end of the first year, persisting through the second year of UME.58

Strayhorn59 (n = 478; MERSQI, 12.0) compared one school’s curriculum changes with a comparison school’s curriculum and found significant time × school interactions that favored the changes with regard to reported stressors (F1467 = 6.41; P = .01), mental well-being (F1460 = 9.32; P = .002), and social well-being (F1466 = 5.37; P = .02).

Miscellaneous Wellness Programs (Mean MERSQI Score, 9.0)

In a self-hypnosis training RCT, Whitehouse et al62 (n = 35; MERSQI, 12.0) reported significant improvements in anxiety (Brief Symptom Inventory70 [range, 20-80; higher scores indicate higher anxiety]; orientation mean, 59.23 [SD, 9.41]; late semester mean, 56.31 [SD, 9.29]; examination stressor mean, 58.59 [SD, 10.43]; recovery mean, 52.64 [SD, 9.66]; interaction F3,96 = 2.96; P < .05). A cross-sectional survey (n = 26; MERSQI, 9.0) about access to student support groups reported that a majority of students felt less lonely and unique with their problems.63 An evaluation of a wellness elective (n = 66; MERSQI, 7.0) reported that only a minority of students agreed or strongly agreed that it altered their report of the importance of well-being or permission for self-care or provided coping strategies (no significance values reported).64 Kushner et al65 (n = 343; MERSQI, 8.0) evaluated a wellness course that included a section on behavior change plans; of the 9 students who set mental/emotional health goals, 6 reported achieving their goals (no significance values reported).

Group-Based Faculty Advisor/Mentor Programs (Mean MERSQI Score, 8.2)

Three studies evaluated small group–based faculty advisor/mentor programs that were formally integrated into the academic curriculum. Sastre et al66 (n = 318; MERSQI, 9.5) evaluated a program in which competitively selected faculty had protected time for advising groups of students. Compared with students with traditional one-on-one volunteer faculty advisors, intervention students were significantly more likely to report that they agreed or strongly agreed that they were satisfied with how faculty advisors promoted wellness (72% vs 27%; P < .001) and that they agreed or strongly agreed that they would feel comfortable discussing their personal stress (62% vs 24%; P < .001) or mental health (51% vs 27%; P < .001) with their advisor.66 Coates et al67 (n = 100; MERSQI, 8.0) reported that fourth-year medical students involved in an intervention said they felt connected with faculty and with classmates (no significance values reported).

The evaluation of a program exclusively for first-year students by Ficklin et al68 (n = 151; MERSQI, 7.0) reported that students stated they were better acquainted with their peers, became close with some classmates, and were helped with anxiety related to starting medical school as a result of the program, but there was no comparison group and no significance values were reported.

Discussion

This systematic review identified hundreds of articles on the UME learning environment, but only a small subset contained empirically evaluated interventions. No studies included in this systematic review met a quality cutoff of 14.0.12 Improving the content and context of the delivery of UME will benefit from studies with rigorous design, objective data collection, and appropriate intervention comparators, as used in other scientific and educational fields. Despite these limitations in the evidence, there are a number of key findings from this review that may be relevant for US medical schools.

Quiz Ref IDFirst, implementation of a preclinical pass/fail grading system should be considered. All of the studies reviewed here show that a preclinical pass/fail grading system improves medical student well-being. The duration of benefit can be finite, with any positive effect perhaps more likely to persist in the context of good medical school class cohesion.15 It is also important to consider educational repercussions of changing grading systems to ensure that rigorous mastery of educational material and professional preparedness is balanced with student well-being. Two studies in this review addressed this concern by showing that pass/fail grading systems can be associated with improved well-being without any significant change in course test scores, including United States Medical Licensing Examination Step 1 and 2 scores and subsequent postresidency specialty board certification scores.13,15 This is consistent with other literature exclusively focused on academic outcomes of pass/fail grading.71-73 According to the 2014-2015 Liaison Committee on Medical Education Annual Medical School Questionnaire, 87 of the 144 participating schools used pass/fail grading systems for at least some portion of the preclinical courses.74

Second, the accessibility and quality of mental health programs for medical students, as well as any stigma associated with these programs, should be taken into account.75 Students with mental health problems may be undertreated; in one study, fewer than half of the students who reported having contemplated suicide during medical school received counseling for their depression.76 Addressing mental health conditions with a formal program that includes treatment services is essential, and a multipronged program aimed at improving awareness, reducing stigma, and improving access to mental health care professionals seems to be an efficacious approach and is associated with lower depression and suicidal ideation rates.24

Quiz Ref IDThere are specific components of mental health programs that can be critical to improving students’ well-being. Barriers to medical students’ mental health treatment reported elsewhere include concern about stigma and lack of confidentiality, such as fear of documentation in the academic record and evaluators’ knowledge of student mental health conditions with subsequent career implications.4,47,48,76,77 Medical students reported preferring help from a mental health specialist, family, or friends rather than medical school personnel47 and reported preferring accessing mental health services through a location other than the office of student affairs.48 In other studies, students have reported concerns about time, convenience of office hours, location, and financial costs.4,47,48,77 Although these are small studies of implementation issues, they are worth considering for the introduction of student mental health programs.

Quiz Ref IDThird, introducing wellness programs that teach mind-body–based stress-reduction skills should be considered. The majority of studies in this category, including 2 RCTs, indicate that such programs are associated with reduced stress, anxiety, better mood, and higher distress tolerance. This association was found even when skills were taught in condensed workshops lasting only 4 weeks,44 which is an important factor because programs must balance benefit derived from wellness programs with time investment.

Quiz Ref IDFourth, implementation of formal faculty advisor/mentor programs based in small groups and linked with curricular content should be examined. All 3 studies in this review that evaluated faulty advisor/mentor programs were highly regarded by students as a method of promoting wellness, although only 1 study tested for statistical significance.66 However, it is important that mentors do not grade students to keep their role as advisors separate from assessment to foster open communication.68 A small group–based mentoring model—rather than a one-on-one mentoring system—reduces the number of required faculty mentoring positions, allowing medical schools to have competitive selection for a subset of excellent faculty and may even enable financial support for this function.68 Outstanding faculty mentors are critical to the success of any mentoring program because they both relay explicit academic knowledge and exemplify implicit knowledge on professionalism, ethics, and values—the “hidden curriculum.”78

Fifth, the curriculum should be structured to balance clinical and nonclinical learning environments. Medical students report less burnout and stress when clinical time is increased.20 Many recent changes to curriculum have decreased clinical learning exposures, so consideration of where this movement can be reversed will be useful.

Sixth, comprehensive reform of the learning environment that incorporates many of these interventions is likely required. A detailed evaluation of the sequential implementation phases indicates that there may have been synergies among program components that were associated with improvements in medical student well-being.58

This study has a number of limitations. First, the primary studies varied widely in design, intervention content, and outcomes collected, precluding meta-analytic pooling. Second, the scope of the review was restricted to studies evaluating the quantitative effect of learning environment interventions on medical student well-being, although there are other aspects of the learning environment that deserve attention in a comprehensive redesign of the learning offered to medical students. Third, qualitative research was not included in this systematic review. Fourth, there are concerns about the ethics of randomization of education research.79,80 Historically, research conducted in established educational settings and involving normal educational practices were considered exempt from institutional review board oversight.81 However, issues of coercion and lack of informed consent about randomization of medical students when conducting learning environment interventions tests have recently been raised.82,83 These issues are complex and include whether there is a research component to the investigation of the education practice; whether there is an intent to publish; whether empirically established practices already exist; and whether the investigator has a hierarchical relationship to the participants, such as that held by a clerkship director or faculty advisor. Guidance is provided elsewhere for future UME educators to decide when and under what circumstances randomization is ethical and practical for learning environment interventions.82,83

Conclusions

In this systematic review, limited evidence suggested that some specific learning environment interventions were associated with improved medical student emotional well-being. However, the overall quality of these studies was low, highlighting the need for high-quality medical education research.

Back to top
Article Information

Corresponding Author: Karina W. Davidson, PhD, MASc, Center for Behavioral Cardiovascular Health, Columbia University Medical Center, 622 W 168th St, PH 9-314, New York, NY 10032 (kd2124@cumc.columbia.edu).

Author Contributions: Drs Davidson and Wasson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Wasson, Cusmano, Meli, Louh, Young, Davidson.

Acquisition, analysis, or interpretation of data: Wasson, Cusmano, Meli, Louh, Falzon, Hampsey, Shaffer.

Drafting of the manuscript: Wasson, Cusmano, Meli, Louh, Falzon, Hampsey, Davidson.

Critical revision of the manuscript for important intellectual content: Wasson, Meli, Louh, Young, Shaffer, Davidson.

Obtained funding: Davidson.

Administrative, technical, or material support: Cusmano, Meli, Falzon, Hampsey, Davidson.

Supervision: Wasson.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: Dr Davidson was supported by research grant K24 HL084034 from the National Heart, Lung, and Blood Institute. Dr Wasson was supported by research grant K08 HS024598 from the Agency for Healthcare Research and Quality. Ms Falzon was supported by research grant U24AG052175 and contract S15-0142, both from the National Institutes of Health, and by Columbia University Medical Center.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

References
1.
Dezee  KJ, Artino  AR, Elnicki  DM, Hemmer  PA, Durning  SJ.  Medical education in the United States of America.  Med Teach. 2012;34(7):521-525.PubMedGoogle ScholarCrossref
2.
Brazeau  CM, Shanafelt  T, Durning  SJ,  et al.  Distress among matriculating medical students relative to the general population.  Acad Med. 2014;89(11):1520-1525.PubMedGoogle ScholarCrossref
3.
Dyrbye  LN, Massie  FS  Jr, Eacker  A,  et al.  Relationship between burnout and professional conduct and attitudes among US medical students.  JAMA. 2010;304(11):1173-1180.PubMedGoogle ScholarCrossref
4.
Schwenk  TL, Davis  L, Wimsatt  LA.  Depression, stigma, and suicidal ideation in medical students.  JAMA. 2010;304(11):1181-1190.PubMedGoogle ScholarCrossref
5.
Dyrbye  LN, Thomas  MR, Massie  FS,  et al.  Burnout and suicidal ideation among US medical students.  Ann Intern Med. 2008;149(5):334-341.PubMedGoogle ScholarCrossref
6.
Association of American Medical Colleges.  Educating Doctors to Provide High Quality Medical Care: A Vision for Medical Education in the United States. Washington, DC: Association of American Medical Colleges; 2004.
7.
Gelfand  J. Grey literature: plausible applications for distance education practices. In: GL’99 Conference Proceedings: Fourth International Conference on Grey Literature: “New Frontiers in Grey Literature.” Amsterdam, the Netherlands: GreyNet; January 2000:1-11. GL Conference Series No. 4.
8.
Stacy  R, Spencer  J.  Assessing the evidence in qualitative medical education research.  Med Educ. 2000;34(7):498-500.PubMedGoogle ScholarCrossref
9.
Côté  L, Turgeon  J.  Appraising qualitative research articles in medicine and medical education.  Med Teach. 2005;27(1):71-75.PubMedGoogle ScholarCrossref
10.
Reed  DA, Cook  DA, Beckman  TJ, Levine  RB, Kern  DE, Wright  SM.  Association between funding and quality of published medical education research.  JAMA. 2007;298(9):1002-1009.PubMedGoogle ScholarCrossref
11.
Cook  DA, Reed  DA.  Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale–Education.  Acad Med. 2015;90(8):1067-1076.PubMedGoogle ScholarCrossref
12.
Lin  H, Lin  E, Auditore  S, Fanning  J.  A narrative review of high-quality literature on the effects of resident duty hours reforms.  Acad Med. 2016;91(1):140-150.PubMedGoogle ScholarCrossref
13.
Bloodgood  RA, Short  JG, Jackson  JM, Martindale  JR.  A change to pass/fail grading in the first two years at one medical school results in improved psychological well-being.  Acad Med. 2009;84(5):655-662.PubMedGoogle ScholarCrossref
14.
DuPuy  HJ. Self-representations of general psychological well-being of American adults. Presented at: American Public Health Association Meeting; Los Angeles, CA; October 15-19, 1978.
15.
Rohe  DE, Barrier  PA, Clark  MM, Cook  DA, Vickers  KS, Decker  PA.  The benefits of pass-fail grading on stress, mood, and group cohesion in medical students.  Mayo Clin Proc. 2006;81(11):1443-1448.PubMedGoogle ScholarCrossref
16.
Cohen  S, Kamarck  T, Mermelstein  R.  A global measure of perceived stress.  J Health Soc Behav. 1983;24(4):385-396.PubMedGoogle ScholarCrossref
17.
Cohen  S. Perceived stress in a probability sample of the United States. In: Oskamp  SSS, ed.  The Social Psychology of Health. Thousand Oaks, CA: Sage Publications; 1988:31-67.
18.
McNair  DM, Lorr  M, Droppleman  LF.  POMS Manual Profile of Mood States. San Diego, CA: EdiTS/Educational and Industrial Testing Service; 1992.
19.
Bollen  KA, Hoyle  RH.  Perceived cohesion: a conceptual and empirical examination.  Soc Forces. 1990;69(2):479-504.Google ScholarCrossref
20.
Reed  DA, Shanafelt  TD, Satele  DW,  et al.  Relationship of pass/fail grading and curriculum structure with well-being among preclinical medical students: a multi-institutional study.  Acad Med. 2011;86(11):1367-1373.PubMedGoogle ScholarCrossref
21.
Maslach  C, Jackson  SE.  The measurement of experienced burnout.  J Organ Behav. 1981;2(2):99-113.Google ScholarCrossref
22.
Stewart  AL, Ware  JE, eds.  Measuring Functioning and Well-being: the Medical Outcome Study Approach. Durham, NC: Duke University Press; 1992.
23.
Tarlov  AR, Ware  JE  Jr, Greenfield  S, Nelson  EC, Perrin  E, Zubkoff  M.  The Medical Outcomes Study: an application of methods for monitoring the results of medical care.  JAMA. 1989;262(7):925-930.PubMedGoogle ScholarCrossref
24.
Thompson  D, Goebert  D, Takeshita  J.  A program for reducing depressive symptoms and suicidal ideation in medical students.  Acad Med. 2010;85(10):1635-1639.PubMedGoogle ScholarCrossref
25.
Radloff  LS.  The CES-D Scale: a self-report depression scale for research in the general population.  Appl Psychol Meas. 1977;1(3):385-401.Google ScholarCrossref
26.
Downs  N, Feng  W, Kirby  B,  et al.  Listening to depression and suicide risk in medical students: the Healer Education Assessment and Referral (HEAR) Program.  Acad Psychiatry. 2014;38(5):547-553.PubMedGoogle ScholarCrossref
27.
Spitzer  RL, Kroenke  K, Williams  JB.  Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study.  JAMA. 1999;282(18):1737-1744.PubMedGoogle ScholarCrossref
28.
Kroenke  K, Spitzer  RL, Williams  JB.  The PHQ-9: validity of a brief depression severity measure.  J Gen Intern Med. 2001;16(9):606-613.PubMedGoogle ScholarCrossref
29.
Seritan  AL, Rai  G, Servis  M, Pomeroy  C.  The office of student wellness: innovating to improve student mental health.  Acad Psychiatry. 2015;39(1):80-84.PubMedGoogle ScholarCrossref
30.
American Association of Medical Colleges. Graduation Questionnaire (GQ). https://www.aamc.org/data/gq/. Accessed October 21, 2016.
31.
Moutier  C, Norcross  W, Jong  P,  et al.  The suicide prevention and depression awareness program at the University of California, San Diego School of Medicine.  Acad Med. 2012;87(3):320-326.PubMedGoogle ScholarCrossref
32.
Erogul  M, Singer  G, McIntyre  T, Stefanov  DG.  Abridged mindfulness intervention to support wellness in first-year medical students.  Teach Learn Med. 2014;26(4):350-356.PubMedGoogle ScholarCrossref
33.
Neff  KD.  The development and validation of a scale to measure self-compassion.  Self Identity. 2003;2(3):223-250.Google ScholarCrossref
34.
Holtzworth-Munroe  A, Munroe  MS, Smith  RE.  Effects of a stress-management training program on first- and second-year medical students.  J Med Educ. 1985;60(5):417-419.PubMedGoogle Scholar
35.
Spielberger  CD, Gorsuch  RL, Lushene  RE, Vagg  PR, Jacobs  GA.  Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press; 1983.
36.
Kraemer  KM, Luberto  CM, O’Bryan  EM, Mysinger  E, Cotton  S.  Mind-body skills training to improve distress tolerance in medical students: a pilot study.  Teach Learn Med. 2016;28(2):219-228.PubMedGoogle ScholarCrossref
37.
Simons  JS, Gaher  RM.  The Distress Tolerance Scale: development and validation of a self-report measure.  Motiv Emot. 2005;29(2):83-102.Google ScholarCrossref
38.
Thompson  ER.  Development and validation of an internationally reliable short-form of the Positive and Negative Affect Schedule (PANAS).  J Cross Cult Psychol. 2007;38(2):227-242.Google ScholarCrossref
39.
Rosenzweig  S, Reibel  DK, Greeson  JM, Brainard  GC, Hojat  M.  Mindfulness-based stress reduction lowers psychological distress in medical students.  Teach Learn Med. 2003;15(2):88-92.PubMedGoogle ScholarCrossref
40.
Finkelstein  C, Brownstein  A, Scott  C, Lan  YL.  Anxiety and stress reduction in medical education: an intervention.  Med Educ. 2007;41(3):258-264.PubMedGoogle ScholarCrossref
41.
Derogatis  LR.  SCL-90-R Administration, Scoring and Procedures Manual, Vol ll. Towson, MD: Clinical Psychometric Research; 1983.
42.
Vitaliano  PP, Russo  J, Carr  JE, Heerwagen  JH.  Medical school pressures and their relationship to anxiety.  J Nerv Ment Dis. 1984;172(12):730-736.PubMedGoogle ScholarCrossref
43.
Whooley  MA, Avins  AL, Miranda  J, Browner  WS.  Case-finding instruments for depression: two questions are as good as many.  J Gen Intern Med. 1997;12(7):439-445.PubMedGoogle ScholarCrossref
44.
Greeson  JM, Toohey  MJ, Pearce  MJ.  An adapted, four-week mind-body skills group for medical students: reducing stress, increasing mindfulness, and enhancing self-care.  Explore (NY). 2015;11(3):186-192.PubMedGoogle ScholarCrossref
45.
Feldman  G, Hayes  A, Kumar  S,  et al.  Mindfulness and emotion regulation: the development and initial validation of the Cognitive and Affective Mindfulness Scale-Revised (CAMS-R).  J Psychopathol Behav Assess. 2006;29(3):177.Google ScholarCrossref
46.
Bond  AR, Mason  HF, Lemaster  CM,  et al.  Embodied health: the effects of a mind-body course for medical students.  Med Educ Online. 2013;18:1-8.PubMedGoogle ScholarCrossref
47.
Dyrbye  LN, Eacker  A, Durning  SJ,  et al.  The impact of stigma and personal experiences on the help-seeking behaviors of medical students with burnout.  Acad Med. 2015;90(7):961-969.PubMedGoogle ScholarCrossref
48.
Plaut  SM, Maxwell  SA, Seng  L, O’Brien  JJ, Fairclough  GF  Jr.  Mental health services for medical students: perceptions of students, student affairs deans, and mental health providers.  Acad Med. 1993;68(5):360-365.PubMedGoogle ScholarCrossref
49.
University of New Mexico Center on Alcoholism, Substance Abuse, and Addictions. The Self-Regulation Questionnaire. http://casaa.unm.edu/inst/SelfRegulation%20Questionnaire%20(SRQ).pdf. Accessed October 26, 2016.
50.
Hojat  M, Mangione  S, Nasca  TJ,  et al.  The Jefferson Scale of Physician Empathy: development and preliminary psychometric data.  Educ Psychol Meas. 2001;61(2):349-365.Google ScholarCrossref
51.
Camp  DL, Hollingsworth  MA, Zaccaro  DJ, Cariaga-Lo  LD, Richards  BF.  Does a problem-based learning curriculum affect depression in medical students?  Acad Med. 1994;69(10)(suppl):S25-S27.PubMedGoogle ScholarCrossref
52.
Zung  WW.  A self-rating depression scale.  Arch Gen Psychiatry. 1965;12:63-70.PubMedGoogle ScholarCrossref
53.
Kornitzer  B, Ronan  E, Rifkin  MR.  Improving the adjustment of educationally disadvantaged students to medical school: the Summer Enrichment Program.  Mt Sinai J Med. 2005;72(5):317-321.PubMedGoogle Scholar
54.
Drolet  BC, Rodgers  S.  A comprehensive medical student wellness program—design and implementation at Vanderbilt School of Medicine.  Acad Med. 2010;85(1):103-110.PubMedGoogle ScholarCrossref
55.
Fleming  A, Cutrer  W, Moutsios  S,  et al.  Building learning communities: evolution of the colleges at Vanderbilt University School of Medicine.  Acad Med. 2013;88(9):1246-1251.PubMedGoogle ScholarCrossref
56.
Real  FJ, Zackoff  MW, Davidson  MA, Yakes  EA.  Medical student distress and the impact of a school-sponsored wellness initiative.  Med Sci Educ. 2015;25(4):397-406.Google ScholarCrossref
57.
Spitzer  RL, Williams  JB, Kroenke  K,  et al.  Utility of a new procedure for diagnosing mental disorders in primary care: the PRIME-MD 1000 study.  JAMA. 1994;272(22):1749-1756.PubMedGoogle ScholarCrossref
58.
Slavin  SJ, Schindler  DL, Chibnall  JT.  Medical student mental health 3.0: improving student wellness through curricular changes.  Acad Med. 2014;89(4):573-577.PubMedGoogle ScholarCrossref
59.
Strayhorn  G.  Effect of a major curriculum revision on students’ perceptions of well-being.  Acad Med. 1989;64(1):25-29.PubMedGoogle ScholarCrossref
60.
Rothman  AI, Ayoade  F.  The development of a learning environment: a questionnaire for use in curriculum evaluation.  J Med Educ. 1970;45(10):754-759.PubMedGoogle Scholar
61.
Brook  RH, Ware  JE, Davies  AR,  et al.  Conceptualization and Measurement of Health for Adults in the Health Insurance Study. Vol VIII. Santa Monica, CA: RAND Corp; 1979.
62.
Whitehouse  WG, Dinges  DF, Orne  EC,  et al.  Psychosocial and immune effects of self-hypnosis training for stress management throughout the first semester of medical school.  Psychosom Med. 1996;58(3):249-263.PubMedGoogle ScholarCrossref
63.
Goetzel  RZ, Croen  LG, Shelov  S, Boufford  JI, Levin  G.  Evaluating self-help support groups for medical students.  J Med Educ. 1984;59(4):331-340.PubMedGoogle Scholar
64.
Lee  J, Graham  AV.  Students’ perception of medical school stress and their evaluation of a wellness elective.  Med Educ. 2001;35(7):652-659.PubMedGoogle ScholarCrossref
65.
Kushner  RF, Kessler  S, McGaghie  WC.  Using behavior change plans to improve medical student self-care.  Acad Med. 2011;86(7):901-906.PubMedGoogle ScholarCrossref
66.
Sastre  EA, Burke  EE, Silverstein  E,  et al.  Improvements in medical school wellness and career counseling: a comparison of one-on-one advising to an advisory college program.  Med Teach. 2010;32(10):e429-e435.PubMedGoogle ScholarCrossref
67.
Coates  WC, Crooks  K, Slavin  SJ, Guiton  G, Wilkerson  L.  Medical school curricular reform: fourth-year colleges improve access to career mentoring and overall satisfaction.  Acad Med. 2008;83(8):754-760.PubMedGoogle ScholarCrossref
68.
Ficklin  FL, Hazelwood  JD, Carter  JE, Shellhamer  RH.  Evaluation of a small-group support program for first-year medical students.  J Med Educ. 1983;58(10):817-819.PubMedGoogle Scholar
69.
Egger  M, Smith  GD, Phillips  AN.  Meta-analysis: principles and procedures.  BMJ. 1997;315(7121):1533-1537.PubMedGoogle ScholarCrossref
70.
Derogatis  LR, Melisaratos  N.  The Brief Symptom Inventory: an introductory report.  Psychol Med. 1983;13(3):595-605.PubMedGoogle ScholarCrossref
71.
Jones  KG, Pedersen  RL, Carmichael  SW, Pawlina  W.  Effects of pass/fail grading system on academic performance of first year medical students in gross anatomy course.  FASEB J. 2003;17:A385.Google Scholar
72.
Vosti  KL, Jacobs  CD.  Outcome measurement in postgraduate year one of graduates from a medical school with a pass/fail grading system.  Acad Med. 1999;74(5):547-549.PubMedGoogle ScholarCrossref
73.
Robins  LS, Fantone  JC, Oh  MS, Alexander  GL, Shlafer  M, Davis  WK.  The effect of pass/fail grading and weekly quizzes on first-year students’ performances and satisfaction.  Acad Med. 1995;70(4):327-329.PubMedGoogle ScholarCrossref
74.
Association of American Medical Colleges. Number of Medical Schools Using Selected Grading Systems in Pre-Clerkship Courses (Excluding Physical Diagnosis/Clinical Skills). https://www.aamc.org/initiatives/cir/406418/11.html. Accessed October 16, 2016.
75.
Dyrbye  LN, Thomas  MR, Shanafelt  TD.  Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students.  Acad Med. 2006;81(4):354-373.PubMedGoogle ScholarCrossref
76.
Givens  JL, Tjia  J.  Depressed medical students’ use of mental health services and barriers to use.  Acad Med. 2002;77(9):918-921.PubMedGoogle ScholarCrossref
77.
Gentile  JP, Roman  B.  Medical student mental health services: psychiatrists treating medical students.  Psychiatry (Edgmont). 2009;6(5):38-45.PubMedGoogle Scholar
78.
Rose  GL, Rukstalis  MR, Schuckit  MA.  Informal mentoring between faculty and medical students.  Acad Med. 2005;80(4):344-348.PubMedGoogle ScholarCrossref
79.
Mosteller  F, Boruch  RF.  Evidence Matters: Randomized Trials in Education Research. Washington, DC: Brookings Institution Press; 2002.
80.
Green  JL, Camilli  G, Elmore  PB.  Handbook of Complementary Methods in Education Research. 3rd ed. Washington, DC: American Educational Research Association; 2006.
81.
US Department of Health and Human Services. Federal Policy for the Protection of Human Subjects. In: Federal Register. http://archive.hhs.gov/ohrp/documents/19881110.pdf. Accessed October 16, 2016.
82.
Johansson  AC, Durning  SJ, Gruppen  LD, Olson  ME, Schwartzstein  RM, Higgins  PA.  Perspective: medical education research and the institutional review board: reexamining the process.  Acad Med. 2011;86(7):809-817.PubMedGoogle ScholarCrossref
83.
Keune  JD, Brunsvold  ME, Hohmann  E, Korndorffer  JR  Jr, Weinstein  DF, Smink  DS.  The ethics of conducting graduate medical education research on residents.  Acad Med. 2013;88(4):449-453.PubMedGoogle ScholarCrossref
84.
Russell  DW.  UCLA Loneliness Scale (Version 3): reliability, validity, and factor structure.  J Pers Assess. 1996;66(1):20-40.PubMedGoogle ScholarCrossref
85.
Moos  RH.  The assessment of the social climate of correctional institutions.  J Res Crime Delinq. 1968;5:174-188.Google ScholarCrossref
86.
Moos  RH, Houts  PS.  Assessment of the social atmospheres of psychiatric wards.  J Abnorm Psychol. 1968;73(6):595-604.PubMedGoogle ScholarCrossref
87.
Moos  RH, Insel  PM, Humphrey  B.  Preliminary Manual for Family Environment Scale, Work Environment Scale, Group Environment Scale. Palo Alto, CA: Consulting Psychologists Press; 1974.
×