Examination of Medical College Admission Test Scores and US Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge Scores Among Students With Disabilities | Health Care Workforce | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure.  Medical College Admission Test (MCAT) Scores by US Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge (CK) Scores, for Students With and Without Disabilities
Medical College Admission Test (MCAT) Scores by US Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge (CK) Scores, for Students With and Without Disabilities

Lines indicate unadjusted regression fit lines between MCAT and US Medical Licensing Examination scores for students with disabilities (SWD) and the control group.

Table.  Association of MCAT Score With US Medical Licensing Examination Step 1 and Step 2 CK Scores for Medical Students With and Without Disabilities
Association of MCAT Score With US Medical Licensing Examination Step 1 and Step 2 CK Scores for Medical Students With and Without Disabilities
1.
Meeks  LM, Case  B, Herzer  K, Plegue  M, Swenor  BK.  Change in prevalence of disabilities and accommodation practices among US medical schools, 2016 vs 2019.   JAMA. 2019;322(20):2022-2024. doi:10.1001/jama.2019.15372PubMedGoogle ScholarCrossref
2.
Searcy  CA, Dowd  KW, Hughes  MG, Baldwin  S, Pigg  T.  Association of MCAT scores obtained with standard vs extra administration time with medical school admission, medical student performance, and time to graduation.   JAMA. 2015;313(22):2253-2262. doi:10.1001/jama.2015.5511PubMedGoogle ScholarCrossref
3.
Teherani  A, Papadakis  MA.  Clinical performance of medical students with protected disabilities.   JAMA. 2013;310(21):2309-2311. doi:10.1001/jama.2013.283198PubMedGoogle ScholarCrossref
4.
Girotti  JA, Chanatry  JA, Clinchot  DM,  et al.  Investigating group differences in examinees’ preparation for and performance on the new MCAT exam.   Acad Med. 2020;95(3):365-374. doi:10.1097/ACM.0000000000002940PubMedGoogle ScholarCrossref
5.
Busche  K, Elks  ML, Hanson  JT,  et al.  The validity of scores from the new MCAT exam in predicting student performance: results from a multisite study.   Acad Med. 2020;95(3):387-395. doi:10.1097/ACM.0000000000002942PubMedGoogle ScholarCrossref
6.
Smith  WT, Allen  WL.  Implications of the 2008 amendments to the Americans with Disabilities Act for medical education.   Acad Med. 2011;86(6):768-772. doi:10.1097/ACM.0b013e318217e325PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Research Letter
    Medical Education
    May 21, 2021

    Examination of Medical College Admission Test Scores and US Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge Scores Among Students With Disabilities

    Author Affiliations
    • 1Division of Evaluation, Assessment, and Education Research–School of Medicine, Department of Education, Innovation, and Technology, Baylor College of Medicine, Houston, Texas
    • 2Department of Family Medicine, The University of Michigan Medical School, Ann Arbor
    • 3Department of Medical Education, University of Alabama at Birmingham School of Medicine
    • 4Department of Medicine, University of Minnesota Medical School, Minneapolis
    • 5Office of Medical Education, Department of Internal Medicine, School of Medicine, University of California, Davis, Sacramento
    • 6Office of Admissions and Outreach, School of Medicine, University of California, Davis, Sacramento
    • 7Center for a Diverse Healthcare Workforce, School of Medicine, University of California, Davis, Sacramento
    JAMA Netw Open. 2021;4(5):e2110914. doi:10.1001/jamanetworkopen.2021.10914
    Introduction

    An increasing number of medical students are disclosing disabilities,1 yet little is known about the association between Medical College Admission Test (MCAT) and US Medical Licensing Examination (USMLE) performance in this population. Previous studies were focused on a single site3 and were performed prior to changes in disability law and increases in disability disclosure.2,3 Historical literature suggests that students with disabilities (SWD) have lower USMLE pass rates2 and lower Step 1 and Step 2 Clinical Knowledge (CK) scores.3 Despite recent attention to differential MCAT performance among students with lower socioeconomic status backgrounds and those identifying as races/ethnicities underrepresented in medicine,4 disability has been conspicuously absent from these discussions. If the MCAT is to remain a useful tool for assessing the likelihood of success in medical school, data on performance outcomes in diverse cohorts of students are needed.5 We examined the association of MCAT scores, disability status and category, and performance on Step 1 and Step 2 CK scores in a multisite, multiyear cohort of SWD who matriculated following amendments to the Americans with Disabilities Act.6

    Methods

    We conducted a retrospective cohort study of 163 graduating SWD from 11 US medical schools in 2018 and 2019 matched with 2 nondisabled control (NDC) participants by self-reported gender at application and MCAT score, yielding a final sample of 488 students. The University of Michigan Medical School institutional review board approved this study. The requirement for informed consent was waived because data were deidentified. The study followed the American Association for Public Opinion Research (AAPOR) reporting guideline. SWD were dichotomized into 2 groups following previous literature.3 The cognitive group included students with psychological, learning, and attention deficit disorders; the noncognitive group included all others. Primary outcome measures included scores on USMLE Step 1 and Step 2 CK. To assess the association between MCAT and USMLE scores, we ran linear mixed models using MCAT, disability group, and the interaction between MCAT and disability group as covariates. Random effects for school and matched pairs were included to account for clustering. Secondary analyses separated SWD into those with cognitive vs noncognitive disabilities and compared each with NDC group participants using a 3-group approach. The NDC group was the reference group for all comparisons. Model assumptions included normality of error terms and random effects; both were assessed and determined to be reasonably met. A significance level of P < .05 was used in determining significant associations, and all tests were 2-sided. Statistical analyses were conducted in Stata IC version 15.1 (StataCorp).

    Results

    The sample consisted of 488 participants, with 284 (58.2%) female participants and a mean (SD) MCAT score of 31.6 (3.5). Among 163 SWD, 111 (68.1%) reported cognitive disabilities, 47 (28.8%) reported noncognitive disabilities, and disabilities for 5 (3.1%) were unknown. Our models showed that MCAT scores were positively associated with USMLE Step 1 and Step 2 CK scores for all students (Table). After adjusting for MCAT score, SWD status was associated with lower mean USMLE scores (B = −11.2; 95% CI, −14.0 to −8.4). Neither model had a significant disability by MCAT interaction, suggesting that changes in MCAT scores were associated with similar changes in USMLE scores regardless of disability status (Figure). Students with noncognitive disabilities had significantly lower Step 1 scores than students in the NDC group (B = −5.8; 95% CI, −10.6 to −1.1) but did not differ significantly from students in the NDC group on Step 2 CK scores (B = −3.4; 95% CI, −7.7 to 0.9). Students with cognitive disabilities had significantly lower mean scores than both students in the NDC group and those with noncognitive disabilities on both Step 1 (cognitive disability vs NDC: B = −13.3; 95% CI, −16.6 to −10.0; cognitive vs noncognitive disability: B = −7.3; 95% CI, −12.7 to −1.9) and Step 2 CK (cognitive disability vs NDC: B = −10.7; 95% CI, −13.7 to −7.7; cognitive vs noncognitive disability: B = −7.3; 95% CI, −12.2 to −2.4).

    Discussion

    SWD are a growing and important medical school population. Our findings confirm previous studies2 showing that MCAT is strongly associated with USMLE examination scores for both SWD and students without disabilities. This study has several limitations. Dichotomized categorical groups may not fully represent the association of disability with performance. Furthermore, no data on MCAT testing accommodations were available. As the number of SWD grows, understanding the association between MCAT and USMLE scores may better inform admissions committees with concerns regarding the academic performance of applicants with disabilities.

    Back to top
    Article Information

    Accepted for Publication: March 28, 2021.

    Published: May 21, 2021. doi:10.1001/jamanetworkopen.2021.10914

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Purkiss J et al. JAMA Network Open.

    Corresponding Author: Lisa M. Meeks, PhD, Department of Family Medicine, The University of Michigan Medical School, 1018 Fuller St, Ann Arbor, MI 48104-1213 (meeksli@med.umich.edu).

    Author Contributions: Dr Meeks had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Grabowski, Kim, Meeks.

    Acquisition, analysis, or interpretation of data: Purkiss, Plegue, Kim, Jain, Henderson, Meeks.

    Drafting of the manuscript: Purkiss, Plegue, Grabowski, Kim, Meeks.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Plegue, Kim, Meeks.

    Obtained funding: Meeks.

    Administrative, technical, or material support: Purkiss, Kim, Jain, Henderson.

    Supervision: Meeks.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: Dr Meeks was supported by grant UH1HP29965 from the Health Resources and Services Administration. The Center for a Diverse Healthcare Workforce is supported by the Health Resources and Services Administration of the US Department of Health and Human Services as part of an award totaling $3 791 026 with 0% financed with nongovernmental sources.

    Role of the Funder/Sponsor: The Health Resources and Services Administration approved the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement, by Health Resources and Services Administration, US Department of Health and Human Services, or the US government. For more information, please visit https://www.hrsa.gov/.

    References
    1.
    Meeks  LM, Case  B, Herzer  K, Plegue  M, Swenor  BK.  Change in prevalence of disabilities and accommodation practices among US medical schools, 2016 vs 2019.   JAMA. 2019;322(20):2022-2024. doi:10.1001/jama.2019.15372PubMedGoogle ScholarCrossref
    2.
    Searcy  CA, Dowd  KW, Hughes  MG, Baldwin  S, Pigg  T.  Association of MCAT scores obtained with standard vs extra administration time with medical school admission, medical student performance, and time to graduation.   JAMA. 2015;313(22):2253-2262. doi:10.1001/jama.2015.5511PubMedGoogle ScholarCrossref
    3.
    Teherani  A, Papadakis  MA.  Clinical performance of medical students with protected disabilities.   JAMA. 2013;310(21):2309-2311. doi:10.1001/jama.2013.283198PubMedGoogle ScholarCrossref
    4.
    Girotti  JA, Chanatry  JA, Clinchot  DM,  et al.  Investigating group differences in examinees’ preparation for and performance on the new MCAT exam.   Acad Med. 2020;95(3):365-374. doi:10.1097/ACM.0000000000002940PubMedGoogle ScholarCrossref
    5.
    Busche  K, Elks  ML, Hanson  JT,  et al.  The validity of scores from the new MCAT exam in predicting student performance: results from a multisite study.   Acad Med. 2020;95(3):387-395. doi:10.1097/ACM.0000000000002942PubMedGoogle ScholarCrossref
    6.
    Smith  WT, Allen  WL.  Implications of the 2008 amendments to the Americans with Disabilities Act for medical education.   Acad Med. 2011;86(6):768-772. doi:10.1097/ACM.0b013e318217e325PubMedGoogle ScholarCrossref
    ×