Examination of Medical College Admission Test Scores and US Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge Scores Among Students With Disabilities

An increasing number of medical students are disclosing disabilities,¹ yet little is known about the association between Medical College Admission Test (MCAT) and US Medical Licensing Examination (USMLE) performance in this population. Previous studies were focused on a single site³ and were performed prior to changes in disability law and increases in disability disclosure.^2,3 Historical literature suggests that students with disabilities (SWD) have lower USMLE pass rates² and lower Step 1 and Step 2 Clinical Knowledge (CK) scores.³ Despite recent attention to differential MCAT performance among students with lower socioeconomic status backgrounds and those identifying as races/ethnicities underrepresented in medicine,⁴ disability has been conspicuously absent from these discussions. If the MCAT is to remain a useful tool for assessing the likelihood of success in medical school, data on performance outcomes in diverse cohorts of students are needed.⁵ We examined the association of MCAT scores, disability status and category, and performance on Step 1 and Step 2 CK scores in a multisite, multiyear cohort of SWD who matriculated following amendments to the Americans with Disabilities Act.⁶

We conducted a retrospective cohort study of 163 graduating SWD from 11 US medical schools in 2018 and 2019 matched with 2 nondisabled control (NDC) participants by self-reported gender at application and MCAT score, yielding a final sample of 488 students. The University of Michigan Medical School institutional review board approved this study. The requirement for informed consent was waived because data were deidentified. The study followed the American Association for Public Opinion Research (AAPOR) reporting guideline. SWD were dichotomized into 2 groups following previous literature.³ The cognitive group included students with psychological, learning, and attention deficit disorders; the noncognitive group included all others. Primary outcome measures included scores on USMLE Step 1 and Step 2 CK. To assess the association between MCAT and USMLE scores, we ran linear mixed models using MCAT, disability group, and the interaction between MCAT and disability group as covariates. Random effects for school and matched pairs were included to account for clustering. Secondary analyses separated SWD into those with cognitive vs noncognitive disabilities and compared each with NDC group participants using a 3-group approach. The NDC group was the reference group for all comparisons. Model assumptions included normality of error terms and random effects; both were assessed and determined to be reasonably met. A significance level of P < .05 was used in determining significant associations, and all tests were 2-sided. Statistical analyses were conducted in Stata IC version 15.1 (StataCorp).

The sample consisted of 488 participants, with 284 (58.2%) female participants and a mean (SD) MCAT score of 31.6 (3.5). Among 163 SWD, 111 (68.1%) reported cognitive disabilities, 47 (28.8%) reported noncognitive disabilities, and disabilities for 5 (3.1%) were unknown. Our models showed that MCAT scores were positively associated with USMLE Step 1 and Step 2 CK scores for all students (Table). After adjusting for MCAT score, SWD status was associated with lower mean USMLE scores (B = −11.2; 95% CI, −14.0 to −8.4). Neither model had a significant disability by MCAT interaction, suggesting that changes in MCAT scores were associated with similar changes in USMLE scores regardless of disability status (Figure). Students with noncognitive disabilities had significantly lower Step 1 scores than students in the NDC group (B = −5.8; 95% CI, −10.6 to −1.1) but did not differ significantly from students in the NDC group on Step 2 CK scores (B = −3.4; 95% CI, −7.7 to 0.9). Students with cognitive disabilities had significantly lower mean scores than both students in the NDC group and those with noncognitive disabilities on both Step 1 (cognitive disability vs NDC: B = −13.3; 95% CI, −16.6 to −10.0; cognitive vs noncognitive disability: B = −7.3; 95% CI, −12.7 to −1.9) and Step 2 CK (cognitive disability vs NDC: B = −10.7; 95% CI, −13.7 to −7.7; cognitive vs noncognitive disability: B = −7.3; 95% CI, −12.2 to −2.4).

SWD are a growing and important medical school population. Our findings confirm previous studies² showing that MCAT is strongly associated with USMLE examination scores for both SWD and students without disabilities. This study has several limitations. Dichotomized categorical groups may not fully represent the association of disability with performance. Furthermore, no data on MCAT testing accommodations were available. As the number of SWD grows, understanding the association between MCAT and USMLE scores may better inform admissions committees with concerns regarding the academic performance of applicants with disabilities.

Accepted for Publication: March 28, 2021.

Published: May 21, 2021. doi:10.1001/jamanetworkopen.2021.10914

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Purkiss J et al. JAMA Network Open.

Corresponding Author: Lisa M. Meeks, PhD, Department of Family Medicine, The University of Michigan Medical School, 1018 Fuller St, Ann Arbor, MI 48104-1213 (meeksli@med.umich.edu).

Author Contributions: Dr Meeks had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Grabowski, Kim, Meeks.

Acquisition, analysis, or interpretation of data: Purkiss, Plegue, Kim, Jain, Henderson, Meeks.

Drafting of the manuscript: Purkiss, Plegue, Grabowski, Kim, Meeks.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Plegue, Kim, Meeks.

Obtained funding: Meeks.

Administrative, technical, or material support: Purkiss, Kim, Jain, Henderson.

Supervision: Meeks.

Conflict of Interest Disclosures: None reported.

Funding/Support: Dr Meeks was supported by grant UH1HP29965 from the Health Resources and Services Administration. The Center for a Diverse Healthcare Workforce is supported by the Health Resources and Services Administration of the US Department of Health and Human Services as part of an award totaling $3 791 026 with 0% financed with nongovernmental sources.

Role of the Funder/Sponsor: The Health Resources and Services Administration approved the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement, by Health Resources and Services Administration, US Department of Health and Human Services, or the US government. For more information, please visit https://www.hrsa.gov/.