Residency Program Director Perspectives on Changes to US Medical Licensing Examination

This survey study examines the perspectives of residency program directors regarding reporting the US Medical Licensing Examination (USMLE) Step 1 as pass/fail and discontinuing Step 2 Clinical Skills.


Introduction
In recent landmark decisions on February 12, 2020, and January 26, 2021, the United States Medical Licensing Examination (USMLE) cosponsors announced the adoption of reporting Step 1 pass/fail and the discontinuation of Step 2 Clinical Skills (CS), respectively.These changes were met with mixed reviews from program directors and medical students applying to residency. 1,2 Conversely, Step 2 CS was rated as slightly more important than Step 1 and Step 2 CK, supporting the discontinued examination's value when ranking applicants.However, with Step 1 now reported as pass/fail and Step 2 CS discontinued, there remains uncertainty regarding how PDs will tailor their review of applications.Understanding PDs' perspectives on these consequential changes can guide educators reshaping their curricula and students aiming to strengthen their candidacy for residency.

Methods
The authors (A.W., J.D.S., K.L.K.) manually queried a subset (1600 of more than 5000, outreach >50% for every medical specialty except internal medicine and family medicine) of valid PD emails through the Accreditation Council for Graduate Medical Education's public 2019 to 2020 List of Specialty Programs (n = 31) across all medical specialties.In rounds, PDs were allotted 3 months (January to April 2021) to respond to the survey, with a reminder email sent after the first week.The University of California at Los Angeles institutional review board deemed this study exempt from review and waived informed consent because it used deidentified data.This study followed the American Association for Public Opinion Research (AAPOR) reporting guideline.
We created a 14-item anonymous online survey using the ExpertReview validation tool (Qualtrics XM operating system version X4 [Qualtrics International Inc]) (eTable 1 in the Supplement).
The survey (using Qualtrics and Google Forms) included questions on PD demographics including age, gender, tenure, and residency program specialty.PD race and ethnicity data were not collected to preserve anonymity.PDs were prompted for their general perceptions regarding the impact of residency selection in the context of changes to USMLE Step 1 and Step 2 CS.Responses were recorded as binary (yes or no) or on 3-point Likert scales (disagree, neutral, or agree) or 5-point Likert scales (strongly disagree, disagree, neutral, agree, or strongly agree).
Categorical variables were reported as counts and percentages.Derived 95% CIs were from the margin of errors of total sample (±3.1%) and subgroups (±4.3%) defined by AAPOR (eTable 1 in the Supplement).Subgroup analyses between regions and between Association of American Medical Colleges (AAMC)-defined primary care (internal medicine, family medicine, pediatrics, internal medicine/pediatrics) and nonprimary care specialties were conducted for all results.For final analysis, surveys with incomplete PD demographics were excluded (n = 25) and further grouping analyses were performed by converting 5-point Likert responses to 3-point (strongly disagree and

+ Supplemental content
Author affiliations and article information are listed at the end of this article.
Open Access.This is an open access article distributed under the terms of the CC-BY License.

Results
The

Discussion
Our findings suggest that PDs will consider evaluations of clinical core competency as even more important moving forward.However, narrative assessments and subinternship evaluations may be limited in their utility given increasing concern regarding racial, ethnic and gender disparities in grading, [3] further highlighting the increasing need for standardized measures of clinical performance.
Reduced competition and improved student wellness are commonly cited benefits of pass/fail scoring. 4However, the vast majority of PDs responded that they would increase emphasis on Step 2 CK and NBME subject examinations, suggesting that PDs remain resolute in using standardized scores when reviewing applications.Without Step 2 CS and numerical scores for Step 1, heightened academic pressure may be placed on students to perform well on Step 2 CK as the only standardized metric.
With the quantitative changes to USMLE, PDs may be encouraged to consider other application components.Although only PDs from nonprimary care specialties reported that the personal statement will become more important, PDs across specialties will consider leadership experience and academic awards or honor society membership as more important.The latter finding is particularly important as rising concern regarding racial disparities in the Alpha Omega Alpha (AΩA) Step 2 CK 860 0.4 (0.0-3.5) 3.6 (0.5-6.7) 12.6 (9.5-15.7)25.6 (22.5-28.7)57.8 (54.7-60.9) In the National Resident Matching Program's (NRMP) 2018 survey, 78% of program directors (PD) reported that they cite Step 1/Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 1 when reviewing applications, compared with 70% of PDs for Step 2 Clinical Knowledge (CK)/COMLEX 2 Performance Evaluation (PE) and 51% for Step 2 CS/COMLEX 2 Cognitive Evaluation (CE).
Statistically significance (P < .05)was considered with plurality of Likert 3-point responses by nonoverlapping 95% CI.Statistical analysis was performed using Stata statistical software version 16.1 (StataCorp) from February to June 2021.

Table 2 ,
many PDs reported that they will place greater emphasis on tiered core clerkship

Table 2 .
Program Directors' Perspectives on Which Application Elements They Will Emphasize Following the Discontinuation of Step 2 Clinical Skills and Pass/Fail Scoring of Step 1