[Skip to Content]
[Skip to Content Landing]
Views 19,501
Citations 0
Viewpoint
March 6, 2020

Change in Reporting of USMLE Step 1 Scores and Potential Implications for International Medical Graduates

Author Affiliations
  • 1Department of Medicine, University of Connecticut, Farmington
  • 2Department of Hematology and Oncology, University of Alabama at Birmingham
JAMA. Published online March 6, 2020. doi:10.1001/jama.2020.2956

The US physician workforce includes allopathic physicians, osteopathic physicians, and international medical graduates (IMGs), who are physicians who received their medical school education outside the US or Canada. These physicians comprise both US citizens (US IMGs) and citizens from other countries (non-US IMGs) who have trained abroad. The US health care system has depended on IMGs to fill residency positions since the 1970s. Today, 1 in 4 physicians practicing in the US is an IMG.1 One estimate from 2001 suggested that if IMGs in primary care practice were removed, 1 of every 5 “adequately served” nonmetropolitan counties may become underserved and the percentage of rural counties with physician shortages could increase to 44.4%.2 This trend continues with the J-1 exchange waiver called the Conrad 30 Waiver, which enables IMGs to continue practicing in the US only if they commit to practice in a federally designated Health Professional Shortage Area, Medically Underserved Area, or Medically Underserved Population for at least 3 years.3 With a projected shortage of an estimated 125 000 physicians by 2025, IMGs will remain an important source of primary care physicians in rural and underserved areas.

In the life cycle of an IMG, scoring well on the United States Medical Licensing Examination (USMLE) Step 1 is an important accomplishment. Doing so not only indicates success on the examination but also increases the likelihood that an IMG could secure a residency position in a training program in the US. The USMLE Step 1, often considered one of the most difficult tests in medical education, assesses concepts of basic medical knowledge with special emphasis on principles and mechanisms underlying health and disease.

In 2018, a total of 42 420 students took the test, and the cumulative pass percentage was 86%.4 Until recently, USMLE Step 1 results were reported on a 3-digit scale with a minimum passing score of 194. On February 12, 2020, the USMLE announced that there will be a change in score reporting from a 3-digit numerical score to reporting only a pass/fail outcome, beginning sometime after January 1, 2022.5

Traditionally, USMLE Step 1 scores have been an important component in the residency application process and selection of candidates. According to the National Resident Matching Program 2018 Program Director Survey, across all specialties, 94% of the 1233 programs cited the USMLE Step 1 score as an important factor to select candidates for interview.6 Furthermore, 64% of the programs reported that they require a target score to screen applicants, whereas only 12% of programs reported that they often consider interviewing applicants who failed in the first attempt to pass Step 1. After completion of interviews, 78% of the programs still considered Step 1 scores for ranking applicants.6

Recently, an Invitational Conference on USMLE Scoring (InCUS) was held by the USMLE along with the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME), during which recommendations regarding the current structure of medical and residency application process were discussed.7 An important focus of this conference was to implement system-wide changes to improve the transition from undergraduate to graduate medical education, a matter of great concern in academic medicine. With the goal of supporting the educational engagement and overall experience of medical students, reducing the current overemphasis on Step 1 scores, and promoting student well-being, a decision was made to change the Step 1 score reporting to pass/fail.5

The NBME and FSMB indicate that moving to pass/fail reporting of Step 1 while retaining a scored Step 2 Clinical Knowledge (scored on a 3-digit scoring system between 1 and 300) represents a positive step toward system-wide change while limiting large-scale disruption to the overall educational and licensing environment. Furthermore, the Educational Commission of Foreign Medical Graduates (ECFMG) has supported the decision of the NBME and FSMB boards on these policy changes.8

While this change to pass/fail reporting of Step 1 examination scores might have an overall positive effect on educational engagement and overall experience of medical students, it might make the current scenario of residency match more complicated for both medical students and program directors. There is no doubt that some material on USMLE Step 1 is esoteric, and it is unclear how well the results on this examination are associated with “success” as a physician. If the goal is to reduce test anxiety and the content cannot be fixed, why not abolish the examination? This would also reduce the financial burden on students.

Despite their limitations, standardized tests such as the USMLE provide an objective measure of the knowledge, problem-solving skills, and resilience of an IMG. Changing the scoring system without global reform amplifies the already prevalent issues of IMGs obtaining clinical experience in the US, potentially conducting research to enhance their application, and obtaining letters of recommendation from individuals who are recognized by residency directors. The change to pass/fail may make an already uphill battle more challenging for applicants, for several reasons.

First, despite the perception that the USMLE Step 1 score was overused and overemphasized in the Electronic Residency Application Service, for IMGs it often provided a visible metric of their credibility. Traditionally, the mean Step 1 scores of matched IMGs (mean score, 228) have been higher than unmatched IMGs (mean score, 216.5), indicating that USMLE Step 1 scores play a significant role in the National Resident Matching Program match for IMGs.9 The scores may have served as a stamp of the strong candidacy of an IMG, given that some program directors may be unfamiliar with training in international medical schools. In addition, grades from international medical schools are not congruent with the grading system of US medical schools. Given this significant variation in the duration of training, curriculum, and grading systems across various countries, until recently the USMLE served as a “standardization” tool for this process. Thus, having a good Step 1 score helped to level the playing field and enable IMGs to have an opportunity for being selected for a position in competitive residency programs.

Second, this change could lead to increased emphasis on bolstering other aspects of an IMG’s application for residency programs, such as competitive clinical electives, excellent letters of recommendation, a strong research profile, and networking. Despite having clinical experience in their home countries, IMGs are required to demonstrate US clinical experience and provide letters of recommendation from US faculty. Currently, a formal and uniform pathway for IMGs to obtain this experience does not exist. This process is largely dependent on and limited by contacts, Health Insurance Portability and Accountability Act restrictions, US visa sponsorship, and the financial health of applicants. These challenges in obtaining clinical electives while living abroad coupled with the uncertainty of obtaining strong letters of recommendation in a 4-week elective rotation may severely limit the competitiveness of applications from IMGs if a standardized testing score is not used.

Third, similar issues affect the process of obtaining research experience. Because of limited formal channels for gaining this experience, IMGs often accept unpaid volunteer or postdoctoral research positions that have visa restrictions that prevent them from taking additional jobs to financially support themselves. This could expose IMGs to a risk of exploitation and additional financial pressure.

Fourth, IMGs have to pass rigorous board examinations in their home countries (similar to USMLEs) and, in some countries, are required to complete a mandatory year-long internship to graduate from medical school. Also, prior to US residency program application, many IMGs take all 4 USMLE tests (Step 1, Step 2 Clinical Knowledge, Step 2 Clinical Skills, and Step 3), which require at least a few months to a year to complete. These tests, along with accumulating US clinical experience and research experience, also lead to a significant delay in time from graduation until application to residency programs, which could affect some IMG applications negatively. Many residency programs have graduation cutoff dates and could effectively filter out applicants who have been out of medical school for more than a prespecified time.

Fifth, with increasing enrollment in US medical and osteopathic schools10 and the DO-MD single accreditation merger in 2020, IMGs are increasingly required to distinguish themselves from applicants who are not at a similar disadvantage.

It has also been suggested that “application caps” could be instituted to prevent overwhelmed program directors from using scores as a filter as opposed to performing a holistic review of the applications. However, this would require the residency programs to have up-to-date application criteria listed on their website, including their ability to sponsor the various immigrant and nonimmigrant training visas.

All IMGs applying to US residency programs need to be certified by the ECFMG to demonstrate their readiness to engage in US graduate medical education. While the USMLE has been an important component of ECFMG certification, US clinical and research experience is not. Yet US clinical experience and research experience are important metrics considered by residency programs when evaluating IMGs. An outstanding USMLE Step 1 score at least partly makes up for limited US clinical or research experience on an IMG’s application. Changing USMLE Step 1 assessment to pass/fail without reforming other aspects of the application process for IMGs could amplify existing disadvantages.

Studying and working in the US is attractive for IMGs because of the promise of meritocracy and fairness. In return, IMGs provide accessible, high-quality health care to patients in the US through their talents, diversity, and international perspectives. Therefore, entities such as the ECFMG, NBME, and FSMB should endeavor to establish a fair and formal pathway for IMGs to demonstrate equivalency of medical training.

An effective, equitable, and high-quality assessment is essential for IMGs to demonstrate their qualifications, compete for US residency positions, and continue to contribute to the US health care system. Ultimately, IMGs are valuable not because of outstanding test scores but because they succeed despite the odds stacked against them. This is the “American dream” exemplified.

(This Viewpoint is available for online commenting.)

Back to top
Article Information

Corresponding Author: Aakash Desai, MD, MPH, Department of Medicine, University of Connecticut, 263 Farmington Ave, Farmington, CT 06030 (dr.aakashdesai@gmail.com).

Published Online: March 6, 2020. doi:10.1001/jama.2020.2956

Conflict of Interest Disclosures: None reported.

Additional Information: Dr Desai is an IMG from India (B. J. Medical College), a current resident at the University of Connecticut, and an incoming hematology-oncology fellow at Mayo Clinic. Dr Hegde is an IMG from India (J. J. M. Medical College) and a current assistant professor of medicine in the Department of Hematology Oncology at the University of Alabama. Dr Das is an IMG from India (Grant Medical College) and a current assistant professor of medicine in the Department of Hematology Oncology and associate program director for Hematology Oncology fellowship at the University of Alabama.

References
1.
About ECFMG: overview. Educational Commission of Foreign Medical Graduates website. Accessed February 28, 2020. https://www.ecfmg.org/about/
2.
Baer  L, Konrad  T, Slifkin  R. If Fewer International Medical Graduates Are Allowed in the US, Who Might Replace Them in Rural, Underserved Areas. University of North Carolina at Chapel Hill; 2001. Working paper No. 71. Accessed February 28, 2020. https://www.shepscenter.unc.edu/rural/pubs/report/wp71.pdf
3.
Rural J-1 visa waiver. Rural Health Information Hub. Accessed March 2, 2020. https://www.ruralhealthinfo.org/topics/j-1-visa-waiver
4.
USMLE Step 1. United States Medical Licensing Examination website. Accessed March 2, 2020. https://www.usmle.org/step-1/
5.
USMLE program announces upcoming policy changes. United States Medical Licensing Examination website. Published February 12, 2020. Accessed March 2, 2020. https://www.usmle.org/announcements/
6.
National Resident Matching Program Data Release and Research Committee. Results of the 2018 NRMP Program Director Survey. Accessed February 28, 2020. https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf
7.
InCUS Planning Committee. Summary Report and Preliminary Recommendations from the Invitational Conference on USMLE Scoring (InCUS), March 11-12, 2019. Accessed February 28, 2020. https://www.usmle.org/pdfs/incus/incus_summary_report.pdf
8.
ECFMG statement on USMLE policy changes. Educational Commission of Foreign Medical Graduates website. Accessed February 28, 2020. https://www.ecfmg.org/news/2020/02/12/ecfmg-statement-on-usmle-policy-changes/
9.
National Resident Matching Program. Charting Outcomes in the Match: International Medical Graduates. 2nd ed. National Resident Matching Program; July 2018. Accessed February 28, 2020. https://www.nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-in-the-Match-2018-IMGs.pdf
10.
Association of American Medical Colleges. Results of the 2018 Medical School Enrollment Survey. Association of American Medical Colleges; July 2019.
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    ×