[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1. US Medical Licensing Examinations Taken by International Medical Graduates
Image description not available.
*No Step 1 cohort scores released.
†Combined 1999 and 2000 examinees.
Figure 2. Certificates Issued by the Educational Commission for Foreign Medical Graduates
Image description not available.
Figure 3. US Medical Licensing Examination Pass Rates for First-Time Takers
Image description not available.
IMG indicates international medical graduate; USIMG, international medical graduate (US citizen); and USMS, United States medical student.
Table 1. International Medical Graduate (IMG) Registrations for US Medical Licensing Examination*
Image description not available.
Table 2. Clinical Skills Assessment by Citizenship at Medical School by Match Year*
Image description not available.
Table 3. Percentages of Pass Rates for Clinical Skills Assessment Comparing US International Medical Graduates With Non-US International Medical Graduates by Year*
Image description not available.
1.
Educational Commission for Foreign Medical Graduates (ECFMG).  ECFMG Certification and Application for Step 1 and Step 2 of the United States Medical Licensing Examination (USMLE) and ECFMG English TestPhiladelphia, Pa: ECFMG; 1996.
2.
World Health Organization.  World Directory of Medical Schools. 7th ed. Geneva, Switzerland: World Health Organization; 2000.
3.
Whelan GP. Educational Commission for Foreign Medical Graduates: Clinical Skills Assessment prototype.  Med Teach.1999;21:156-160.
4.
Ayers WR, Boulet JR. Establishing the validity of test score inferences: performance of 4th-year US medical students on the ECFMG Clinical Skills Assessment.  Teach Learn Med.2001;13:214-220. http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11727386
5.
Boulet JR, McKinley DW, Norcini JJ, Whelan GP. Assessing the comparability of standardized patient and physician evaluations of clinical skills.  Adv Health Sci Educ Theory Pract.In press.
6.
Boulet JR, van Zanten M, McKinley DW, Gary NE. Evaluating the spoken English proficiency of graduates of foreign medical schools.  Med Educ.2001;35:767-773. http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11489105
7.
Boulet JR, Ben David MF, Ziv A.  et al.  Using standardized patients to assess the interpersonal skills of physicians.  Acad Med.1998;73(suppl):S94-S96.
8.
Whelan GP, McKinley DW, Boulet JR, Macrae J, Kamholz S. Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment.  Med Educ.2001;35:757-761.
9.
United States Medical Licensing Examination (USMLE) Secretariat.  USMLE Clinical Skills Examination: A Fact SheetPhiladelphia, Pa: USMLE Secretariat; 2002. Available at: http://www.usmle.org/news/newscse.htm. Accessibility verified July 30, 2002.
Special Communication
September 4, 2002

The Changing Pool of International Medical Graduates Seeking Certification Training in US Graduate Medical Education Programs

Author Affiliations

Author Affiliations: Educational Commission for Foreign Medical Graduates, Philadelphia, Pa (Drs Whelan, Gary, Boulet, and Hallock); and University of Medicine and Dentistry of New Jersey, Newark (Dr Kostis).

JAMA. 2002;288(9):1079-1084. doi:10.1001/jama.288.9.1079
Abstract

International medical graduates (IMGs) consistently represent approximately one fourth of both the physician workforce and the graduate medical education (GME) population of the United States. To enter into accredited US GME programs, IMGs must be certified by the Educational Commission for Foreign Medical Graduates (ECFMG). Changes in the number and characteristics of those seeking certification directly affect the GME population and the future physician workforce in the United States. In July 1998, in response to concerns that IMGs might be lacking in basic clinical skills (eg, history taking, physical examinations, communicating with patients in spoken English), the ECFMG initiated a requirement that IMGs pass a clinical skills assessment (CSA) to achieve ECFMG certification. In this study we examined the pool of IMGs seeking certification, using databases reporting on all individuals beginning the certification process from 1995 through 2001. For this period, we found that the number of IMG candidates taking the Step 1 examination decreased by 45.5% (36 983 vs 16 828), and the number of IMGs registered to take Step 2 decreased by 38.1% (31 751 vs 12 122). The number of ECFMG certificates issued annually decreased, from a range of 9000 to 12000 (1995-1998) to fewer than 6000 (1999-2001). Although the number of IMGs annually seeking and receiving certification has decreased, the quality of the applicants appears to have improved and the number of IMGs certified annually continues to adequately fill GME positions not taken by US medical graduates.

Physicians who have completed their undergraduate medical education and received their medical degrees from medical schools outside the United States, Puerto Rico, and Canada have historically comprised a significant portion of the US physician workforce, encompassing on average one quarter of the physicians practicing in the United States. Since 1958 the Educational Commission for Foreign Medical Graduates (ECFMG) has certified international medical graduates (IMGs) who wish to pursue advanced medical training in US programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). International medical graduates include graduates of all medical schools except schools in the United States, Puerto Rico, and Canada accredited by the Liaison Committee for Medical Education (LCME) or the American Osteopathic Association (AOA). Although there have been ongoing modifications in certification requirements and processes over the years, a number of recent changes may affect the size and profile of the pool of IMG applicants seeking certification by the ECFMG. Since more than 5000 IMGs enter US graduate medical education (GME) programs each year, changes in the numbers or characteristics of those seeking ECFMG certification could have a significant impact on the overall GME population and the resulting US medical work force.

ECFMG CERTIFICATION AND CLINICAL SKILLS ASSESSMENT

The detailed requirements for ECFMG certification can be found in the ECFMG's information booklet1 or online at the ECFMG Web site (http://www.ecfmg.org). Briefly, applicants must meet all requirements to obtain the final medical degree from a medical school listed in the World Directory of Medical Schools published by the World Health Organization2 and must have verification of their diplomas from a primary source. They must also submit an acceptable score on the Test of English as a Foreign Language (TOEFL) and achieve passing scores on the United States Medical Licensing Examination (USMLE) Step 1 (basic science) and USMLE Step 2 (clinical science). In 1998, a passing score on the ECFMG Clinical Skills Assessment (CSA) was added to these requirements. This was developed in response to concerns that medical students were not graduating with basic clinical skills. While that concern was addressed in the United States and Canada by an LCME mandate for the teaching and assessment of clinical skills, there is no analogous international organization that could establish similar requirements. Thus, on July 1, 1998, the ECFMG instituted the CSA3 as an additional requirement for its Standard Certificate. The CSA is a performance assessment, the purpose of which is to ensure that IMGs demonstrate the ability to gather and interpret clinical patient data and communicate effectively at a level comparable to a standard reasonably expected of students graduating from US and Canadian medical schools accredited by the LCME.

The CSA consists of 10 encounters in which candidates interact with standardized patients to obtain a relevant medical history, perform a focused physical examination, and communicate in spoken English. Candidates also must compose a patient note that is a written record of the encounter. The standardized patients document candidates' compliance with a checklist of medical history and physical examination items, and provide scores for interpersonal skills in 4 independent dimensions, as well as for spoken English proficiency. The patient notes are rated by practicing physicians, using a holistic scoring method. It is likely that the Clinical Skills Examination proposed for incorporation into the USMLE will have a very similar design and focus.

To receive a passing grade on the CSA, the candidate must meet standards for both the Integrated Clinical Encounter (ICE), representing performance in data gathering (history and physical examination) combined with performance in composing the patient note, as well as for Doctor-Patient Communications (COM), which represents interpersonal skills and spoken English proficiency. Candidates must meet standards for both the ICE and the COM components to receive a passing grade.

The CSA is offered only in Philadelphia, Pa. Thus, those coming from overseas require appropriate travel documents. The CSA is offered on a schedule, which is driven by demand. In the few months prior to the deadline for participation in the National Resident Matching Program (NRMP), the CSA typically is offered 7 days a week with both morning and afternoon sessions. In its first 3 1/2 years of operation, more than 20 000 assessments were completed. Since this examination is so new and only IMGs who pass it can proceed to GME programs, validation remains incomplete. Nevertheless, several studies48 have provided data to support the use of CSA scores and final pass/fail decisions.

The ECFMG retained all other examination requirements, including passage of the USMLE Steps 1 and 2 (or equivalents) and reporting of an acceptable score on the TOEFL (or equivalent). Initially, passing scores on all 3 examinations were required as a prerequisite for taking the CSA, but subsequently the passage of USMLE Step 2 was dropped as a prerequisite for the CSA, although it is still required for certification. Given these changes in the requirements for certification, as well as the international computerization of both Step examinations in 1999, we examined data from the ECFMG for recent trends in IMGs seeking certification.

METHODS

We examined operational and archival databases within the ECFMG to obtain numbers of IMG registrants for all examinations, performance data for USMLE Step Examinations prior to 2000 and for the CSA, TOEFL scores submitted, numbers of ECFMG certificates issued, and demographic data. Performance data for IMGs on the USMLE Step Examinations for 2000 and 2001 were obtained from the Common Identification and Biographic Information System (CIBIS), which is managed and maintained by the ECFMG and the US National Board of Medical Examiners (NBME). Unless otherwise specified, all data presented reflect only IMGs, and do not include US and Canadian graduates of LCME–accredited medical schools.

RESULTS
ECFMG Certificates Issued

Although the number of standard ECFMG certificates issued from 1990 through 1999 exceeded those issued in any previous decade in the ECFMG's history, those numbers have significantly decreased in the last few years. Although the number of certificates issued from 1995 through 1998 ranged from 9000 to more than 12 000 certificates annually, fewer than 6000 certificates per year were issued in 1999, 2000, and 2001 (Table 1).

Issuance of the certificate represents the completion of the certification process for IMGs, but looking at the number of individuals who are beginning the process, ie, the numbers at the front end of the "pipeline," can actually help project trends beyond 2001. This can best be done by looking at the numbers of IMGs taking the earlier qualifying examinations, particularly the USMLE Step Examinations.

Numbers of Applicants

Step Applicants.Table 1 shows IMG registrations for USMLE Step Examinations from 1995 through 2001. The numbers of Step 1 registrations were fairly stable from 1995 through 1997, decreased moderately in 1998, and decreased rather precipitously in 1999, with a moderate increase since then. Step 2 registrations remained relatively constant from 1995 through 1998 and then followed a course similar to those for Step 1. Comparison of 1995 registrations with those for 2001 shows an overall decrease of 58%. Because there is a significant failure rate among IMGs for the Step examinations, applicants may register several times. Thus, these numbers are considerably greater than the number of individual candidates seeking certification.

Candidates for ECFMG certification have also been required to demonstrate English comprehension either by obtaining a passing grade on the ECFMG English examination (which was discontinued in 1999) or by submitting an acceptable TOEFL score. In 1998, a total of 24 194 ECFMG English examinations were administered. In 1999 the combined total of ECFMG English examinations and TOEFL score reports was 15 094, and in 2000 a total of 12 789 TOEFL scores were reported.

Although the CSA is not necessarily the final examination in the ECFMG certification process, a candidate can take it only after passing USMLE Step 1 and either passing the ECFMG English examination or reporting an acceptable TOEFL score. Hence, the number of IMGs taking the CSA best defines the number who will ultimately attempt to actually qualify for ECFMG certification. Only years of data are available herein, since the CSA did not exist prior to 1998 and since in the first half of 1998, candidates could achieve certification without taking the CSA. Since administration of the CSA in anticipation of the yearly NRMP continues from one calendar year into January of the subsequent year, it is most useful to look at CSA volumes based on "match years," ie, February 1 through January 31. The CSA was not initiated until July 1, 1998, hence the 1999 match year was only 7 months (July 1998-January 1999), and only 1949 assessments were administered. In the subsequent full years of operation, the total number of assessments was 6429 in the 2000 match year, 6141 in the 2001 match year, and 7196 in the 2002 match year.

Demographics of Step Registrants Who Apply for Certification. There have been no recent significant changes in the distribution of registrations by geographical regions. A consistent 75% to 77% of Step registrants who graduated from non-US medical schools have been from the United States, Canada, and Europe since 1996, without any significant change since that time. Registrations from Asia were also stable over the same period, ranging from 13% to 15%. Hence, although the volume of registrations for the Step examinations has considerably decreased, the geographical distributions have been consistent.

In analyzing the distribution of registrations by geographical regions, it is important to note that the site at which the applicant chooses to take the examination does not necessarily reflect the applicant's citizenship or even country of residence at the time of the examination. Hence, the large number of registrations within the United States includes an unknown and potentially quite large number of individuals who were neither citizens of nor residing in the United States at the time of the examination but chose to come to a test site within the United States.

Figure 1 shows the total number of IMGs, by US vs non–US citizenship, who took Step 1 and Step 2 from 1995 through 2001. (The totals differ from those for USMLE registrations shown in Table 1 due to "no shows." Also, due to the introduction of computer-based testing, no Step 1 cohort scores were released in 1999, and 2000 data reflect combined 1999-2000 examinees.)

CSA Applicants.Table 2 shows basic demographics of the CSA cohorts. Average age and sex ratio have been stable over the time the CSA has been in place. Not all candidates reported their ethnicity, but based on those who did, the largest cohorts were white and Asian candidates. Although there was a small number of candidates in 1999, white candidates constituted a significantly higher percentage in that year, while in 2001 and 2002 Asian candidates were more prevalent. Perhaps consistent with that evolution, the percentage of native English speakers decreased from 39% in the 1999 match year to 22% in the 2001 match year.

In the first year of testing, US citizens comprised some 41% of all candidates, but that prevalence has since decreased so that in the 2001 and 2002 match years, US citizens had decreased to 23% of candidates. Candidates from India and Pakistan, who had historically comprised a relatively large percentage of those receiving ECFMG certification, were relatively underrepresented in the first year (16%), but in the 2001 and 2002 match years they comprised 25% and 26%, respectively, surpassing the cohort of US citizens. These figures somewhat parallel the USMLE computer-based testing (CBT) registrations, although, as previously noted, those registration regions are not synonymous with citizenship.

ECFMG Certification. The completion of the certification process is the issuance of the Standard ECFMG Certificate. The geographical distribution of the addresses to which certificates were sent over the past 7 years is shown in Figure 2. Aside from the overall decrease in numbers, the most obvious trend is the decrease in the number of certificates sent outside the United States and Canada. Again, individuals receiving these certificates in the United States would not necessarily be US citizens but may simply be residing in the United States or have a US mailing address.

Performance Data

We examined performance differences between US citizens who go abroad for their undergraduate medical education (USIMGs) and individuals whose citizenship and location of undergraduate medical education are both outside the US and Canada (non-USIMGs). Because Canadian medical schools are accredited by the LCME, graduates of those schools are included with graduates of US medical schools regardless of citizenship, and for purposes of analysis are designated as US medical graduates (USMGs).

Step Examinations.Figure 3 shows the pass rates for total IMGs, USIMGs, and non-USIMGs for first-time takers of Step 1 from 1997 through 2000. For comparison purposes, pass rates for first-time US medical students are also included. There has been a marked improvement in performance for total IMGs and non-USIMGs, with a smaller improvement in scores for USIMGs. Total IMG first-time Step 1 takers went from a pass rate of 57.5% in 1997 to 65.7% in 2001 while non-USIMGs showed similar gains, increasing from a 57.6% pass rate in 1997 to 68.3% in 2001. USIMG performance showed some improvement from 1997 to 1999 but has worsened since 1999; the 55.4% pass rate in 2001 was the lowest in the study period.

Figure 3 also shows the pass rates for Step 2 examinations. A similar pattern of improvement is seen in both groups of IMGs. The pass rates for total IMGs and non-USIMGs improved by more than 25%, from 53% in 1997 to 79% to 80% in 2001. USIMGs also had an almost 20% gain in pass rates, increasing from 56.9% in 1997 to 76.1% in 2001.

CSA Pass Rates.Table 3 shows overall and component CSA pass rates by citizenship for the 4 match years in which the CSA has been administered. When examining these data it is important to note that an original set of standards was established for the COM and ICE components of the CSA before the assessment became operational, and a second set of standards was put in place after the first 18 months of operation. Pass rates reported for the 2001 match year reflect candidates who tested under the current standard. Pass rates will likely continue to be more similar to the 2001 match year than to the original 2 years.

Overall pass rates, which had been in the 96% to 97% range in the first 2 years, currently are closer to 80%. Whereas failing candidates were more likely not to meet the standard on the COM component under the initial standards, under the current standards candidates more often fail by not meeting the standard for the ICE component. Nevertheless, the pass rates for both the ICE and the COM components tend to be similar with respect to the overall CSA candidate pool.

In match year 2002, it is evident the USIMGs had a significantly higher overall pass rate compared with non-USMGs (88.6% vs 79.7%). This is due in large part to their higher pass rates on the COM component (99.2% vs 88.8%), which in part reflects proficiency in spoken English. Pass rates for the ICE component were more similar between the groups, with the USIMGs marginally outperforming the non-USIMGs.

COMMENT

Since 1998 there has been a significant decrease in the number of graduates of non-US medical schools pursuing ECFMG certification. It is possible that the addition of the CSA to the required examinations introduced several factors that may be related to this. Many IMGs may have had little or no experience with a performance assessment examination such as the CSA, which, unlike the TOEFL, involves a rigorous assessment of spoken English proficiency. The CSA is offered only in Philadelphia, Pa. Travel to this site incurs added expense and requires appropriate travel documents. In a small number of cases the necessary documents have been denied. Finally, the examination fee, although similar to those for other examinations, may be significant for many applicants and would add to the overall costs involved in pursuing ECFMG certification.

We also found a trend toward a higher percentage of US citizens in the total ECFMG certification applicant pool. Whereas in 1995 US citizens constituted 10% or less of those taking the Step examinations, in 2001 they comprised nearly 25%. This is primarily due to the decreased number of non-US citizens, although the absolute number of US citizens was significantly increased for the 2000 Step 1 examination. Since most USIMGs take Step 1 in the earlier years of medical school, this may suggest an increasing trend of US citizens studying abroad and seeking ECFMG certification.

In the first year it was implemented, the CSA attracted a fairly high percentage (40.9%) of USIMGs from July 1998 through January 1999. Data for subsequent years show an increasing percentage of non-USIMGs, which closely parallels the percentage of nonnative English speakers. This change is primarily a reflection of increasing numbers of non-USIMGs since the number of USIMGs has been relatively steady, varying by less than 300 over the last 3 full match years.

The performance data presented on USMLE Step examinations suggests that IMGs in the current pool are more likely to pass on their first attempts than even a few years ago, despite the fact that USMLE standards have actually been raised during this period. This may represent self-selection, particularly on the part of non-USIMGs. It may be that given the increased complexity and cost of pursuing ECFMG certification, only those non-USIMGs who have reason to believe that they will do reasonably well on the examinations actually engage in the process.

Additional analysis with respect to the size and composition of the actual pool of applicants for entry into US training programs, as well as their performance in the process of obtaining positions in ACGME-accredited programs, needs to be performed and such studies are under way. Only with the addition of that data and continuous monitoring of the data presented herein can meaningful conclusions be drawn with respect to the current and future role of IMGs in American medicine. Finally, the NBME plans to institute a similar clinical skills examination for USMGs, which is scheduled to begin implementation in the fall of 2004.9 It will be instructive to compare the resulting trends with those reported herein.

References
1.
Educational Commission for Foreign Medical Graduates (ECFMG).  ECFMG Certification and Application for Step 1 and Step 2 of the United States Medical Licensing Examination (USMLE) and ECFMG English TestPhiladelphia, Pa: ECFMG; 1996.
2.
World Health Organization.  World Directory of Medical Schools. 7th ed. Geneva, Switzerland: World Health Organization; 2000.
3.
Whelan GP. Educational Commission for Foreign Medical Graduates: Clinical Skills Assessment prototype.  Med Teach.1999;21:156-160.
4.
Ayers WR, Boulet JR. Establishing the validity of test score inferences: performance of 4th-year US medical students on the ECFMG Clinical Skills Assessment.  Teach Learn Med.2001;13:214-220. http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11727386
5.
Boulet JR, McKinley DW, Norcini JJ, Whelan GP. Assessing the comparability of standardized patient and physician evaluations of clinical skills.  Adv Health Sci Educ Theory Pract.In press.
6.
Boulet JR, van Zanten M, McKinley DW, Gary NE. Evaluating the spoken English proficiency of graduates of foreign medical schools.  Med Educ.2001;35:767-773. http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11489105
7.
Boulet JR, Ben David MF, Ziv A.  et al.  Using standardized patients to assess the interpersonal skills of physicians.  Acad Med.1998;73(suppl):S94-S96.
8.
Whelan GP, McKinley DW, Boulet JR, Macrae J, Kamholz S. Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment.  Med Educ.2001;35:757-761.
9.
United States Medical Licensing Examination (USMLE) Secretariat.  USMLE Clinical Skills Examination: A Fact SheetPhiladelphia, Pa: USMLE Secretariat; 2002. Available at: http://www.usmle.org/news/newscse.htm. Accessibility verified July 30, 2002.
×