Whelan GP, Gary NE, Kostis J, Boulet JR, Hallock JA. The Changing Pool of International Medical Graduates Seeking Certification Training in US Graduate Medical Education Programs. JAMA. 2002;288(9):1079-1084. doi:10.1001/jama.288.9.1079
Author Affiliations: Educational Commission for Foreign Medical Graduates, Philadelphia, Pa (Drs Whelan, Gary, Boulet, and Hallock); and University of Medicine and Dentistry of New Jersey, Newark (Dr Kostis).
International medical graduates (IMGs) consistently represent approximately
one fourth of both the physician workforce and the graduate medical education
(GME) population of the United States. To enter into accredited US GME programs,
IMGs must be certified by the Educational Commission for Foreign Medical Graduates
(ECFMG). Changes in the number and characteristics of those seeking certification
directly affect the GME population and the future physician workforce in the
United States. In July 1998, in response to concerns that IMGs might be lacking
in basic clinical skills (eg, history taking, physical examinations, communicating
with patients in spoken English), the ECFMG initiated a requirement that IMGs
pass a clinical skills assessment (CSA) to achieve ECFMG certification. In
this study we examined the pool of IMGs seeking certification, using databases
reporting on all individuals beginning the certification process from 1995
through 2001. For this period, we found that the number of IMG candidates
taking the Step 1 examination decreased by 45.5% (36 983 vs 16 828),
and the number of IMGs registered to take Step 2 decreased by 38.1% (31 751
vs 12 122). The number of ECFMG certificates issued annually decreased,
from a range of 9000 to 12000 (1995-1998) to fewer than 6000 (1999-2001).
Although the number of IMGs annually seeking and receiving certification has
decreased, the quality of the applicants appears to have improved and the
number of IMGs certified annually continues to adequately fill GME positions
not taken by US medical graduates.
Physicians who have completed their undergraduate medical education
and received their medical degrees from medical schools outside the United
States, Puerto Rico, and Canada have historically comprised a significant
portion of the US physician workforce, encompassing on average one quarter
of the physicians practicing in the United States. Since 1958 the Educational
Commission for Foreign Medical Graduates (ECFMG) has certified international
medical graduates (IMGs) who wish to pursue advanced medical training in US
programs accredited by the Accreditation Council for Graduate Medical Education
(ACGME). International medical graduates include graduates of all medical
schools except schools in the United States, Puerto Rico, and Canada accredited
by the Liaison Committee for Medical Education (LCME) or the American Osteopathic
Association (AOA). Although there have been ongoing modifications in certification
requirements and processes over the years, a number of recent changes may
affect the size and profile of the pool of IMG applicants seeking certification
by the ECFMG. Since more than 5000 IMGs enter US graduate medical education
(GME) programs each year, changes in the numbers or characteristics of those
seeking ECFMG certification could have a significant impact on the overall
GME population and the resulting US medical work force.
The detailed requirements for ECFMG certification can be found in the
ECFMG's information booklet1 or online at the
ECFMG Web site (http://www.ecfmg.org). Briefly, applicants must
meet all requirements to obtain the final medical degree from a medical school
listed in the World Directory of Medical Schools
published by the World Health Organization2
and must have verification of their diplomas from a primary source. They must
also submit an acceptable score on the Test of English as a Foreign Language
(TOEFL) and achieve passing scores on the United States Medical Licensing
Examination (USMLE) Step 1 (basic science) and USMLE Step 2 (clinical science).
In 1998, a passing score on the ECFMG Clinical Skills Assessment (CSA) was
added to these requirements. This was developed in response to concerns that
medical students were not graduating with basic clinical skills. While that
concern was addressed in the United States and Canada by an LCME mandate for
the teaching and assessment of clinical skills, there is no analogous international
organization that could establish similar requirements. Thus, on July 1, 1998,
the ECFMG instituted the CSA3 as an additional
requirement for its Standard Certificate. The CSA is a performance assessment,
the purpose of which is to ensure that IMGs demonstrate the ability to gather
and interpret clinical patient data and communicate effectively at a level
comparable to a standard reasonably expected of students graduating from US
and Canadian medical schools accredited by the LCME.
The CSA consists of 10 encounters in which candidates interact with
standardized patients to obtain a relevant medical history, perform a focused
physical examination, and communicate in spoken English. Candidates also must
compose a patient note that is a written record of the encounter. The standardized
patients document candidates' compliance with a checklist of medical history
and physical examination items, and provide scores for interpersonal skills
in 4 independent dimensions, as well as for spoken English proficiency. The
patient notes are rated by practicing physicians, using a holistic scoring
method. It is likely that the Clinical Skills Examination proposed for incorporation
into the USMLE will have a very similar design and focus.
To receive a passing grade on the CSA, the candidate must meet standards
for both the Integrated Clinical Encounter (ICE), representing performance
in data gathering (history and physical examination) combined with performance
in composing the patient note, as well as for Doctor-Patient Communications
(COM), which represents interpersonal skills and spoken English proficiency.
Candidates must meet standards for both the ICE and the COM components to
receive a passing grade.
The CSA is offered only in Philadelphia, Pa. Thus, those coming from
overseas require appropriate travel documents. The CSA is offered on a schedule,
which is driven by demand. In the few months prior to the deadline for participation
in the National Resident Matching Program (NRMP), the CSA typically is offered
7 days a week with both morning and afternoon sessions. In its first 3 1/2
years of operation, more than 20 000 assessments were completed. Since
this examination is so new and only IMGs who pass it can proceed to GME programs,
validation remains incomplete. Nevertheless, several studies4- 8
have provided data to support the use of CSA scores and final pass/fail decisions.
The ECFMG retained all other examination requirements, including passage
of the USMLE Steps 1 and 2 (or equivalents) and reporting of an acceptable
score on the TOEFL (or equivalent). Initially, passing scores on all 3 examinations
were required as a prerequisite for taking the CSA, but subsequently the passage
of USMLE Step 2 was dropped as a prerequisite for the CSA, although it is
still required for certification. Given these changes in the requirements
for certification, as well as the international computerization of both Step
examinations in 1999, we examined data from the ECFMG for recent trends in
IMGs seeking certification.
We examined operational and archival databases within the ECFMG to obtain
numbers of IMG registrants for all examinations, performance data for USMLE
Step Examinations prior to 2000 and for the CSA, TOEFL scores submitted, numbers
of ECFMG certificates issued, and demographic data. Performance data for IMGs
on the USMLE Step Examinations for 2000 and 2001 were obtained from the Common
Identification and Biographic Information System (CIBIS), which is managed
and maintained by the ECFMG and the US National Board of Medical Examiners
(NBME). Unless otherwise specified, all data presented reflect only IMGs,
and do not include US and Canadian graduates of LCME–accredited medical
Although the number of standard ECFMG certificates issued from 1990
through 1999 exceeded those issued in any previous decade in the ECFMG's history,
those numbers have significantly decreased in the last few years. Although
the number of certificates issued from 1995 through 1998 ranged from 9000
to more than 12 000 certificates annually, fewer than 6000 certificates
per year were issued in 1999, 2000, and 2001 (Table 1).
Issuance of the certificate represents the completion of the certification
process for IMGs, but looking at the number of individuals who are beginning
the process, ie, the numbers at the front end of the "pipeline," can actually
help project trends beyond 2001. This can best be done by looking at the numbers
of IMGs taking the earlier qualifying examinations, particularly the USMLE
Step Applicants.Table 1 shows IMG registrations
for USMLE Step Examinations from 1995 through 2001. The numbers of Step 1
registrations were fairly stable from 1995 through 1997, decreased moderately
in 1998, and decreased rather precipitously in 1999, with a moderate increase
since then. Step 2 registrations remained relatively constant from 1995 through
1998 and then followed a course similar to those for Step 1. Comparison of
1995 registrations with those for 2001 shows an overall decrease of 58%. Because
there is a significant failure rate among IMGs for the Step examinations,
applicants may register several times. Thus, these numbers are considerably
greater than the number of individual candidates seeking certification.
Candidates for ECFMG certification have also been required to demonstrate
English comprehension either by obtaining a passing grade on the ECFMG English
examination (which was discontinued in 1999) or by submitting an acceptable
TOEFL score. In 1998, a total of 24 194 ECFMG English examinations were
administered. In 1999 the combined total of ECFMG English examinations and
TOEFL score reports was 15 094, and in 2000 a total of 12 789 TOEFL
scores were reported.
Although the CSA is not necessarily the final examination in the ECFMG
certification process, a candidate can take it only after passing USMLE Step
1 and either passing the ECFMG English examination or reporting an acceptable
TOEFL score. Hence, the number of IMGs taking the CSA best defines the number
who will ultimately attempt to actually qualify for ECFMG certification. Only
years of data are available herein, since the CSA did not exist prior to 1998
and since in the first half of 1998, candidates could achieve certification
without taking the CSA. Since administration of the CSA in anticipation of
the yearly NRMP continues from one calendar year into January of the subsequent
year, it is most useful to look at CSA volumes based on "match years," ie,
February 1 through January 31. The CSA was not initiated until July 1, 1998,
hence the 1999 match year was only 7 months (July 1998-January 1999), and
only 1949 assessments were administered. In the subsequent full years of operation,
the total number of assessments was 6429 in the 2000 match year, 6141 in the
2001 match year, and 7196 in the 2002 match year.
Demographics of Step Registrants Who Apply for Certification. There have been no recent significant changes in the distribution of
registrations by geographical regions. A consistent 75% to 77% of Step registrants
who graduated from non-US medical schools have been from the United States,
Canada, and Europe since 1996, without any significant change since that time.
Registrations from Asia were also stable over the same period, ranging from
13% to 15%. Hence, although the volume of registrations for the Step examinations
has considerably decreased, the geographical distributions have been consistent.
In analyzing the distribution of registrations by geographical regions,
it is important to note that the site at which the applicant chooses to take
the examination does not necessarily reflect the applicant's citizenship or
even country of residence at the time of the examination. Hence, the large
number of registrations within the United States includes an unknown and potentially
quite large number of individuals who were neither citizens of nor residing
in the United States at the time of the examination but chose to come to a
test site within the United States.
Figure 1 shows the total number
of IMGs, by US vs non–US citizenship, who took Step 1 and Step 2 from
1995 through 2001. (The totals differ from those for USMLE registrations shown
in Table 1 due to "no shows."
Also, due to the introduction of computer-based testing, no Step 1 cohort
scores were released in 1999, and 2000 data reflect combined 1999-2000 examinees.)
CSA Applicants.Table 2 shows basic demographics
of the CSA cohorts. Average age and sex ratio have been stable over the time
the CSA has been in place. Not all candidates reported their ethnicity, but
based on those who did, the largest cohorts were white and Asian candidates.
Although there was a small number of candidates in 1999, white candidates
constituted a significantly higher percentage in that year, while in 2001
and 2002 Asian candidates were more prevalent. Perhaps consistent with that
evolution, the percentage of native English speakers decreased from 39% in
the 1999 match year to 22% in the 2001 match year.
In the first year of testing, US citizens comprised some 41% of all
candidates, but that prevalence has since decreased so that in the 2001 and
2002 match years, US citizens had decreased to 23% of candidates. Candidates
from India and Pakistan, who had historically comprised a relatively large
percentage of those receiving ECFMG certification, were relatively underrepresented
in the first year (16%), but in the 2001 and 2002 match years they comprised
25% and 26%, respectively, surpassing the cohort of US citizens. These figures
somewhat parallel the USMLE computer-based testing (CBT) registrations, although,
as previously noted, those registration regions are not synonymous with citizenship.
ECFMG Certification. The completion of the certification process is the issuance of the Standard
ECFMG Certificate. The geographical distribution of the addresses to which
certificates were sent over the past 7 years is shown in Figure 2. Aside from the overall decrease in numbers, the most obvious
trend is the decrease in the number of certificates sent outside the United
States and Canada. Again, individuals receiving these certificates in the
United States would not necessarily be US citizens but may simply be residing
in the United States or have a US mailing address.
We examined performance differences between US citizens who go abroad
for their undergraduate medical education (USIMGs) and individuals whose citizenship
and location of undergraduate medical education are both outside the US and
Canada (non-USIMGs). Because Canadian medical schools are accredited by the
LCME, graduates of those schools are included with graduates of US medical
schools regardless of citizenship, and for purposes of analysis are designated
as US medical graduates (USMGs).
Step Examinations.Figure 3 shows the pass rates
for total IMGs, USIMGs, and non-USIMGs for first-time takers of Step 1 from
1997 through 2000. For comparison purposes, pass rates for first-time US medical
students are also included. There has been a marked improvement in performance
for total IMGs and non-USIMGs, with a smaller improvement in scores for USIMGs.
Total IMG first-time Step 1 takers went from a pass rate of 57.5% in 1997
to 65.7% in 2001 while non-USIMGs showed similar gains, increasing from a
57.6% pass rate in 1997 to 68.3% in 2001. USIMG performance showed some improvement
from 1997 to 1999 but has worsened since 1999; the 55.4% pass rate in 2001
was the lowest in the study period.
Figure 3 also shows the pass
rates for Step 2 examinations. A similar pattern of improvement is seen in
both groups of IMGs. The pass rates for total IMGs and non-USIMGs improved
by more than 25%, from 53% in 1997 to 79% to 80% in 2001. USIMGs also had
an almost 20% gain in pass rates, increasing from 56.9% in 1997 to 76.1% in
CSA Pass Rates.Table 3 shows overall and
component CSA pass rates by citizenship for the 4 match years in which the
CSA has been administered. When examining these data it is important to note
that an original set of standards was established for the COM and ICE components
of the CSA before the assessment became operational, and a second set of standards
was put in place after the first 18 months of operation. Pass rates reported
for the 2001 match year reflect candidates who tested under the current standard.
Pass rates will likely continue to be more similar to the 2001 match year
than to the original 2 years.
Overall pass rates, which had been in the 96% to 97% range in the first
2 years, currently are closer to 80%. Whereas failing candidates were more
likely not to meet the standard on the COM component under the initial standards,
under the current standards candidates more often fail by not meeting the
standard for the ICE component. Nevertheless, the pass rates for both the
ICE and the COM components tend to be similar with respect to the overall
CSA candidate pool.
In match year 2002, it is evident the USIMGs had a significantly higher
overall pass rate compared with non-USMGs (88.6% vs 79.7%). This is due in
large part to their higher pass rates on the COM component (99.2% vs 88.8%),
which in part reflects proficiency in spoken English. Pass rates for the ICE
component were more similar between the groups, with the USIMGs marginally
outperforming the non-USIMGs.
Since 1998 there has been a significant decrease in the number of graduates
of non-US medical schools pursuing ECFMG certification. It is possible that
the addition of the CSA to the required examinations introduced several factors
that may be related to this. Many IMGs may have had little or no experience
with a performance assessment examination such as the CSA, which, unlike the
TOEFL, involves a rigorous assessment of spoken English proficiency. The CSA
is offered only in Philadelphia, Pa. Travel to this site incurs added expense
and requires appropriate travel documents. In a small number of cases the
necessary documents have been denied. Finally, the examination fee, although
similar to those for other examinations, may be significant for many applicants
and would add to the overall costs involved in pursuing ECFMG certification.
We also found a trend toward a higher percentage of US citizens in the
total ECFMG certification applicant pool. Whereas in 1995 US citizens constituted
10% or less of those taking the Step examinations, in 2001 they comprised
nearly 25%. This is primarily due to the decreased number of non-US citizens,
although the absolute number of US citizens was significantly increased for
the 2000 Step 1 examination. Since most USIMGs take Step 1 in the earlier
years of medical school, this may suggest an increasing trend of US citizens
studying abroad and seeking ECFMG certification.
In the first year it was implemented, the CSA attracted a fairly high
percentage (40.9%) of USIMGs from July 1998 through January 1999. Data for
subsequent years show an increasing percentage of non-USIMGs, which closely
parallels the percentage of nonnative English speakers. This change is primarily
a reflection of increasing numbers of non-USIMGs since the number of USIMGs
has been relatively steady, varying by less than 300 over the last 3 full
The performance data presented on USMLE Step examinations suggests that
IMGs in the current pool are more likely to pass on their first attempts than
even a few years ago, despite the fact that USMLE standards have actually
been raised during this period. This may represent self-selection, particularly
on the part of non-USIMGs. It may be that given the increased complexity and
cost of pursuing ECFMG certification, only those non-USIMGs who have reason
to believe that they will do reasonably well on the examinations actually
engage in the process.
Additional analysis with respect to the size and composition of the
actual pool of applicants for entry into US training programs, as well as
their performance in the process of obtaining positions in ACGME-accredited
programs, needs to be performed and such studies are under way. Only with
the addition of that data and continuous monitoring of the data presented
herein can meaningful conclusions be drawn with respect to the current and
future role of IMGs in American medicine. Finally, the NBME plans to institute
a similar clinical skills examination for USMGs, which is scheduled to begin
implementation in the fall of 2004.9 It will
be instructive to compare the resulting trends with those reported herein.