Swing S. ACGME Launches Outcomes Assessment Project. JAMA. 1998;279(18):1492. doi:10.1001/jama.279.18.1492
Edited by Gloria Robertson, Department of Resident Physician Services,
American Medical Association.
The Accreditation Council for Graduate Medical Education (ACGME) recently
launched a 3-year project to study ways to better incorporate educational
outcomes assessment into the residency review process. The long-term goal
of the project is to develop accreditation procedures that take into account
not only whether residency programs provide residents with adequate opportunities
to learn but also whether a program's residents actually develop into competent
patient care providers and professionals. The ACGME believes that this project
will lead to better ways to measure the quality of graduate medical education.
The ACGME is responsible for evaluating the quality of the 7619 residency
programs in the United States and Puerto Rico. The ACGME delegates accrediting
authority to 27 Residency Review Committees. The Residency Review Committees
conduct the evaluations within their specialty and/or subspecialties. Traditionally,
accreditation reviews have focused on process and structure variables, such
as availability of facilities and equipment, number of faculty, conferences
held, and rotations offered. This project will look into changing the reviews
to put more emphasis on outcomes.
For outcomes assessment to be truly useful, we must first determine
precisely what should be learned and measured as educational outcomes. The
changes in our nation's health system have created the need to examine whether
what residents are learning during residency is sufficient to provide care
in an environment where some stakeholders advocate evidence-based, cost-effective,
patient-centered care as a new standard. As part of its project, the ACGME
will consider the emerging needs of our health system and will identify the
general competencies that should be better developed during residency training
to address those needs. The ACGME will also continue to ensure that residents
receive specialty-specific knowledge and skills. Among the general competencies
we are examining are knowledge, skills, and attitudes pertaining to medical
informatics, professionalism, evidence-based decision making, medical ethics,
interpersonal communication, teamwork, continuous quality improvement, and
practice management in diverse health systems and settings.
Outcomes assessment will be most valuable if the results provide evidence
that residents have attained essential patient care and related professional
competencies and if the results offer feedback that will help improve the
performance of individual residents and residency programs. Currently, the
only outcome measure used by most Residency Review Committees to judge the
quality of a residency program is residents' aggregate performance on certification
board examinations, and the most commonly used method of assessing clinical
skills during training is global ratings on categories of clinical performance,
both of which fall short in one or more ways. For example, board examinations
test clinical knowledge and cognitive skills but not clinical performance.
Global ratings assess clinical performance but provide no specific feedback
to improve performance.
The ACGME project will attempt to identify other assessment methods
to supplement those currently used. Among the methods to be examined are ratings
tied to specific patient encounters, ratings of patient satisfaction, objective-structured
clinical examinations, graduates' perceptions of the adequacy of their education,
and possibly, practice patterns.
As the ACGME proceeds with its study, we will be seeking input from
different groups. We want to gather resident physicians' perceptions regarding
competencies they will need in the future, the effectiveness of current educational
practices for inculcating competencies, the effectiveness of assessment tools
for evaluating learning and providing useful feedback, and how best to use
outcomes data to improve the quality of residency education. We will gather
residents' input on these issues through short surveys and discussion forums
conducted with selected participants in residency programs and professional
We look forward to obtaining your viewpoints.