Customize your JAMA Network experience by selecting one or more topics from the list below.
Sanfey H, DaRosa DA, Hickson GB, et al. Pursuing Professional Accountability: An Evidence-Based Approach to Addressing Residents With Behavioral Problems. Arch Surg. 2012;147(7):642–647. doi:10.1001/archsurg.2012.832
Author Affiliations: Departments of Surgery (Drs Sanfey, Mellinger, R. G. Williams, and Dunnington, and Mss Boehler and Schwind) and Medical Education (Drs Klamen and Roberts), Southern Illinois University School of Medicine, Springfield; Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois (Dr DaRosa); Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, Tennessee (Dr Hickson); Department of Behavioral Sciences, Rush University Medical Center, Chicago, Illinois (Dr B. Williams); Department of Surgery, Duke University School of Medicine, Durham, North Carolina (Dr Sudan); Department of Surgery, Washington University School of Medicine, St Louis, Missouri (Dr Klingensmith); Department of Surgery, University of Vermont, Burlington (Dr Hebert); MedStar Health, Inc, Arlington, Virginia (Ms Richard); and Division of Education, American College of Surgeons, Chicago, Illinois (Dr Sachdeva).
Objective To develop an evidence-based approach to the identification, prevention, and management of surgical residents with behavioral problems.
Design The American College of Surgeons and Southern Illinois University Department of Surgery hosted a 1-day think tank to develop strategies for early identification of problem residents and appropriate interventions. Participants read a selection of relevant literature before the meeting and reviewed case reports.
Setting American College of Surgeons headquarters, Chicago, Illinois.
Participants Medical and nursing leaders in the field of resident education; individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians; and surgical residents.
Main Outcome Measures Evidence-based strategies for the identification, prevention, and management of problem residents.
Results Recommendations based on the literature and expert opinions have been made for the identification, remediation, and reassessment of problem residents.
Conclusions It is essential to set clear expectations for professional behavior with faculty and residents. A notice of deficiency should define the expected acceptable behavior, timeline for improvement, and consequences for noncompliance. Faculty should note and address systems problems that unintentionally reinforce and thus enable unprofessional behavior. Complaints, particularly by new residents, should be investigated and addressed promptly through a process that is transparent, fair, and reasonable. The importance of early intervention is emphasized.
A problem resident (PR) is “one who fails to meet the standard of performance in one or more Accreditation Council for Graduate Medical Education (ACGME) competency.”1(p201) Such residents consume substantial program director (PD) and staff time, adversely affect patient care, and disrupt team function.2,3 Deficits may exist in all core competencies; this article focuses on the resident with behavioral problems. Quiz Ref IDThis is defined by the American Medical Association3 as “personal conduct, whether verbal or physical, that negatively affects or potentially affects patient care including conduct that interferes with one's ability to work with members of a health care team” and by The Joint Commission as behavior that undermines a culture of safety.4 This behavior includes verbal outbursts, physical threats, refusing to perform tasks, or exhibiting an uncooperative attitude. Institutional leaders are required to have policies that address such behaviors whether caused by impairment due to substance abuse or other psychiatric disorder, external life stressors, personality characteristics, lack of training, or system factors.
Although disruptive physicians consume considerable attention, 50% of the concerns are associated with only 9% to 14% of physicians.5 This minority is responsible for 50% of malpractice claims costs.6 Preliminary results on a 360° instrument assessing the core competencies suggest that disruptive physicians may not differ significantly from other physicians in mean performance but have an increased frequency of low ratings (the tail of the distribution is skewed).7 In a single-institution study,8 25% of residents, who graduated on time and passed the American Board of Surgery examinations on the first attempt, received marginal performance evaluations. Nationally, the cumulative risks of termination are 3.0% and 19.5%, respectively, for all surgical residents.9 Although many “voluntary” resignations may not be entirely voluntary, there is still some discrepancy between the 3% who are terminated and the reported prevalence of PRs. Some of these residents might be successfully remediated; however, it is likely that many graduate without correction.
Identifying PRs is challenged by poorly defined standards for achievement of the ACGME competencies.10 Furthermore, residents are employees who provide an essential service; therefore, faculty are reluctant to increase the workload on colleagues by removing a resident from clinical duties. In addition, deficits in one domain may be overlooked because of excellence in another. For example, lapses in professionalism might be tolerated in the surgical resident who is well liked and has excellent technical skills. Barriers to reporting a PR include lack of knowledge as to what should be documented, concern about an appeal process, lack of remediation options, and a sense that nothing will change after reporting the behavior. Additional impediments include time restrictions, infrequent contact with the resident, and unfamiliarity with program expectations.11 Finally, faculty have concerns about losing their role as resident advocate, how these negative evaluations will be used, and retaliation from the resident.2 Therefore, the evaluation information available to the PD is often scanty or contradictory.
Although these behaviors are prevalent nationally, individual PDs will encounter PRs less frequently and can experience conflict between their roles as resident advocate and disciplinarian. In a 2009 literature review,12 only 2 studies13,14 addressing resident remedial interventions were identified. Both addressed knowledge deficits. Subsequently, 2 remediation programs have been described: 1 for radiology residents incorporating an educational agreement with the resident, PD, and a faculty educational liaison15 and another, more detailed approach for emergency medicine residents.16 Torbeck and Canal17 found that surgery PDs experience greater difficulty with remediating professionalism, interpersonal communication, and practice-based learning and improvement (PBLI) compared with other ACGME competencies. To address this need, the Southern Illinois University School of Medicine Department of Surgery, in collaboration with the American College of Surgeons, sponsored a 1-day think tank to develop an evidence-based approach to the identification, prevention, and management of PRs. The invited participants included residents, medical and nursing leaders in the field of resident education, and individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians.
The 2011 meeting took place at the American College of Surgeons offices in Chicago, Illinois. Attendees read relevant literature before the meeting and participated in short didactics on institutional and legal approaches to PRs. This was followed by a brainstorming session using the nominal group technique to identify potential strategies for remediating PRs. The nominal group technique is a structured process for generating numerous ideas.18 Two of us (N.K.R. and R.G.W.) prepared 2 cases of PRs based on structured interviews conducted with surgery PDs during their research.19 Both cases had characteristics found in many PRs and were discussed in detail to elucidate common themes as a platform to making recommendations for remediation.
Common findings were that both residents had advanced degrees on admission to residency training and red flags in their application, that is, poor grades in 1 or more clerkships, a US Medical Licensing Examination score of 200 or less, and few glowing comments in letters of recommendation. Both had concerns noted in the first year of training. Paradoxically, each had a significant percentage of end-of-rotation evaluations that were better than the middle anchor “good.” In their programs, only 5% of all resident ratings are at “good” or below. Both PRs completed training on schedule. Hauer et al12 noted that remediation begins with identification of the problem; is followed by remedial education, including the opportunity for feedback and reflection; and concludes with a postintervention assessment. We used this framework to propose remediation strategies based on what is known from the literature and on the expert opinions.
The literature on early warning signs focuses predominantly on identifying knowledge deficits.8,12,13 Risk factors include applicant age older than 29 years, a need to repeat medical school courses, and average grades on transcripts.8,20 Medical schools wish to present their graduates in the best possible light; therefore, negative or neutral comments in deans' letters should be taken seriously.20,21 Transfer from another institution is also a risk factor for subsequent behavioral problems. Recommendations to improve the selection process include ensuring that each institution has a clear understanding of its culture and value system so that candidates who match are a good fit. It is also worth exploring social support, strategies for managing time and for managing failure, and collaborative skills during the applicant interview.22 The use of multiple mini-interviews, provocative Objective Standardized Clinical Examinations,23 or personality testing was also suggested but noted to be time-consuming. The meeting participants recommended getting input from residents and coordinators to detect behaviors suppressed during faculty interactions. Furthermore, providing the selection committee with clear interview and ranking criteria will ensure consistency and avoid capricious decisions. Clearly defined processes are important, but equally important is their consistent implementation. Finally, it is essential to be aware of the effect of the hidden curriculum of attending role modeling on resident behavior and to appreciate that it is unreasonable to hold residents to a higher standard of professional behavior than their faculty mentors.
Because behavioral problems are frequently identified early in training,19,24 the first 6-month review is a time for critical evaluation of new residents; indeed, there is a case for conducting quarterly reviews of new residents. Any problem arising at any time should be brought to the attention of the PD for full investigation and a documented action plan with an end point for further evaluation. Quiz Ref IDIn an internal medicine study,25 60% of PDs identified PRs through critical incident reports. In 75% of those cases, PDs first became aware of PRs through verbal complaints from faculty, and only 31% identified a PR from a written evaluation.25 Thus, all notifications, regardless of formality, are valuable in assessing the need for remediation.26,27 Another consideration is that faculty members with only occasional contact with residents tend to be more generous with their ratings; thus, these ratings need to be interpreted with caution. The differences between the detailed documentation required by the ACGME28 (Table 1) compared with the minimum legal requirement are discussed in the “Additional Legal Considerations” subsection.
An effective, user-friendly evaluation system is essential to identify resident deficiencies. Evaluation should include a review of all letters, e-mails, patient complaints,24 and incident reports, as well as input from multiple team members.26,27 Setting up a confidential hotline is another means of collecting data, but these are open to abuse through retaliatory reporting. Effective evaluation also includes setting standards to measure resident performance, providing rater training, and enforcing consequences for not completing evaluations in a timely manner. Narratives are often more useful than numeric ratings in identifying PRs.29 All evaluations should be based on direct observation, but it should be acknowledged that these will be subjective.30,31 While investigating clinical supervision practices, Kennedy et al32 identified 4 factors that affect supervisor perception of trainee trustworthiness: knowledge/skill, discernment of limitations, truthfulness, and conscientiousness. Two techniques used to assess trustworthiness included double-checking trainees' clinical findings and identifying cues from the trainees' use of language. Language cues included the structure of delivery during case presentations and the ability to anticipate needed information before it was solicited by the supervisor.32
Faculty development efforts aim to teach faculty the dimensions and elements of each competency and provide instruction on frame of reference, rating errors, and the importance of feedback.32 Because PBLI is essential for self-improvement, evaluations should note the extent to which residents take responsibility for their learning.33-35 Residents with a “growth” mind-set believe their success is based on hard work and learning,33 while those deficient in PBLI (“fixed” mind-set) attribute their success to innate ability and their failures to the actions of others. The latter PRs are a challenge to remediate. Problematic behavior can be goal directed, for example, shouting, hanging up the phone before a conversation has ended, or not responding to pages so the nurses do not call again. Such behavior can cause staff to misunderstand their role and provide services out-of-scope, ultimately having a negative effect on the quality of health care.2,36 Recognizing and addressing such behavior through a system-level response will increase the likelihood of successful remediation.
Once a problem is suspected, the resident should be provided with a notice of deficiency that defines the expected acceptable behavior, the timeline for improvement, and the consequences for noncompliance. He or she needs to understand through self-reflection that such behavior is unacceptable and detrimental to the resident, the program, and patients.27 The responsibility for behavioral change rests with the resident, but the program has an obligation to set clear expectations and supply appropriate surveillance, mentorship, and timely feedback. Quiz Ref IDThe Vanderbilt Promoting Professionalism Pyramid for managing disruptive behavior is a useful approach.27 This model focuses on 4 graduated interventions: informal conversations for single incidents, nonpunitive awareness interventions when data suggest patterns, leader-developed action plans if patterns persist, and disciplinary processes if the plans fail.27 This involves commitment from the leadership to support the processes for reviewing allegations and provide the leader training in conducting the important conversations.5 Exempted from this graduated process are egregious events consistent with The Joint Commission's Sentinel Event Alert.4
The Professional Renewal Center in Lawrence, Kansas,2,7,36 provides assessment, treatment, and professional development opportunities to trainees and practitioners for issues of professionalism, interpersonal and communication skills, disruptive behavior, and psychiatric problems. This is accomplished through a multidisciplinary assessment using standardized assessment instruments, interviews, and collateral data collection. In the course of the think tank, the center director (B.W.) emphasized that understanding the behavior as well as its contributory factors and consequences is critical to providing the appropriate remediation. The director reported that practicing physicians were more likely to be referred for substance use or disruptive behavior; residents were more likely to be referred for lack of timeliness, such as delinquency in completing administrative duties that are disruptive to health care team function. Residents had significantly more deficiencies in PBLI than practitioners. In most circumstances, residents with behavioral problems did not have personality disorders or impairment problems. Most professionals, both in training and in practice, responded to a tiered intervention approach. The meeting participants identified additional remedial interventions; these are summarized in Table 2.37
If self-reflection or remediation fails to achieve the desired result, PDs must follow through on the previously discussed consequences of probation, failure to promote, or dismissal. Failure to enforce consequences has a negative effect on the behavior and morale of all residents and the care delivery system. The best way to ensure that decisions are not arbitrary or capricious is to use a clinical competency committee. Problems are often identified in committee discussions that are not raised by individuals,15,29,38 permitting the identification of patterns of behavior when an individual saw only a single instance. Such committees serve as checks and balances, particularly in identifying the marginal resident. Roberts and Williams39 suggest that committees consider whether the resident's performance can be improved sufficiently to perform effectively as a member of the health care team and whether this improvement is likely to be sustained in practice as well as during training. Other considerations are the cost of remediation in time, effort, and resources, as well as the hidden cost of retaining a resident in terms of the increased workload on colleagues necessitated by “work-arounds,” double-checking, and low morale. Finally, it is essential to consider the effect on patient care and on a patient's perception of the health care team and institution.39 The meeting participants noted that the amount of time spent discussing a resident is frequently a measure of the severity of the problem.
Academic faculty are the gatekeepers of the profession, with a responsibility to assess resident competency for independent practice.40 If standards are not met after remediation, the faculty have a responsibility to dismiss the resident. The ACGME requires that PDs complete a final summative assessment of each resident.41 This should be balanced and include any significant weaknesses or unremedied deficiencies in the core competencies, as well as deficiencies that were successfully remediated. The program must provide copies of a final assessment, consistent with what has been communicated to the resident, to future employers on request. Finally, since an important impediment to terminating a resident is the difficulty of finding a qualified replacement, the meeting participants suggested creating an agency for preliminaries and a national ranking system for their evaluation.
The courts have unfailingly confirmed that as long as the minimal process of Horowitz28 has been met, that is, the individual was provided with “notice and an opportunity to cure and the faculty decision is conscientious and deliberate,”28(p17) courts will not second-guess the academic decision.42 Richard and Padmore43 emphasized that misconduct must be distinguished from academic deficiency. By definition, misconduct is behavior that is wrong and that one knows (or should know) is wrong and therefore will not be cured by remediation. Treating misconduct as academic deficiency could be legally precarious by holding residents to different legal and performance standards than other institutional employees.43Quiz Ref IDMisconduct includes such fatal flaws as dishonesty, inappropriate touching, patient abandonment, criminal activity, and covering up mistakes, thereby putting patients at risk. All incidents should be investigated and a report generated that considers extenuating circumstances, if present. In assessing the culpability of an individual accused of such misconduct, Hickson et al5,27 recommended using the Reason44 criteria and asking whether the team member intended to cause harm, came to work impaired, and knowingly and unreasonably increased risk and whether another team member in the same situation would act in a similar manner. In certain situations, recommending a multidisciplinary evaluation can be helpful in elucidating contributing factors and remediation potential. Final decisions about the resident are made by the PD, regardless of the majority opinion or the unpopularity of the decision. The resident may request a review and only one level of review is necessary.
The Americans With Disabilities Act45 mandates that educators must make reasonable accommodation to ensure that a resident with a disability can complete the curriculum; however, the resident must ask for accommodation before a performance deficiency occurs. Performance problems should be addressed as a performance or a behavior problem and not as a health issue. For example, stress must be discussed as it relates to poor performance—not mental health. In addition, the Americans With Disabilities Act limits when a psychiatric evaluation can be required and is usually restricted to decisions about fitness to practice. A physician-patient relationship does not, and should not, exist between a resident and PD; therefore, if there is a fitness to practice concern necessitating a referral to employee health, this must remain confidential and separate from the academic file. After such evaluations, the PD should receive notification that appropriate follow-up is occurring but should not receive medical details. Future employers should not be told about impairment except to the extent that it involved misconduct (or lack of fitness for practice) that resulted in employment action.
There is a distinction between the ACGME requirements for documentation and what could be considered acceptable in a court of law (Table 1). Verbal feedback to the resident is legally acceptable but, although “no documentation may be enough, more is always better.”46(p642) If faculty members are unwilling to put a concern in writing, it can be noted by the recipient of the complaint or through the competency committee meeting minutes. Legally, evaluations of performance will be protected if they are good faith exercises of professional judgment without malicious intent.
Factors known to put individuals at risk for behavioral problems should be considered when ranking applicants. It is essential to set clear expectations for professional behavior with both faculty and residents and to describe problem behaviors as a deficiency in one or more competencies. Program directors should incorporate an assessment of trustworthiness and ability to take responsibility for personal behavior into resident evaluations and note system problems that enable unprofessional behavior by providing secondary gain for such activities. Any complaint or critical incident, particularly in a new resident, should be investigated and addressed promptly. Quiz Ref IDOnce a problem has been identified, the resident must be provided with a notice of deficiency and an opportunity to improve, with consequences for failing to address the deficiency. In addition to participation in a remedial program, the opportunity for feedback and reflection and postintervention assessment are necessary to determine next steps. While the responsibility for improvement rests with the resident, the resident will need guidance in locating appropriate resources. Whatever final decision is made about the resident, as long as the process is fair and reasonable, that is, the decision was not arbitrary or capricious, it will be upheld in court. Legally, verbal concerns are as useful as written concerns, and these can be documented by the PD if the complainant is unwilling to do so. Finally, legal proceedings and grievance hearings are costly and time-consuming, so prevention is better than a cure. Therefore, the importance of intervening early is emphasized.
In conclusion, we have made a number of recommendations for ensuring professional accountability in surgical trainees. The next step will be to conduct a series of workshops with PDs to promote a uniform evidence-based approach at a national level to the PRs.
Correspondence: Hilary Sanfey, MB, BCh, MHPE, Department of Surgery, Southern Illinois University School of Medicine, Room D308, PO Box 19638, Springfield, IL 62794 (firstname.lastname@example.org).
Accepted for Publication: March 13, 2012.
Author Contributions: Dr Sanfey had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Sanfey, DaRosa, Klingensmith, Hebert, Richard, Roberts, Schwind, R. G. Williams, Sachdeva, and Dunnington. Acquisition of data: Sanfey, B. Williams, Boehler, Klamen, Mellinger, Hebert, Schwind, R. G. Williams, and Sachdeva. Analysis and interpretation of data: Sanfey, Hickson, B. Williams, Sudan, Mellinger, Hebert, R. G. Williams, Sachdeva, and Dunnington. Drafting of the manuscript: Sanfey, DaRosa, and Hebert. Critical revision of the manuscript for important intellectual content: Sanfey, DaRosa, Hickson, B. Williams, Sudan, Boehler, Klingensmith, Klamen, Mellinger, Hebert, Richard, Roberts, Schwind, R. G. Williams, Sachdeva, and Dunnington. Statistical analysis: B. Williams. Administrative, technical, and material support: Sanfey, DaRosa, Boehler, Hebert, Richard, Schwind, and Sachdeva. Study supervision: Sanfey, Klamen, Hebert, Richard, and Dunnington.
Financial Disclosure: None reported.
Additional Contributions: We acknowledge individuals who attended the meeting in Chicago and contributed to our discussion. The following were from the American College of Surgeons: Patrice Gabler Blair, MPH, Associate Director, Division of Education, and Kim Echert, BA, C-TAGME, Senior Manager, Programs to Enhance Resident and Medical Student Education. The following residents participated: Brian George, MD, Department of Surgery, Massachusetts General Hospital, Surgical Education Fellow, Northwestern University Feinberg School of Medicine; and Amanda Hayman, MD, and Melissa Pilewskie, MD, Department of Surgery, Northwestern University Feinberg School of Medicine.