A, The EQ-i generates a total score for each individual and 5 composite scale scores. B, The EQ-i also generates 15 subscales of emotional intelligence. The mean score for each composite scale or subscale is 100, a score of 115 or higher is 1 SD above the mean, and a score of 85 or below is 1 SD below the mean.
The department’s Press Ganey Patient Satisfaction Survey scores 3 years before and 6 years after EI training. Patient satisfaction data were missing in 2009 (see explanation in text).
Dugan JW, Weatherly RA, Girod DA, Barber CE, Tsue TT. A Longitudinal Study of Emotional Intelligence Training for Otolaryngology Residents and Faculty. JAMA Otolaryngol Head Neck Surg. 2014;140(8):720-726. doi:10.1001/jamaoto.2014.1169
Emotions underlie and influence physician communications and relationships with patients and colleagues. Training programs to enhance emotional attunement, or emotional intelligence (EI), for physicians and assess training effects are scarce.
To assess whether an EI training program for otolaryngology residents and faculty affects patient satisfaction.
Design, Setting, and Participants
Prospective longitudinal, cohort study of physician residents and faculty in an EI training program at the Department of Otolaryngology, University of Kansas Medical Center, with annual training from 2005 to 2011.
Three levels of interventions included 4 years of repeated EI assessment, 7 years of highly interactive EI training with high-risk/high-stress simulations, and ongoing modeling and mentoring of EI skills by faculty.
Main Outcomes and Measures
Four levels of outcome of the EI training were assessed with the following questions: Did participants enjoy the program? Could they apply the training to their practice? Did it change their behavior? Did it affect patient satisfaction? The Emotional Quotient Inventory (EQ-i) was administered to faculty and residents, and the Press Ganey Patient Satisfaction Survey was completed by patients.
Ninety-seven percent of participants (103 of 106) reported that they enjoyed the programs, and 98% (104 of 106) reported that they have or could have applied what they learned. Participants demonstrated improvement in mean EQ-i scores from 102.19 (baseline/pretraining) to 107.29 (posttraining and assessment 1 year later; change, 6.71; 95% CI, 3.44-9.98). This increase was sustained in successive years, and these results were supported with linear growth curve analysis. The total department mean EQ-i score in pretraining year 2005 was 104.29 (“average” range), with posttraining scores in the “high average” range (112.46 in 2006, 111.67 in 2007, and 113.15 in 2008). An increase in EQ-i scores and EI training corresponded with an increase in patient satisfaction scores. Percentile rank patient satisfaction scores before EI training ranged from 85% to 90%; after training, scores ranged from 92% to 99%.
Conclusions and Relevance
Emotional intelligence training positively influences patient satisfaction and may enhance medical education and health care outcome.
Prompted by the Institute of Medicine report, To Err Is Human,1 the safety initiatives of the Institute for Healthcare Improvement,2 and the challenges posed by the Accreditation Council for Graduate Medical Education3 to assess4 and develop 6 specific competencies for residency education,5 we designed a training program for residents and faculty to enhance emotional intelligence (EI). Emotional intelligence, broadly defined, is a set of skills that facilitates self-awareness, understanding, and management of how our emotions affect self, others, and our performance.6- 8 Advances in affective neuroscience have identified emotions as among the principal drivers of behavior,9 underpinning communication, decision making, and problem solving.10,11 For example, too much anger or not enough empathy can short-circuit the communication,12 leadership,13 and teamwork14 exchanges that are frequently cited as the root cause of medical errors15 and performance deficits.16
We hypothesized that a multilevel EI training program that included repeated EI assessment, simulations of high risk/high stress, and strong faculty involvement would positively affect patient satisfaction scores. Our multilevel outcome approach attempts to address one of the major challenges of medical education research: determining “how the design and conduct of medical education programs affect the clinical outcomes produced by doctors.”17(p1067) Despite the prominent role emotions play in affecting the quality of health care18 and despite the popularity of EI training programs in business and education,19 we could not find any EI training programs in health care or in residency or faculty education described and/or evaluated longitudinally with multiple levels of outcome data. We encourage a reframing of that common adage, “rule thy feelings, lest they rule you,”20 to “attune to your feelings, lest they undermine you.”
To assess the EI training program, we gathered 4 levels of outcome data and used a training evaluation method, the Kirkpatrick method,21 which has been described as “one of the most popular,22 influential,”23 and the gold standard for evaluating programs.24 The 4 levels were assessed with the following questions: Did participants view the training as enjoyable or positive (level 1)? Could they apply the training to their practice (level 2)? Did the training change participants’ behavior (level 3)? Did the training correspond with an increase in patient satisfaction (level 4)? Despite the popularity of this 4-level assessment model in the training literature, we could not find application of its use in resident and physician training.
In early January of each year from 2005 to 2008, each resident and faculty member in the Department of Otolaryngology and Head & Neck Surgery was asked to complete the 133-item Emotional Quotient Inventory (EQ-i).25 Only residents were asked to complete the EQ-i in 2009. Participation was voluntary, and 100% of the residents completed the test. A total of 28 residents completed the EQ-i between 2005 and 2009; 22 of them (79%) completed the EQ-i at least twice. Of those with 2 or more waves of data, 10 residents completed the EQ-i for the first time in 2005 (the first year of administration), with 2 joining in 2006, 4 in 2007, and 3 each in 2008 and 2009. In addition, 19 faculty members participated, with data from 2 or more waves in 17 (89%). Of these 17 faculty members, 13 joined the study in 2005, 1 in 2006, and 3 in 2007. Each year, about 20 days after residents and faculty completed the EQ-i, the department sponsored an 8-hour off-site EI training program. All residents, except those in the third year, participated in the training program; third-year residents remained on site and assisted with patient coverage. Each year, the training program was designed and facilitated by all 3 members of the department leadership team, the chair (D.A.G.), the program director (T.T.T. in 2005-2009 and R.A.W. in 2010-2011), and the associate program director (R.A.W. in 2005-2009), along with a psychologist trained in EI (J.W.D.). Faculty participation in the off-site EI training program was as follows: 2 faculty members attended all 7 years, 1 attended 5 years, 2 attended 2 years, and 9 of the 17 attended 1 year.
Written consents were obtained, and the study was approved by the institutional review board of the University of Kansas Medical Center.
To evaluate the first 2 outcome levels of our EI training program pertaining to whether the participants enjoyed the program and could apply what they learned, we used 2 items from the training evaluation measure that had been developed and used by our university’s Department of Continuing Education (form available from the University of Kansas Medical Center Continuing Education Program). The wording of the item used to assess the level 1 outcome was “The overall quality of the program was excellent.” We concluded that participants rating a program as excellent was tantamount to enjoyment. The wording of the level 2 outcome item was “I could have used the information at this course in my practice recently.” Both items are scored on a 5-point Likert scale: strongly agree, agree, disagree, strongly disagree, or not applicable.
To assess the third outcome level, EI skills, we selected the EQ-i. The EQ-i has strong internal consistency (α = .69-.86) and test-retest reliability (1-month r = 0.85 and 4-month r = 0.75).25 The EQ-i is used widely (>1 million administrations), is administered and interpreted easily, facilitates benchmarking, generates individual and group reports, and provides 10 to 12 pages of feedback on enhancing EI skills. The EQ-i contains a total EQ-i score for each individual, 5 composite scales, and 15 content subscales of EI. Sample items from 3 of the subscales are listed in the Box. Sample EQ-i results are shown in Figure 1. The mean score for each scale is 100; a score of 115 or higher is 1 SD above the mean, and a score of 85 or lower is 1 SD below the mean. The group score (mean of all individuals’ total EQ-i scores) reflects the level of EI skills in that group or department.
1. Emotional self-awareness
I’m aware of the way I feel.
It is hard for me to describe my feelings.
I’m good at understanding the way other people feel.
My friends can tell me intimate things about themselves.
3. Impulse control
When I start talking it is hard to stop.
I’ve got a bad temper.
The respondent marks as follows for each item:
1. Very seldom or not true of me
2. Seldom true of me
3. Sometimes true of me
4. Often true of me
5. Very often true of me or true of me
Sample items from the EQ-i subscales for emotional self-awareness, empathy, and impulse control (reproduced with permission from Multi-Health Systems Inc).
To assess the third level of training regarding whether participants changed their behavior, we compared changes in the residents’ and faculty members’ individual and group EQ-i scores between the baseline year (first year they took the test) and successive years. An increase in the resident and faculty individual and total EQ-i scores from the first administration (preintervention) to successive administrations suggests behavioral change.
To evaluate the fourth level of our program, from December 2005 to December 2011 we tracked whether our EI interventions positively affected patient satisfaction as assessed by the Press Ganey Patient Satisfaction Survey (PGPSS).26 This survey assesses 15 categories of patients’ satisfaction, and 1 category includes 5 items that assess the patient’s perception of satisfaction with the discharge physician. These 5 items assess (1) how much time the physician spent with the patient, (2) whether the patient’s questions or worries were addressed, (3) how well the physician kept the patient informed, (4) the physician’s friendliness or courtesy, and (5) the physician’s skill. At the time of this study, the PGPSS was used in 361 teaching hospitals (A. Holland, Press Ganey Associates, oral communication, April 11, 2013).
The PGPSS generates a total department score on these 5 questions for a year and ranks a given department against similar departments in the other 361 teaching hospitals. Inpatient care in a university teaching hospital in a surgical subspecialty such as otolaryngology is provided by a team of physicians. The attending or discharge otolaryngologist performs the surgery and leads and supervises the patient care in collaboration with the otolaryngology residents. The patient satisfaction measure not only reflects the patient’s perception of his or her relationship with the attending physician but is also representative of the collaborative care provided by the attending otolaryngologist and the residents.
Our EI training program includes 3 specific interventions: assessment, training, and modeling/mentoring. The first intervention is when participants complete the EQ-i. The second is the 8-hour off-site training program that sets the stage for reflection on and development of EI skills. The third is the faculty’s modeling and mentoring of EI skills in the thousands of hours they work with residents in the clinic, at bedside, and in the operating room.
At the off-site training program, the individual test results are distributed; individuals compare their results with the total department group’s score and benchmark their individual results to star performers in other fields. The faculty/leaders describe the added value of EI skills, and examples of emotional disconnect in medicine, sports, and businesses are highlighted. A key ingredient of this second intervention is that the residents and faculty view a video or participate in high-risk/high-stress simulations (eg, giving bad news, talking to a family about a medical error, responding to a leader who is perceived as autocratic, or dealing with a blocked airway in the operating room where everything goes wrong). (Curriculum for the second level can be obtained by contacting R.A.W.)
In the discussion during the simulations and in the debriefing after them, residents are encouraged to think out loud and reflect on their comments, and faculty are encouraged to offer suggestions and share tacit knowledge about managing emotions in difficult situations. The highly interactive style of the debriefings fosters a collaborative culture and supportive learning environment where feedback and practice of EI skills are encouraged. Integral to this developing EI culture is the strong and ongoing participation of faculty, which supports, facilitates, and models the positive role of EI in delivering patient care. The EQ-i was not readministered immediately after the 8 hours of EI training as is frequently done in assessing training effects; instead, in this study, the EQ-i was administered 1 year after the first administration, which allowed ample time for faculty to model and mentor residents in developing EI skills.
The first 2 levels of our training assessment—determining whether the program was liked/enjoyed and whether it had application—were assessed using dichotomous variables; we calculated the percentages of respondents who reported that they had positive perceptions of the program and could apply what they learned. For the third level, because our sample size is small and we wanted to track whether the increases in EI skills were sustained, we used linear growth curve analysis. With a small number of participants, this analysis adds power to the final analysis because it can take participant data into account regardless of the number of time points. We chose to conduct linear growth curve analysis for the 39 participants with 2 or more waves of data. Linear growth analysis encompasses the idea that “individual change or growth should be viewed not as a discrete process (eg, change from Point A to Point B) but as a continuous process (eg, as growth) that underlines development.”27 For the fourth level of the training, using descriptive data, we charted the corresponding increase in PGPSS scores after our training programs.
Of the 22 residents who completed the EQ-i at least twice, 15 were men and 7 were women, with a mean age of 29 years; of the 17 faculty, 13 were men and 4 were women, with a mean age of 43 years. This prospective, longitudinal study shows that an EI training program with multiple levels of intervention was rated positively by participants (level 1) and regarded as having application to their practice (level 2) and was associated with behavioral change, as reflected in the group EQ-i scores for residents and faculty (level 3), and with an increase in the department’s inpatient satisfaction scores (level 4).
From 2005 to 2011, for the first outcome level, 97% of participants (103 of 106) strongly agreed or agreed that they liked the programs. For the second outcome level (applicability of training) 98% (104 of 106) strongly agreed or agreed that they “used or could have used the information in my practice recently.”
For the third outcome level, Table 1 shows the mean EQ-i scores among participants before the training. Participants demonstrated improvement in total EQ-i scores from 102.19 (baseline/pretraining) to 107.29 (posttraining and assessment 1 year later; change, 6.71; 95% CI, 3.44-9.98). Linear growth curve analysis showed that the residents and faculty whose EQ-i scores increased from baseline to the first training year sustained their increase in EQ-i scores over successive years. However, both residents and faculty showed variability in how much their total EQ-i scores increased from pretraining to posttraining years. Looking specifically at those residents and faculty who completed the EQ-i during the first 4 years, 69% of residents (11 of 16) and 47% of faculty (7 of 15) showed an increase of 6 points or more in total EQ-i scores from pretraining to posttraining. Three of the 5 residents whose EQ-i scores did not increase had baseline total EQ-i scores of 111, considered in the “high average” range, and in successive years their scores remained in that range. The EQ-i scores of the other 2 residents varied.
Eight faculty members showed little variation in their EQ-i scores, which increased or decreased by no more than 3 points. The faculty in our study had a mean of 11.3 years postresidency experience. Perhaps the high-risk/high-stress simulations may not have been challenging enough to foster significant reflection on their emotions. Despite this variability, Table 2 shows that the total department EQ-i—the mean score for residents and faculty—increased from 104.29 in the baseline/pretraining year (2005), which is in the “average” range, to posttraining scores in the “high average” range: 112.46 in 2006, 111.67 in 2007, and 113.15 in 2008.
The fourth level of assessment investigated whether an increase in the department’s total EQ-i score and the EI training program corresponded with an increase in patient satisfaction scores. Figure 2 shows the department’s percentile ranking for 3 years before EI training (89% in 2002, 90% in 2003, and 85% in 2004) and for 6 of the 7 years after EI training (96% in 2005, 97% in 2006, 99% in 2007, 95% in 2008, data not available in 2009, 92% in 2010, and 97% in 2011). In 2009, hospital changes in data collection methods resulted in inadvertent aggregation of patient satisfaction measures for multiple departments, and department-specific data were not available. This was corrected for 2010 and 2011.
Overall, the EI training program corresponded with an increase in the patient satisfaction measure as assessed by the PGPSS. An increase in patient satisfaction has been associated with increases in patient adherence,28 volume,29 revenue,30 and employee morale31 and reduction in malpractice claims.12,32
This study describes an EI training program for a surgical specialty, otolaryngology, and provides 4 levels of outcome data to assess this training approach. A comparison between the pretraining and posttraining periods showed that a faculty-led and mentored interactive training program using high-risk/high-stress simulations enhanced participants’ recognition, understanding, and management of emotions; participants enjoyed the training, found applications to their practice, and changed their behavior, and these changes were reflected in increased patient satisfaction with their physicians.
This study has several limitations. First, our data were gathered in a single-specialty department with a small number of residents and faculty. Even though data were gathered 2 or more times for some participants, the small cohort may limit the generalizability of the results. However, our small sample of residents and faculty is representative of the number of residents and faculty in otolaryngology programs nationally, and most residency specialty training programs are similar in size.33 Our EI training approach may serve as a blueprint for other specialty residency training programs with an interest in enhancing patient satisfaction through such a training program for residents and faculty.
Our training program was effective in increasing the EQ-i of the majority of residents and almost half the faculty members. However, for residents and faculty members whose EQ-i scores did not increase, perhaps we should have offered some individual coaching by colleagues with strong EQ-i scores.34
The largest increase in EQ-i scores for faculty and residents occurred from the first time they took the EQ-i (baseline) to the second time they took it 1 year later. This outcome suggests that faculty and residents collectively were quick to adopt EI training, which was reflected by the active participation of faculty and residents in the high-risk/high-stress simulations. Perhaps a 1-year 8-hour EI training program with EQ-i assessment and strong faculty participation, supplemented yearly by brief (1-2 hours) EI booster sessions, may be enough to improve patient satisfaction scores.
There was also variation in the number of faculty members who completed and received written feedback on the EQ-i and attended the off-site training program. Logistics, patient care, and personal demands limited 100% faculty participation in an off-site training program. However, a critical mass of department leaders and faculty participated, creating internal champions for EI who fostered and sustained an EI culture in which reflection, understanding, and management of emotions were considered “the way we do things around here.” Leaders positively influence the transfer of training,35 and their supporting and mentoring of trainees have “resulted in participants exhibiting 42%-52% more behaviors consistent with the trained skills than those with non-supportive leaders.”36(p289)
A multilevel EI training program positively affects patient outcomes as assessed by a patient satisfaction measure. Although the use of the EQ-i assessment heightens awareness of skills and the simulations provide feedback and practice of skills, the active involvement of department leaders and faculty was invaluable in increasing EI skills. Training36,37 and research literature38- 40 abound with the added value of leaders and teachers in motivating learners and the importance of supporting, modeling, and mentoring skills. An EI training program without active involvement of leaders and faculty may be a waste of training time and dollars. As far as we know, our use of Kirkpatrick’s 4 levels of outcome was the first time it was applied to a resident-faculty training program. We encourage other programs to adopt this 4-level approach and invite comparisons to our results. Increasing EI skills may be part of a rising tide that contributes not only to patient satisfaction but also to improvement in interprofessional cooperation, in turn strengthening sign-outs, handoffs, patient-centered care, and conflict reduction in high-risk/high-stress situations.
Submitted for Publication: February 27, 2014; final revision received May 14, 2014; accepted May 18, 2014.
Corresponding Author: James W. Dugan, PhD, Department of Student Affairs, Kansas City University of Medicine and Biosciences, 1750 Independence Ave, Kansas City, MO 64106-1453 (firstname.lastname@example.org).
Published Online: July 10, 2014. doi:10.1001/jamaoto.2014.1169.
Author Contributions: Drs Dugan and Barber had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Dugan, Weatherly, Girod, Tsue.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Dugan, Barber.
Critical revision of the manuscript for important intellectual content: Weatherly, Girod, Barber, Tsue.
Statistical analysis: Dugan, Weatherly, Barber.
Administrative, technical, or material support: Girod, Tsue.
Supervision: Weatherly, Girod, Tsue.
Conflict of Interest Disclosures: None reported.
Funding/Support: This project was funded in part by the Department of Otolaryngology and Head & Neck Surgery at the University of Kansas Medical Center.
Role of the Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We acknowledge the assistance of Dominick Scalise, PhD, in data entry and preliminary analysis. During part of this study, Dr Scalise was a student at the University of Missouri, Kansas City, and was employed as a research specialist in the Department of Otolaryngology and Head & Neck Surgery. We also owe a great deal of appreciation to the faculty and residents in the department who gave of their time and actively participated in this project.