Effect of a Pediatric Hospitalist System on Housestaff Education and Experience

Objective  To determine the effect of a pediatric hospitalist system on housestaff education and experience.

Setting  Free-standing academic children's hospital. Hospitalists were introduced in June 24, 1998, to supervise teaching and care on the general pediatric wards.

Methods  On 2 surveys, housestaff rated their skills, knowledge, and experiences on the wards (experiences survey), and the quality of teaching and supervision by attending physicians (attending survey). Responses before and after the introduction of the hospitalist system were compared using Wilcoxon nonparametric tests.

Results  Seventy-six (70%) of the 109 interns and 54 (62%) of the 87 senior residents responded to the experiences survey. Following introduction of the hospitalist system, the interns' mean overall rating of the general pediatrics wards rose from 4.1 to 4.7 on a 5-point Likert scale (P = .01). Their ratings of comparison rotations did not change significantly. Interns' satisfaction with the educational experience (3.2 to 3.5 of a 4-point Likert scale, P<.05), supervision, and quality of life on the pediatrics wards also improved significantly, as did their self-assessments of skills and knowledge related to general pediatrics training. Senior residents' ratings were generally unchanged. Three hundred seventy-one (63%) of the 593 attending physicians' surveys were completed. Compared with nonhospitalist attendings, hospitalists were rated more effective role models (4.7 vs 4.5 points, P<.05) and teachers (4.7 vs 4.4 points, P<.01). They were rated more knowledgeable (4.8 vs 4.5 of 5, P<.001) and accessible (4.7 vs 4.5 points, P<.05), involved housestaff more in the learning process (4.7 vs 4.4 points, P<.05), and gave better feedback (4.5 vs 4.2 points, P<.01). Hospitalists' bedside teaching, however, was rated lower (3.7 vs 4.2 points, P<.001).

Conclusions  Overall, hospitalists were rated significantly higher as educators than were traditional attending physicians. Introduction of a hospitalist system was associated with improved intern experience and quality of life on general pediatrics wards, and with improved self-reported skills and knowledge in general pediatrics.

HOSPITALISTS, PHYSICIANS spending at least 25% of their time providing care for hospitalized patients referred by their primary care physicians, are increasingly prevalent in the United States.¹ In a national survey of pediatric department chairs, we found that 50% of academic pediatric centers have begun to use hospitalists, and more than three quarters are planning to use them in the future.² Most studies have found that pediatric and adult hospitalist systems decrease inpatient costs and length of stay without adversely affecting mortality or readmission rates.^3-10 The effect of hospitalist systems on trainees' experience is less well documented.

Proponents of hospitalist models suggest that hospitalists' inpatient expertise will lead to improvements in inpatient teaching and supervision.¹¹ Several preliminary studies support this assertion, demonstrating improved housestaff satisfaction or ratings of overall experience on hospitalist services,^3,6,12 but others have found no such improvement.^13,14

Very few studies have attempted to critically compare the training process in hospitalist systems with traditional systems, or to differentiate the effects of hospitalist systems on senior residents and interns. Uncontrolled surveys have found hospitalists' teaching rated highly in a variety of specific domains,^15,16 but comparative data are lacking. The most detailed, controlled evaluation was published by Wachter et al³ at the University of California, San Francisco; they compared residents' learning on hospitalist-staffed services with that on traditionally staffed services.³ Residents reported learning more about cost-effectiveness on the hospitalist service; no differences were found in resident-reported learning of pathophysiology, practice guidelines, evidence-based medicine, or preparation for practice.

While these studies have provided insights into education in hospitalist systems, they have not thoroughly explored the influence of hospitalists on overall resident education.¹⁷ Very few data on the strengths and weaknesses of hospitalists as educators have been provided. Studies of housestaff education and experience in pediatric hospitalist systems have been particularly limited.

We hypothesized that the introduction of hospitalists in a large free-standing pediatric teaching hospital would improve (1) housestaff ratings of their general pediatrics rotations and attending physicians, (2) housestaff skills and knowledge in inpatient general pediatrics, and (3) housestaff knowledge and use of evidence-based medicine. To test these hypotheses, we compared housestaff ratings of their own experiences, skills, and knowledge, as well as the quality of education provided by attending physicians, before and after introduction of a hospitalist system.

On June 24, 1998, the study hospital reorganized its medical services in an effort to improve the quality of patient care and to improve resident and medical student education. Changes occurred throughout the medical center, including the general pediatrics wards. Eight hospitalists were hired as salaried employes to supervise the senior resident (postgraduate year 3 [PGY3]), 4 interns (postgraduate year 1 [PGY1]), and 3 to 4 medical students on each general pediatrics ward team. This hospitalist system replaced a traditional system in which approximately 75 general pediatricians and subspecialists volunteered 2 weeks to 1 month each year to teach and care for hospitalized patients on 3 general pediatrics teams. Most of these attending physicians did not go to residents' daily work rounds, and most were available in-house for only part of the day, although they were available by page.

Six of the hospitalists were drawn from the Division of General Pediatrics and the Division of Emergency Medicine at the study hospital; the remaining 2 came from subspecialties within the Department of Medicine. Each served as a hospitalist for 3 to 4 months per year. While on service, hospitalists were the attendings-of-record for most general pediatric inpatients, and were present at work and teaching rounds every day. They were available for clinical care, supervision, and teaching throughout the day. At night, they were available by telephone. Responsibilities for didactic teaching of housestaff and students were shared with a "teaching attending," a subspecialist or community pediatrician volunteering to help with teaching for 2 weeks to 1 month per year.

Two anonymous surveys were used to address housestaff experiences in inpatient general pediatrics. The first was an annual survey in which housestaff rated their rotations and their skills and knowledge (experiences survey). The second was a survey completed at the end of each general pediatrics ward rotation in which housestaff rated the teaching and supervisory skills of attending physicians (attending survey).

At the end of each academic year, the study hospital administers a detailed survey asking its pediatric housestaff to rate their skills, knowledge, and educational opportunities; feedback, advising, and support provided; conferences; rotations; quality of life; and overall impressions of the program.

Prior to the 1997-1998 survey administration, we added new items to specifically assess housestaff experience of the quality of life, support, supervision, and autonomy on the general pediatrics wards. The residency program directors and representatives of the housestaff participated in the process of generating the new survey questions. The redesigned survey was administered both before the introduction of hospitalists in April 1998 and afterward in April 1999 and April 2000. From the new items specific to the general pediatrics teams, and from items on the survey assessing residents' overall skills and knowledge, we selected 13 items for analysis a priori that we believed might be affected by the presence of hospitalists. For comparison, we also examined overall ratings of other rotations on which housestaff spent time both before and after the reorganization in 1998, to allow detection of overall trends in housestaff experience across the program as a whole. Each item was rated on a 4- or 5-point Likert scale, higher being better.

To assess the ratings of hospitalists as educators, we analyzed data from an instrument filled out at the end of each rotation. In it, each house officer rotating through general pediatrics provides an evaluation of each attending physician's teaching and supervisory skills. Because housestaff rotate through the service more than once, and because there are usually 2 attending physicians on service at a time, the number of possible evaluations completed was greater than the number of residents in the program. In 1997-1998, evaluations were completed on paper. In 1998-1999, a Web site–based survey was piloted that replaced the paper instrument, but it was completed by very few residents during this pilot phase; because of the very low numbers of responses, we excluded 1998-1999 from the primary analysis. Throughout 1999-2000 residents completed the Web site–based survey.

This Web site–based survey was designed using Microsoft's Active Server Pages (Microsoft Corp, Seattle, Wash) and was deployed on a Microsoft Windows NT 4.0 Server computer, with collected data stored automatically to a Microsoft Access database. The survey is only available for access within the network firewall of the hospital. Access to the online survey is limited to active housestaff only. Program directors and faculty in charge of each clinical rotation have password-protected, real-time, online access to aggregate survey results for the rotations they supervise.

To maintain confidentiality, attending physician identities were removed from the data, but faculty evaluated in 1999-2000 were identified as hospitalists or teaching attendings. Similarly, an identifier was left in place on the 1998 data to allow identification of all attending physicians who subsequently became hospitalists. All 8 items present on both the 1997-1998 survey and the 1999-2000 survey were analyzed. Each was rated using a 5-point Likert scale (poor to excellent).

Responses of PGY1s and PGY3s were analyzed separately for the experiences survey. A small number of combined medicine-pediatrics housestaff were excluded from primary analysis, as we felt that their unique experience made grouping them with other pediatric housestaff inappropriate. Ratings by PGY1s and PGY3 in 1997-1998 (before hospitalist system) were compared with ratings in 1998-1999 and 1999-2000 (after hospitalist system) using nonparametric Wilcoxon rank sum tests. All results were confirmed in subsidiary analyses by dichotomizing all survey items into best possible response vs all others, and reanalyzing using Fisher exact tests.

On the attending survey, residents' ratings of hospitalists in 1999-2000 were compared with ratings of attending physicians in 1997-1998. We also looked at ratings of the 1999-2000 teaching attendings as a concurrent comparison group. The PGY1 and PGY3 responses were combined for attending survey analyses because respondents' training levels were not identified on this survey. Wilcoxon rank sum tests were used to compare each item on the attending survey. As with the experiences survey, all results were validated by dichotomizing into best possible response vs all others and reanalyzing using Fisher exact tests.

Seventy-six (70%) of the 109 PGY1s and 54 (62%) of the 87 PGY3s responded to the annual experiences survey. Respondents in 1998, 1999, and 2000 did not differ significantly by age or sex. Three hundred seventy-one (63%) of the 593 attending physicians' surveys were completed.

Following introduction of the hospitalist system and using a 5-point Likert scale, interns' mean rating of the contribution of the general pediatrics inpatient wards to their development of knowledge and skills rose from 4.1 points (of 5) in 1997-1998 to 4.7 points in 1999-2000 (P = .01) (Figure 1). Scoring 4.7 of 5 points, inpatient general pediatrics was the rotation most highly rated by interns in 1999-2000, up from rank 6 of 12 before the introduction of hospitalists. Ratings of none of the 11 comparison rotations changed significantly between 1997-1998 and 1999-2000 (data not shown).

Both before and after institution of the hospitalist system, senior residents rated inpatient general pediatrics very highly (mean of 4.6 points in 1997-1998 and 1999-2000) (Figure 1). At 4.6 points, it was the second most highly rated of the 11 senior resident rotations in 1999-2000 and the highest-ranked rotation of 1997-1998. Ratings of none of the 10 comparison rotations changed significantly between 1997-1998 and 1999-2000 (data not shown).

In addition to rating the general pediatrics rotation higher overall, interns rated almost every aspect of their educational experience, supervision, and quality of life on the general pediatrics wards higher on 4-point Likert scales following introduction of the hospitalist system (Figure 2). From 1997-1998 to 1999-2000, ratings of overall educational experience (3.2 vs 3.5 points, P<.05), degree of autonomy in decision making (2.5 vs 3.3 points, P<.001), amount of bedside teaching (1.7 vs 2.2 points, P<.01), workload while on duty (2.2 vs 3.4 points, P<.001), level of physical stress (2.0 vs 2.9 points, P<.01), and level of psychological stress (1.9 vs 3.0 points, P<.001) all improved substantially, following more modest gains from 1997-1998 to 1998-1999. No items were rated lower.

Similarly, interns rated their opportunities to gain skills and knowledge higher on 5-point Likert scales after introduction of the hospitalist system (Figure 3). While these items reflected experience in the program as a whole rather than experience on the general pediatrics wards alone, we believed a priori that responses to these items might be affected by the introduction of the hospitalists. From 1997-1998 to 1999-2000, ratings of generating a differential diagnosis (4.1 vs 4.6 points, P<.05), ability to make decisions independently (3.1 vs 4.1 points, P<.001), supervisory skills (2.7 vs 3.4 points, P<.05), and evaluating evidence from the medical literature (2.5 vs 3.6 points, P<.001) all improved. No items were rated lower.

Senior residents' experience was largely unchanged (Figure 4). On 4-point Likert scales, overall educational experience, supervision by faculty, and amount of bedside teaching were rated much the same after the hospitalist system as before. From 1997-1998 to 1999-2000 increases in satisfaction with workload (2.7 vs 3.2 points, P = .13), level of physical stress (2.1 vs 2.6 points, P = .09), and level of psychological stress (2.1 vs 2.7 points, P = .12) were not statistically significant. Differences in ratings of autonomy were not significant (3.4 vs 2.9 points, P = .12).

Senior residents' ratings on 5-point Likert scales of their opportunities to gain skills and knowledge were similarly unchanged overall (Figure 5). Though we found differences in ratings of ability to make decisions independently (4.1 vs 3.5 points, P = .07) and supervisory skills (4.2 vs 3.7 points, P = .07), none were statistically significant.

Housestaff generally rated hospitalists' teaching and supervisory skills higher than those of traditional attending physicians (Figure 6). In particular, hospitalists' overall teaching effectiveness (4.7 vs 4.4 points, P<.01), effectiveness as role models (4.7 vs 4.5 points, P<.05), feedback (4.5 vs 4.2 points, P<.01), involvement of housestaff in the learning process (4.7 vs 4.4 points, P<.05), accessibility (4.7 vs 4.5 points, P<.05), and knowledge base (4.8 vs 4.5 points, P<.001) were rated higher on 5-point Likert scales than those of nonhospitalist attending physicians in 1997-1998. Hospitalists' bedside teaching, however, was rated lower (3.7 vs 4.2 points, P<.001).

Ratings of 1999-2000 teaching attendings, by contrast, were not higher than those of the 1997-1998 attending physicians. Their accessibility (4.1 vs 4.5 points, P<.01) and bedside teaching (3.9 vs 4.2 points, P<.05) were rated lower than their historical counterparts; otherwise, they were rated similarly.

We found that interns rated their knowledge, skills, and experience in inpatient general pediatrics higher after the introduction of hospitalists. Senior residents' ratings, already quite high, were unchanged. We also found that hospitalists were more likely than traditional attending physicians to receive high ratings as role models and educators.

Prior work has suggested that time spent teaching improves the likelihood that housestaff will identify attending physicians as excellent role models. Wright et al¹⁸ found that attending physicians spending more than 25% of their time teaching and those spending more than 25 hours per week teaching and conducting rounds when serving as an attending physician were substantially more likely to be named excellent role models by housestaff. Our study similarly finds an association between time on service and ratings by housestaff, though 2 interpretations are possible.

It is possible that the high ratings of hospitalists were a function of their extensive time on the wards. It is also possible, however, that hospitalists received high ratings because they were already excellent teachers at the time they began working as hospitalists. Secondary analyses comparing the 1999-2000 hospitalists with the small subset of 1997-1998 physicians who subsequently became hospitalists suggest that both explanations are partially correct. Although sample sizes were small, mean ratings of future hospitalists in 1997-1998 fell between the mean ratings of other 1997-1998 physicians and 1999-2000 hospitalists on 7 of 8 items (eg, overall teaching effectiveness 4.6 points for future hospitalists vs 4.4 points for other 1997-1998 physicians and 4.7 points for 2000 hospitalists). Regardless of whether the key factor was selecting the best teachers, increasing their experience, or increasing their availability, however, we conclude that the hospitalist program facilitated provision of an improved educational program. Moving to a hospitalist system made it possible to place outstanding teachers on the wards for several months each throughout the year.

Many secondary analyses were performed to test the validity of results. Reanalysis of all items on the experiences survey after inclusion of the few responses by combined medicine-pediatrics residents yielded no significant differences. Analysis of the pilot Web site responses to the 1998-1999 attending physicians' survey revealed that hospitalists in 1998-1999 received ratings similar to hospitalists in 1999-2000, and teaching attendings in 1998-1999 received ratings similar to teaching attendings in 1999-2000. Use of alternative statistical methods likewise yielded no important differences. On both the experiences and attending survey, dichotomization of all items into best possible response vs all others and analysis using the Fisher exact tests produced results fundamentally similar to the Wilcoxon nonparametric comparisons.

The improved ratings of opportunities to evaluate evidence from the medical literature deserve special mention. Objectives in implementing the hospitalist system included improving and standardizing the quality of care and bringing a stronger evidence base to bear on the treatment of pediatric inpatients. While improvement on a single survey item should not be overinterpreted, the improvement in interns' opportunities to use evidence-based medicine is encouraging. Substantial room for improvement remains, however. Additional efforts to provide a structured framework for the interpretation of data behind inpatient management should continue to be a goal of hospitalist programs to further improve the quality of medical education and decrease undesirable variability in care, and the success of hospitalist systems in achieving this goal should be tracked.

The Institute of Medicine recommends evidence-based medicine, shared decision making with patients, and improved communication among clinicians as key components of a health care system redesigned to improve quality.¹⁹ In particular, teamwork and open communication are important in the reduction of medical errors.²⁰ Medical educators have begun to address these issues.^21,22 However, opportunities to model, teach, and measure these skills explicitly in hospitalist systems and to redefine physician autonomy with respect to these skills deserve further investigation.

The decrease in senior residents' ratings of their autonomy and supervisory skills following introduction of the hospitalist system, although not statistically significant, raises questions about the possible influence of hospitalists on senior resident development of autonomy and supervisory skills. It is possible that this hospitalist system exchanged some degree of senior resident autonomy for improved communication and collaborative decision making between attending physicians and housestaff. The concept of autonomy deserves further investigation, and appropriate ways to teach and measure leadership and decision making while optimizing teamwork and patient care should be examined.

Bedside teaching by hospitalists is an area of concern. The quality of bedside teaching by hospitalists in 1999-2000 was rated lower than that provided by attending physicians in 1997-1998. Housestaff rated their satisfaction with the amount of bedside teaching quite low in both the traditional and hospitalist systems. General pediatrics teams at the study hospital were large in both systems (typically 11 physicians and medical students), and as a consequence, relatively little bedside teaching has typically occurred during weekday work rounds. Interventions that address this problem should be pursued.

The before and after design of this study has inherent limitations. While we have identified an association between the institution of a hospitalist system and improvements in housestaff ratings of their education and experience, it is impossible to draw direct causal links between the introduction of the hospitalist system and the changes observed. Changes in the residency program and reorganizations of the medical service (eg, addition of new outpatient subspecialty rotations and changes in subspecialty team organization) were happening concurrently, and it is possible that these cointerventions rather than the hospitalist system itself were responsible for the changes seen. However for several reasons we believe these changes are unlikely to explain our findings. First, we intentionally selected for analysis those items on the experiences survey either specific to the general pediatrics inpatient wards (Figure 2 and Figure 4) or believed a priori to be largely driven by the inpatient general pediatrics experience (Figure 3 and Figure 5). Second, we found no statistically significant changes in the overall ratings of any rotation but the general pediatrics wards following the reorganization of the medical services. If a hospitalwide factor, rather than one specific to general pediatrics, was responsible for the widespread improvements on general pediatrics, significant improvements would most likely have occurred on other rotations as well. Third, we believe that the improved evaluations of hospitalists on the attending survey are unlikely to have been driven by factors external to the hospitalist system. The lack of improvement in ratings of the concurrent comparison group (the teaching attendings) supports this assertion.

There were also changes to the interns' experience on the general pediatrics teams themselves unrelated to hospitalist care, but, again, we do not believe these changes would have led to the pattern of improvements seen. First, there was a slight decrease in the average number of months housestaff spent in inpatient general pediatrics per year, but we would have expected this change to decrease educational opportunities in inpatient general pediatrics and consequently to have lowered the ratings of educational opportunities. Second, the average census on general pediatrics increased 7% from 1998 to 2000. If changes in ratings of workload and quality of life were because of these shifts in census, we would have expected housestaff to rate workload higher and quality of life lower in 2000. Quite the contrary, interns rated their workload lower and quality of life higher.

Because we studied a limited number of housestaff classes, we were concerned that a cohort effect could be responsible for apparent changes. The interns of 1997-1998 are the senior residents of 1999-2000. If this class tended to rate all items lower than other classes, then the experience of interns would artificially appear to have improved and that of senior residents to have deteriorated from 1997-1998 to 1999-2000. This was not the case, however. Interns' mean ratings of the 13 items on the experiences survey increased an average of 0.71 points between 1997-1998 and 1999-2000, while senior residents' ratings remained essentially unchanged (5% of the change in intern ratings). While this does not rule out any cohort influence, it strongly suggests that it is not the principal determinant of the changes seen.

This study adds to the emerging literature suggesting that hospitalist systems have an overall positive effect on housestaff education and experience. By evaluating specific aspects of the teaching and learning process, we gained insights into the educational benefits of hospitalist programs and opportunities for continued improvement. The differences in the reported experiences of PGY1s and PGY3s will guide focused efforts to address the learning needs of each. Future studies should also assess the influence of hospitalists on medical student education. Beyond their effects on those aspects of education and experience that we assessed in this study, it will also be important to understand the influence of hospitalists on other aspects of trainees' experience and training, including their clinical performance, procedure competency, documentation patterns, and medical error rates.

A comprehensive assessment of a hospitalist system would include not only a measurement of its effect on trainees but also an analysis of the influence on other providers' and patients' experiences, efficiency of care, and quality of care (as measured by both outcomes and processes of care), as well as an analysis of the expected costs and the distribution of cost-benefits among the various factions within the health care system.²³ Such an analysis, however, was beyond the scope of this study.

In combined training and care systems, the pressures to maximize efficiency and quality of clinical care are unlikely to abate. The use of hospitalist systems is an increasingly widespread innovation shown to improve the efficiency of care. As hospitalist systems evolve in academic centers, it will be important to capture the new opportunities to improve physician training that these systems offer so that improvements in teaching, care, and efficiency can be made in concert.

Accepted for publication April 10, 2002.

This study was supported in part by grantT32 PE10018 from the Health Resources and Services Administration, Rockville, Md, to the Harvard Pediatric Health Services Research Fellowship Program (Dr Landrigan).

We thank Paul Lerou, MD, and Joshua Nagler, MD, for their review of the manuscript and helpful comments.

Hospitalist systems have been shown to decrease length of stay and inpatient costs for adults and children, without adversely affecting mortality or readmission rates. Their effect on housestaff experience and education, however, is not well understood, particularly in pediatrics.

We found that interns' overall rating of general pediatrics ward rotations, as well as ratings of almost all aspects of their education, experience, and quality of life on the wards, improved significantly following the introduction of a hospitalist system. Self-assessments of general pediatrics skills and knowledge also improved. Senior residents' ratings, already quite high, remained unchanged. In addition, housestaff rated the overall quality of teaching and most specific aspects of teaching and supervision higher for hospitalists than for traditional attending physicians, although bedside teaching was rated lower. These findings support the adoption of hospitalist systems by academic pediatric programs and should help guide educational improvement efforts in academic hospitalist systems.

Corresponding author and reprints: Christopher Landrigan, MD, MPH, Children's Hospital, Main 10E, Room 1032.2, 300 Longwood Ave, Boston, MA 02115 (e-mail: landrigan_c@hub.tch.harvard.edu).