Sackett DL, Straus SE, for Firm A of the Nuffield Department of Medicine . Finding and Applying Evidence During Clinical RoundsThe "Evidence Cart". JAMA. 1998;280(15):1336-1338. doi:10.1001/jama.280.15.1336
From the Nuffield Department of Clinical Medicine, University of Oxford, Oxford, England. A complete list of the members of Firm A of the Nuffield Department of Clinical Medicine appears at the end of this article.
Context.— Physicians need easy access to evidence for clinical decisions while
they care for patients but, to our knowledge, no investigators have assessed
use of evidence during rounds with house staff.
Objective.— To determine if it was feasible to find and apply evidence during clinical
rounds, using an "evidence cart" that contains multiple sources of evidence
and the means for projecting and printing them.
Design.— Descriptive feasibility study of use of evidence during 1 month (April
1997) and anonymous questionnaire (May 1997).
Setting.— General medicine inpatient service.
Patients.— Medical students, house staff, fellows, and attending consultant.
Intervention.— Evidence cart that included 2 secondary sources developed by the department
(critically appraised topics [CATs] and Redbook), Best Evidence, JAMA Rational Clinical Examination series, the Cochrane Library,
MEDLINE, a physical examination textbook, a radiology anatomy textbook, and
a Simulscope, which allows several people to listen simultaneously to the
same signs on physical examination.
Main Outcome Measures.— Number of times sources were used, type of sources searched and success
of searches, time needed to search, and whether the search affected patient
Results.— The evidence cart was used 98 times, but could not be taken on bedside
rounds because of its bulk; hard copies of several sources were taken instead.
When the evidence cart was used during team rounds and student rounds, some
sources could be accessed quickly enough (10.2-25.4 seconds) to be practical
on our service. Of 98 searches, 79 (81%) sought evidence that could affect
diagnostic and/or treatment decisions. Seventy-one (90%) of 79 searches regarding
patient management were successful, and when assessed from the perspective
of the most junior team members responsible for each patient's evaluation
and management, 37 (52%) of the 71 successful searches confirmed their current
or tentative diagnostic or treatment plans, 18 (25%) led to a new diagnostic
skill, an additional test, or a new management decision, and 16 (23%) corrected
a previous clinical skill, diagnostic test, or treatment. When the cart was
removed, the perceived need for evidence rose sharply, but a search for it
was carried out only 12% of the time (5 searches performed out of the 41 times
evidence was needed).
Conclusions.— Making evidence quickly available to clinicians on a busy medical inpatient
service using an evidence cart increased the extent to which evidence was
sought and incorporated into patient care decisions.
EVIDENCE-BASED medicine calls for the integration of our clinical expertise
with the best available external evidence and patients' values by translating
our need for information into an answerable question and then tracking down
the best information with which to answer that question.1
Smith2 has pointed out that although "most
of the questions generated in consultations go unanswered, most of [them]
can be answered, usually from electronic sources, but it is time consuming
and expensive to do so" and concluded that "the ideal information source will
be directly relevant, contain valid information, and be accessed with a minimum
amount of work."
Given the service demands of busy clinical firms and recognizing the
scant amount of time physicians can protect for reading around our patients,3 many hospitals have made access to information quicker
by placing computer terminals on or near the wards. These systems are too
slow to search during rounds, and only a few team members can simultaneously
view the same computer screen. We constructed an "evidence cart" that would
both contain the evidence we thought might meet Smith's criteria and provide
the means for accessing, projecting, and printing it and assessed whether
it was feasible to use in clinical practice.
Our evidence cart was a trolley (Supertrans Demtruck, Kentinental Engineering
Limited, Sevenoaks, Kent, England) modified to house a number of resources
including a notebook computer with CD-ROM drive (Toshiba Tecra 730XCDT, Hemel,
Hempstead); a computer projector with a collapsing screen (Epson EMP-3300;
Orbit Portable Projection Screen, Apollo Presentation Products, Leatherhead,
Surrey, England); compact discs of MEDLINE, Best Evidence, Radiological Anatomy,4Scientific American Medicine,5
and the Cochrane Library; reprints of the JAMA Rational Clinical Examination
series6; a physical examination textbook7; and compilations of the best evidence found in response
to clinical questions made by consultants (attending), house staff, and fellows,
called critically appraised topics (CATs)8
(51 topics) and the Redbook (98 topics), both of which were compiled on computer
as document files and printed out as hard copies. A Simulscope (Cardionics,
Houston, Tex) permitted several members of the team to listen simultaneously
to the same stethoscope to examine evidence acquired during physical examination.
The Redbook summaries were peer-reviewed, 1- to 3-page summaries of
critically appraised evidence created by the consultant or fellows and were
available in Redbook summaries updated the month prior to service. The CATs
also were created by the consultant, fellows, or house staff and were updated
annually or more often if needed.8 These summaries
addressed clinical examination and diagnostic tests, prognostic markers, or
treatments. Best Evidence contains the cumulated
contents of 2 journals of secondary publication, ACP Journal
Club and Evidence-Based Medicine.
Our clinical team comprised 9 first-year clinical medical students,
2 house officers, 1 full-time and 2 part-time senior house officers, 2 registrars,
a research fellow, and a consultant. The study took place in April (month
of use) and May 1997 (questionnaire). The cart could be used on 3 different
types of rounds. First, "posttake rounds" included bedside assessment of all
patients admitted to the general medicine service during the previous 8 to
16 hours and were attended by all team members who admitted patients. Second,
"team rounds" included the review of each patient on our service, including
bedside evaluation if necessary. Third, "student teaching rounds" were attended
by the students, research fellow, and consultant, and the students presented
individual patients in the team meeting room and at the bedside.
Cart use was usually precipitated by senior or junior staff asking 1
of 2 questions: "What's our/your evidence for that statement/diagnosis/treatment?"
or "Is there any other diagnosis/treatment that we should be considering for
Initial searches scanned sources of previously critically appraised
evidence in the Redbook and CATs. If these sources were insufficient, Best Evidence was searched next. MEDLINE (with a WinSPIRS
search engine) and Cochrane Library searching were used only when the former
sources were insufficient.
Each use of the cart was logged by the research fellow or consultant,
who recorded the source used and determined the reason for the search, whether
the search was successful, and whether it affected patient care. All team
members completed exit questionnaires at the end of the month and those who
remained on service the following month completed a follow-up questionnaire
after a weekend on-call period.
To determine the extent to which material addressed in the Redbook or
CATs was available from other sources, the consultant and fellow determined
the degree of overlap between these sources and Best Evidence, MEDLINE, JAMA Rational Clinical Examination series,6
and the physical examination textbook.7 "Time
trials" were conducted in which the consultant and research fellow logged
the time elapsed for a search that began in the team meeting room and searched
by random allocation of order and site (either on the cart or on a reserved
computer in the hospital library 4 floors away) for evidence on 5 clinical
questions selected randomly from the questions that had been successfully
answered during the month.
We cared for 166 inpatients during the month (and evaluated 30 more
who were not admitted), with approximately 21 admissions during each of our
8 "takes." The evidence cart was used 98 times during rounds.
After a single attempt to wheel the cart along the ward corridors, we
kept it in the team meeting room for rounds. Hard copies of the CATs and Redbook
and the Simulscope were taken on posttake rounds and to the bedside.
Examples of questions included the following: In a patient with stroke,
can the clinical examination reveal the prognosis? In a patient with syncope,
what diagnostic evaluation is necessary? In a patient with congestive heart
failure who cannot tolerate angiotensin-converting enzyme inhibitors, will
digoxin and nitrates decrease morbidity and mortality? In patients who are
depressed following a stroke, do antidepressants decrease morbidity and mortality?
In a patient with chronic obstructive pulmonary disease, do antibiotics reduce
morbidity and mortality? Most consultations (16 of 18 times) of the CATs and
Redbook were brief (<30 seconds) and were made to confirm, initiate, or
change a clinical decision. Copies of the relevant evidence was distributed
to team members after the round.
The Redbook was used most frequently (Table) on team rounds. Although MEDLINE was also used frequently
and was successful at answering 8 (57%) of 14 questions, it was so slow (minimum
searching time, 90 seconds) that most searches were completed outside rounds.
Mean searching times for the Redbook (39 searches) and CATs (21 searches)
were similar (10.2 seconds [SD, 3.0] and 11.7 seconds [SD, 2.3], respectively). Best Evidence (9 searches; mean searching time, 25.5 seconds
[SD, 4.2]) was usually consulted for evidence about therapy. A total of 16.4
clinical questions could be answered on the ward, using the cart, in the time
it took for a round-trip to the library to answer only 1 of them.
Best Evidence could be used to answer the same
clinical question that was answered by 7 (19%) of the 39 Redbook entries and
9 (44%) of the 21 CATs. The Cochrane Library answered 4 (10%) of the 39 questions
that were answered by the Redbook and 5 (22%) of the 21 answered by CATs.
Virtually all the contents of the Redbook and CATs could be found using MEDLINE.
Of the clinical questions addressed by the JAMA Rational Clinical Examination
series6 or the physical examination textbook,7 33% were addressed by the Redbook.
Ten percent of searches (10 of 98 searches) were unsuccessful in finding
useful evidence in the time available on rounds. These searches formed the
basis for "educational prescriptions"1 to search,
appraise, and prepare new CATs. Redbook and CATs had particularly high rates
of successful searching because their contents were well-known by the research
fellow and consultant. Seventy-nine (81%) of the 98 searches carried out were
for evidence that could affect diagnostic and/or treatment decisions, and
71 (90%) of those were successful. After the evidence was found, team members
(house officer, senior house officer, or registrar) were asked what impact
the evidence had on patient care. Of the 71 successful searches, 37 (52%)
confirmed their current or tentative diagnostic or treatment plans, 18 (25%)
led to a new diagnostic skill, an additional test, or a new management decision,
and 16 (23%) corrected a previous clinical skill, diagnostic test selection,
or treatment decision.
Of the 18 team members, 15 (83%) completed and returned the first anonymous
questionnaire. All respondents reported using the Redbook, CATs, and Best Evidence and found them useful and easy to use. MEDLINE
was the only resource considered not easy to use, with 4 (27%) of the 15 team
members finding it not easy to use (split equally between medical students
and house staff). Thirteen (87%) of 15 team members used the Simulscope, 11
(73%) used the JAMA Rational Clinical Examination series and 6 (40%) used
the physical examination text. All the individuals who reported using these
sources found they were easy to use and useful.
All of the 12 team members who remained on the service in the month
after the cart was assessed, after the cart had been removed, returned the
second questionnaire. The questionnaire was administered immediately after
a 2-day postcall period, and they reported needing evidence on 41 occasions
(10 occasions were observed during a similar period when the cart was available)
but carried out searches only 5 times (12%).
We found that evidence made available within seconds during rounds altered
the clinical approach of at least 1 team member 48% of the time, but when
the evidence was not readily available, the clinicians rarely searched for
This feasibility study has several limitations. First, the clinical
service has a strong evidence-based medicine orientation.9
Whether other services with other orientations would achieve these same results
is unknown. Second, the effect of evidence-based medicine on patient outcomes
was not determined. However, studies suggest that our results are generalizable
and that improved patient outcomes may result from this evidence-based approach.
A cohort of 448 US physicians was invited to request evidence from local libraries
about one of their patients.10 Half participated,
and 80% of the physicians reported that they changed the care of their patients
as a result of the evidence, by reducing mortality risks in 19% of patients,
changing their choices of diagnostic tests (51%) or drugs (45%), avoiding
hospitalization (12%), additional tests (49%), or surgery (21%), and reducing
their patients' length of hospital stay (19%). When Haynes et al11
provided brief instruction and free search time to trainees and attending
staff at a Canadian hospital, the rates with which that evidence, when pertinent
to clinical decisions, confirmed (70%), initiated (18%), or changed (13%)
clinical decisions were in the same range as those documented in this report.
Finally, a similar rate of change (18%) in patient management, following successful
searches, was reported by physicians and medical students who were provided
ready access and training in the use of MEDLINE in an emergency department.12
The third limitation of this study is that the study is a during-after
design, not a randomized trial. We do not know the extent to which other services
were seeking and using evidence during these same periods. It is uncertain
how the low rate of searching that occurred after the cart was removed was
affected by the opposing forces of a greater recognition of the need for evidence
vs the realization of the time it would take to search for it. Our descriptive
observations are supported by a Canadian randomized trial13
of "clinical librarians" joining rounds in which clinicians in the experimental
group were much more likely to use the library for direct patient care (their
searching frequency was 14 times that of control clinicians) and rated the
librarian and library resources more highly as sources of information.
Fourth, it is not known whether clinical services busier than ours could
afford the time required to find and discuss evidence in this fashion, although
most searching in the Redbook or among CATs could be completed in less than
15 seconds and in Best Evidence in less than 30 seconds.
Its feasibility would depend on a variety of factors including the number
of patients and house staff.
Fifth, 2 of the major sources used in the study, the CATs and Redbook,
were developed by our clinical teams for this purpose, and their contents
were well-known to our faculty. However, such customized clinical information
is being developed in a number of centers. Our experience, added to the prior
studies, suggests that clinicians will both seek and respond to relevant evidence
about their patients when the time required to access that evidence is within
the bounds of feasibility on a busy clinical service. This conclusion, if
it continues to be borne out, is both a cause for optimism and a call for
increasingly quicker access to valid, useful evidence at the bedside and in
the examining room.