From the Division of Medical Informatics & Outcomes Research (Drs Hersh and Hickam) and the Department of Medicine (Dr Hersh), Oregon Health Sciences University, and Health Services Research and Development, Department of Medicine, Portland VA Medical Center (Dr Hickam), Portland, Ore.
Objective.— Despite the proliferation of electronic information retrieval (IR) systems
for physicians, their effectiveness has not been well assessed. The purpose
of this review is to provide a conceptual framework and to apply the results
of previous studies to this framework.
Data Sources.— All sources of medical informatics and information science literature,
including MEDLINE, along with bibliographies of textbooks in these areas,
were searched from 1966 to January 1998.
Study Selection.— All articles presenting either classifications of evaluation studies
or their results, with an emphasis on those studying use by physicians.
Data Extraction.— A framework for evaluation was developed, consisting of frequency of
use, purpose of use, user satisfaction, searching utility, search failure,
and outcomes. All studies were then assessed based on the framework.
Data Synthesis.— Due to the heterogeneity and simplistic study designs, no meta-analysis
of studies could be done. General conclusions were drawn from data where appropriate.
A total of 47 articles were found to include an evaluation component and were
used to develop the framework. Of these, 21 articles met the inclusion criteria
for 1 or more of the categories in the framework. Most use of IR systems by
physicians still occurs with bibliographic rather than full-text databases.
Overall use of IR systems occurs just 0.3 to 9 times per physician per month,
whereas physicians have 2 unanswered questions for every 3 patients.
Conclusions.— Studies comparing IR systems with different searching features have
not shown that advanced searching methods are significantly more effective
than simple text word methods. Most searches retrieve only one fourth to one
half of the relevant articles on a given topic and, once retrieved, little
is known about how these articles are interpreted or applied. These studies
imply that further research and development are needed to improve system utility
Hersh WR, Hickam DH. How Well Do Physicians Use Electronic Information Retrieval Systems?A Framework for Investigation and Systematic Review. JAMA. 1998;280(15):1347-1352. doi:10.1001/jama.280.15.1347