Awareness of Sources of Peer-Reviewed Research Evidence on the Internet

Context Peer-reviewed research evidence that was once available only to clinicians is now posted on the Internet and accessible to everyone, but levels of awareness of this evidence among patients and clinicians is not known.

Methods Cross-sectional survey, conducted from July 1998 through January 2000, of cancer patients (n = 1998), their family physicians (n = 871), and all oncologists (n = 30) and oncology nurses (n = 44) at the Hamilton Regional Cancer Centre, Hamilton, Ontario. Comparisons made between and within groups to examine use of the Internet by patients and clinicians to find health information, and their awareness of organizations using Internet-based resources to promote evidence-based decision making.

Results Response rates were 72%, 44%, 97%, and 84% for patients, family physicians, oncologists, and nurses, respectively; 47% of patients, 64% of family physicians, 100% of oncologists, and 72% of nurses reported that they used the Internet. Few patients were aware of the existence of the Cochrane Collaboration (1%), MEDLINE (13%), or the Program in Evidence-Based Care of Cancer Care Ontario (3%). Oncologists had the highest reported levels of awareness of the sources of evidence on the Internet. Most family physicians had not heard of any of the sources.

Conclusions The awareness of evidence sources on the Internet varies between patients and clinicians and across groups of clinicians, and some of the most rigorously developed sources of information are still unknown.

Evidence-based decision making (EBDM) is an initiative that seeks to facilitate the judicious, systematic, explicit, and conscientious consideration of the current best evidence from research to guide health-related decisions. This initiative is facing a series of barriers that hinder its implementation. Some of these barriers are associated with the decision makers (eg, lack of awareness or understanding of EBDM, lack of time), the evidence per se (eg, abundance, bias, imprecision, irrelevance), and the health care system (eg, poor infrastructure, lack of financial incentives, ineffective training programs).¹

Since the introduction of the World Wide Web, many organizations promoting EBDM have used Internet-based resources to facilitate the dissemination, access, and use of the best available research evidence by anyone interested in health-related decisions.² To our knowledge, there have been no systematic efforts to examine decision makers' awareness of such resources and the organizations that sponsor them. We searched MEDLINE in January 1998 and October 2001, using the terms Internet, awareness (or survey), and evidence, and found no published study that addressed this issue.

The main objective of this study was to estimate and compare the levels of awareness of important sources of research evidence that can be found on the Internet by patients with cancer, their family physicians, oncologists, and oncology nurses. Cancer was selected as the focus for this study because it is a highly prevalent disease; has high mortality, morbidity, stress, and costs; and has been the focus of efforts to promote effective dissemination of treatment information through the Internet.

The study was based at the Hamilton Regional Cancer Centre (HRCC), a tertiary-level institution in Hamilton, Ontario, affiliated with the faculty of health sciences at McMaster University—the place where the term evidence-based medicine was coined, and the host of the Canadian Cochrane Centre and the Program in Evidence-Based Care (PEBC) of Cancer Care Ontario.

Three groups of patients attending the HRCC were randomly selected and surveyed at 6-month intervals from July 1998 through January 2000. Individuals were excluded if they were unable to speak or read English, if they were too ill to answer, or if they died within 2 weeks of the mailings. Eligible patients were assured that their participation was voluntary and that they would not be identified individually.

The survey was also administered during the first 12 months of the study to all clinical oncologists and oncology nurses affiliated with the institution. At each period, the questionnaire was offered to all family physicians listed as involved in the care of the study patients.

We selected 2 initiatives that provide information about the whole spectrum of health care (the Cochrane Collaboration, PubMed/MEDLINE) and 3 organizations that focus on cancer and its treatment (CancerNet, CancerLit, and the PEBC³).

The questionnaire was developed with input from health professionals, researchers, and patients, with slight modifications to address each of the groups. The questionnaire asked for information on use of the Internet to find health information and awareness of initiatives promoting EBDM that have produced Internet-based resources to facilitate access to research evidence (Table 1). Participants were asked if they had heard of these sources, not if they had used them. The survey followed a modified Dillman method.⁴ Questionnaires were sent to oncologists and nurses through internal institutional mail and to patients and family physicians by post. All data gathering was completed on January 31, 2000.

Relevant comparisons of descriptive statistics were made between and within groups (Table 1) using χ² and Fisher exact tests (2-sided). P<.05 was considered statistically significant. The statistical analysis was conducted using SAS v8.02 (SAS Institute Inc, Cary, NC).

Response rates were 72%, 44%, 97%, and 84% for patients, family physicians, oncologists, and nurses, respectively; 47% of patients, 64% of family physicians, 72% of nurses, and 100% of oncologists reported using the Internet.

Of the Internet users, 63% of the patients were female, 67% were older than 50 years, and 51% (204/403) had education beyond high school; 22% of family physicians were women and 28% were older than 50 years; all nurses and 14% of oncologists were women, whereas both groups had similar proportions of people older than 50 years (15% and 11%, respectively).

Significantly fewer patients than clinicians reported that they looked for health information on the Internet (Table 1). Few patients were aware of the existence of the Cochrane Collaboration (1%), MEDLINE (13%), or the PEBC (3%).

Among the health care professionals, oncologists had the highest reported levels of awareness of the sources of evidence on the Internet. As expected, oncology clinicians were significantly more aware about the existence of cancer-specific sources than family physicians. The only significant difference between oncologists and nurses was related to awareness of the Cochrane Collaboration and PubMed.

This study shows that efforts to make research evidence available on the Internet remain unknown to many people involved in health-related decisions, regardless of their background. The results also suggest that there are major differences across groups, even within the same institution. The gap in Internet use between patients and health care professionals is wide.⁵

Our findings could have at least a couple of explanations. First, it is possible that organizations promoting EBDM are not placing enough emphasis on efforts to make people aware of their existence. Second, most decision makers may not be interested in research evidence at all, being less prone to notice and remember organizations promoting EBDM.

Although the results of the study seem plausible, their validity may have been compromised by the small sample size for oncologists and nurses and by the relatively low response rate of family physicians. The response rate from the family physicians, although low, is typical of other surveys.⁶ The focus on a specialized cancer treatment facility also limits the generalizability of the results. We have initiated similar studies in other regions of Ontario, focusing on other health conditions. We encourage similar efforts by research groups in other parts of the world. Another limitation of this study is that it focused on awareness, not use, of the information produced by organizations promoting EBDM through Internet-based resources.

In summary, our findings suggest that the potential for the Internet to promote inexpensive, fast, and efficient access to the most up-to-date, valid, and relevant knowledge has not been realized. Unless organizations promoting EBDM make stronger and more effective efforts to help their target audiences become aware of their existence, the Internet may become a missed opportunity to help decision makers access evidence.