Qualitative and Quantitative Measures of Indexed Health Sciences Electronic Journals

Context Little is known about qualitative and quantitative characteristics of indexed health sciences electronic journals (e-journals).

Methods To determine peer-review practices and qualitative and quantitative characteristics of different types of indexed health sciences e-journals, 3 types of e-journals indexed in MEDLINE were compared (type 1, completely electronic with no print counterpart; type 2, print and electronic versions with the same title but each publishing some unique content; and type 3, print and electronic versions containing equal content).

Results There were 13 type 1 journals, 16 type 2 journals, and 16 type 3 journals. Most journals in each category (85%-94%) imply or state the use of peer review. Significant differences (P<.05, analysis of variance) exist among the e-journals for the inclusion of complex types of publications (clinical trials, randomized controlled trials, meta-analyses, and practice guidelines) (15%-100%), editorials (0%-75%), letters to the editor (10%-88%), and case reports (17%-94%); the average number of items indexed in MEDLINE (22.5-544.5); and the number of complex publication types, editorials, letters, and case reports.

Conclusions Type 1 e-journals do not have the qualitative or quantitative complexity of traditional print journals. Although editors' statements on editorial peer review are similar, there are differences in number and type of materials included in the 3 different types of e-journals.

Electronic publishing has the potential to offer many advantages over traditional print publishing, but concerns over quality exist.¹ Several studies have identified the following qualitative and quantitative measures that are appropriate for electronic journals (e-journals): peer-review practices,² citation patterns,³ structured abstracts,⁴ and editorial boards.⁵ The objectives of this study were to determine if there are different editorial peer-review practices or variations in qualitative and quantitative measures among different types of indexed health sciences e-journals.

There are 3 types of e-journals: type 1 are completely electronic with no regular print version; type 2 are titled the same both in the print and electronic versions, but each publish some unique content; and in type 3, both the print and electronic versions publish equal content. Each type 2 version has its own international standard serial number, but type 3 e-journals have 1 international standard serial number.

Thirty e-journals were listed as active MEDLINE titles in June 2001: 13 were type 1, 16 were type 2, and 1 was not yet published. Abridged Index Medicus is a subset of 120 MEDLINE titles, which is considered a solid, core set of health sciences journals, and was used in selection of type 3 e-journals. To compare type 1 and type 2 e-journals with traditional print journals, 16 that had electronic access to current issues and qualified as type 3 e-journals were randomly selected from Abridged Index Medicus titles.

Data for this study were collected from MEDLINE and each journal's Web site homepage. All items published in the year 2000 and indexed by August 2001 were included. No e-journal in this study was selectively indexed by MEDLINE.

Qualitative and quantitative measures included: editorial peer-review statements, inclusion of editorial boards, requirement of original research or structured abstracts, number of articles cited, the inclusion and number of complex publication types, case reports, editorials, and letters to the editor. The index terms clinical trials, randomized control trials, meta-analyses, and practice guidelines were used for complex types of publications.

Type 2 e-journals always shared a Web site homepage and had 1 set of instructions to authors. Items indexed from electronic type 2 e-journals always carried a unique identifier in MEDLINE, usually an "e" preceded the page number. Table 1 lists those e-journal characteristics that were the same for both type 2 electronic and print versions. More than 80% of all e-journals stated or implied the use of peer review and required original material. These data were located by searching the Web site homepages and reading the instructions to authors sections and other information about the e-journals. The average number of type 1 e-journals that required structured abstracts (36%) was less than for either type 2 or type 3. All type 2 and type 3 and 92% of type 1 e-journals listed an editorial board.

A search in August 2001 of Institute for Scientific Information's Web of Science determined frequency of citations to each e-journal. The ISI captures all cited references in each journal it covers independent of whether the title is covered by the database; 54% of type 1 e-journals are covered by ISI. An average of 4.62 articles published in type 1 e-journals in the year 2000 were cited, whereas more than 350 articles published in type 2 and type 3 e-journals in the year 2000 were cited (P = .01, single-factor analysis of variance).

Table 2 compares results when separate data exist for type 2 e-journals. There were statistically significant differences among the e-journals (P<.001) indexed as complex publication types. The type 1 and the electronic type 2 were less likely (P<.001) than print type 2 or type 3 e-journals to publish editorials, letters to the editor, and case reports.

For the year 2000, type 1 and electronic type 2 published significantly fewer items than print type 2 and type 3 (P<.001). There were also statistically significant differences for the average number of items indexed as complex types of publication (P = .01). Both type 1 and electronic type 2 published significantly fewer editorials (P = .03), letters (P = .002), and case reports (P = .004) than print type 2 and type 3.

The following qualitative measures were similar for all types of e-journals: peer-review statements, requirement of original research, and the inclusion of editorial boards. A lack of a statement on peer review or original research does not mean that no policy exists. Editors address the editorial peer-review process in journal guidelines more frequently today than they did in 1987.² The structured abstract was proposed for medical journals in 1987.⁴ One third to two thirds of all types of e-journals in this study used this format. This study confirmed the earlier study by Harter³ that type 1 e-journals continue to be infrequently cited. This phenomenon may continue until type 1 e-journals become established, mainstream medical literature.

Many electronic type 2 publications serve a specialized function and may represent a trend to publish journals with different content in the print and electronic versions. The electronic type 2 publications often include only 1 type of feature, such as short reports, rapid communication, or letters. For example, one electronic type 2 e-journal published all letters electronically, while another published only "ultra-rapid" communications, and another let the author choose which version for case reports. Eleven (69%) of the 16 type 2 journals stated specific criteria for inclusion in the electronic only version.

The 13 indexed health sciences type 1 e-journals lack the depth and breadth of traditional print journals. The type 1 and electronic type 2 e-journals published significantly fewer indexed complex types of publications, editorials, letters, and case reports than either type 2 print or type 3 e-journals. Data from this study provide an early snapshot of e-journals. The type 1 e-journals will probably become more numerous and more accepted, but they currently do not have the complexity of traditional print journals.