Effects of Technical Editing in Biomedical Journals: A Systematic Review

Context Technical editing supposedly improves the accuracy and clarity of journal articles. We examined evidence of its effects on research reports in biomedical journals.

Methods Subset of a systematic review using Cochrane methods, searching MEDLINE, EMBASE, and other databases from earliest entries to February 2000 by using inclusive search terms; hand searching relevant journals. We selected comparative studies of the effects of editorial processes on original research articles between acceptance and publication in biomedical journals. Two reviewers assessed each study and performed independent data extraction.

Results The 11 studies on technical editing indicate that it improves the readability of articles slightly (as measured by Gunning Fog and Flesch reading ease scores), may improve other aspects of their quality, can increase the accuracy of references and quotations, and raises the quality of abstracts. Supplying authors with abstract preparation instructions had no discernible effect.

Conclusions Considering the time and resources devoted to technical editing, remarkably little is known about its effects or the effects of imposing different house styles. Studies performed at 3 journals employing relatively large numbers of professional technical editors suggest that their editorial processes are associated with increases in readability and quality of articles, but these findings may not be generalizable to other journals.

Most articles in biomedical journals undergo some form of editing between acceptance and publication. The intensity and type of editing vary between journals but the process often includes applying house style to references, abbreviations, and numbers, checking articles for consistency, clarity, and completeness, and correcting grammatical errors. For this review, we examined any processes applied to articles between acceptance and publication that were designed to improve accuracy or clarity or impose a predefined style.

We performed a systematic review using Cochrane methods,¹ searching MEDLINE, EMBASE, Current Contents, and 10 other databases from earliest entries to February 2000. We used inclusive search terms such as writing, editing, accuracy, and readability. Relevant journals (eg, Learned Publishing, Journal of Information Sciences, and JAMA special issues on peer review) were hand searched. We contacted researchers working in the field to request publications that we had missed. The full search strategy is described in the protocol.¹

The full review included any comparative studies of the effects of processes designed to improve the quality of accepted research articles in biomedical journals.¹ Those on editorial decision making and peer review formed the basis of a separate review.² Although we sought evidence about biomedical journals, we did not restrict our search to studies published in such journals. Studies of readability or comprehension had to involve journal readers (ie, health care professionals). We included articles in any language but restricted our searches on evidence about improving writing style to English. This article presents the findings from the subset of articles that focused on editorial processes between acceptance and publication. Details of studies on the effects of other processes, such as providing instructions to contributors and imposing a structured abstract format, can be found in the Cochrane review.¹

Because few studies used comparable methods, we performed a descriptive review of 11 articles (Table 1).

The study at the Dutch Medical Journal³ demonstrated significant improvements in papers between acceptance and publication, as measured by readers using a purpose-designed scoring system. Another study at Annals of Internal Medicine⁴ showed similar improvements but compared submitted and published versions and so was unable to distinguish the effects of peer review from technical editing.

Two studies compared readability scores and demonstrated improvements between submission and publication.^5,6 Although the improvements were statistically significant, the absolute increases were small, and scores indicate that, even after editing, the articles remained difficult to read. Guidelines for using the Flesch readability scale suggest that scores of 50 to 60 are desirable for standard documents, but articles in studies scored around 30.¹⁴ The Gunning Fog test, which gives lower scores for clearer writing, indicated that papers remained difficult to read, with scores of 15 to 17, on a par with legal contracts or corporate reports (as opposed to quality newspapers, which score about 9).¹⁵ The articles did not provide details about which interventions were thought to affect readability.

Three studies considered the effects of technical editing on the accuracy of reference citations and quotations.^7-9 Those that compared submitted and published versions showed clear improvements in accuracy.^7,8 A comparison of journals with different checking policies was less conclusive.⁹

A study of abstract completeness and consistency indicated variation between journals but could not identify the processes associated with better results,¹⁰ while 2 studies relating to the introduction of more intensive abstract editing at JAMA showed clear improvements in accuracy after the intervention.^11,12 Another study by Pitkin and Branagan¹³ examined the effects of sending abstract preparation instructions to authors after their article had been accepted but did not detect any improvement in the quality of abstracts from authors who had received the intervention.

Remarkably little research into the effects of technical editing performed on articles in biomedical journals has been published.

The quality of study methods varied; 6 compared different versions of articles, 2 used before and after designs, 2 made comparisons between journals, and 1 used a randomized design. Some of the studies comparing different versions of articles (eg, accepted vs published) attempted to mask the articles' status, but often unsuccessfully (eg, in the Dutch Medical Journal study, 72% of assessors correctly identified the version of the article³). Comparisons between journals are hard to interpret because findings may be influenced by extraneous factors. Comparisons over time may be similarly confounded.

Another aspect of methodological quality is the use of appropriate rating scales and assessors. Two studies used Flesch and Gunning readability scores,^5,6 which have not been validated on scholarly literature and are based not on adult readers, but on the ways in which children learn to read. Both produce scores based on sentence and word length but do not take into account word familiarity or sentence complexity. Therefore, they are at best surrogate markers of the comprehensibility of a research report. A study in which students assessed structured and unstructured abstracts found that the structured version was considered significantly easier to read, despite the fact that Flesch scores were similar for both versions.¹⁶ Further work is needed to test the ability of these scores to measure the comprehensibility of biomedical articles.

Only one study recruited journal readers to act as assessors.³ In other cases, the investigators themselves, experts, or professional editors rated the quality of papers. Although this step may have reduced interrater variability, the everyday use of journals may not have been reflected, and there is evidence that readers do not always agree with experts on the quality of papers.¹⁷ Several studies used specially devised scales to rate the quality of papers or abstracts. In most cases, no details of interrater reliability or face validity were given. However, the study by Goodman et al⁴ reported poor reliability (intraclass correlation coefficient of 0.12) for its scoring system.

Most studies that examined changes between acceptance and publication did not specify the processes that took place. However, the Dutch Medical Journal investigators note that "[d]uring editing, the information in the article is checked scientifically and linguistically, corrected and clarified if necessary, numbers are checked when possible, and the references are made to conform to the so-called Vancouver system."³

Studies of reference accuracy indicate how the intensity of editing varies between journals. For example, editors at the Dutch Medical Journal and British Journal of Dermatology check references against MEDLINE,⁷ while those at the Journal of the American Academy of Dermatology check references only from their own journal.⁹ Lowry, who studied letters in the BMJ in 1985, explains that "[a]lthough the journal does not check all references . . . the subeditors correct any obvious errors. References are put into the house style, which allows many mistakes to be spotted, especially where the fault is an incomplete reference, which is inevitably corrected."⁸

Many of the studies of technical editing relate to general medical journals that employ professional technical editors. They are likely to have received more training and to have more time than academic editors who edit their journal in addition to holding a full-time job. Thus, Goodman et al⁴ comment that "the relatively large editorial staff at Annals is not typical of any but the largest medical journals, and the generalization to others with different . . . editing processes cannot easily be made."

Three of the 4 studies examining overall quality or readability failed to distinguish between changes occurring between submission and acceptance, which might be considered part of peer review, and those between acceptance and publication, which meet our definition of technical editing.^4-6 Only the Dutch study specifically examined and demonstrated improvements occurring between acceptance and publication.³ A more recent study at Annals of Internal Medicine found that only 3% of substantive changes introduced in manuscripts resulted from technical editing, although the origin of 47% of changes was unknown, so this figure may have been substantially higher.¹⁸ It is also possible that other textual changes (that were not considered in the Annals study) contributed to manuscript clarity and readability (Frank Davidoff, oral communication, September 2001).

Nearly all journals impose a house style that includes elements of typographic design, such as typeface, and scientific conventions, such as number format. The only aspect of journal style that has attracted research is the structuring of abstracts. Although studies comparing readability scores are inconclusive, structured abstracts are preferred by readers and are more comprehensive but are also longer than unstructured ones.^16,19-24 Apart from this, we found no research about the effects of different styles on legibility or readability, and we conclude that the imposition of such styles is not evidence based, unless journals have undertaken unpublished research.

Our review suggests a pattern of improvement in the accuracy and readability of research articles, which occurs between acceptance and publication. However, few studies have attempted to determine which processes contribute most to this improvement or how variations in these processes might affect the quality of published papers. Without elucidation of the processes involved, these findings may not be generalizable. Despite the time and resources spent on technical editing, articles remain difficult to read, although intensive editing and checking may lead to improvements in the accuracy of references and abstracts. Few studies have consulted journal readers about their needs or views or worked with authors to improve manuscript quality.