Frequency of Abstracts Presented at Eye and Vision Conferences Being Developed Into Full-Length Publications: A Systematic Review and Meta-analysis

Key Points

Question  What is the proportion of abstracts presented at eye and vision conferences that are subsequently published in full articles and what factors are associated with more frequent publications?

Finding  In this systematic review and meta-analysis, 38% of abstracts presented at eye and vision conferences were subsequently published in full. Oral presentations and basic science research were significantly associated with more frequent publications.

Meaning  These data suggest that most research presented at eye and vision conferences does not reach the public domain within 2 years, indicating the need to investigate the underlying reasons for nonpublication within 2 years of presentation.

Importance  Conference proceedings are platforms for early communication and dissemination of relevant and timely topics of interest. More than half of abstracts presented at biomedical conferences fail to be published in full, resulting in wasted time and resources.

Objective  To systematically review reports evaluating the proportion of abstracts presented at eye and vision conferences that are subsequently published in full and investigate factors associated with publication.

Data Sources  MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, Web of Science, and reference lists of included reports were systematically searched from inception to January 11, 2019.

Study Selection  Reports that examined the proportion of abstracts presented at eye and vision conferences and subsequently published in peer-reviewed journals 24 or more months later.

Data Extraction and Synthesis  Two reviewers independently assessed study eligibility, abstracted data, and evaluated the risk of bias. A meta-analysis was conducted to determine the proportion of abstracts published in full and assess factors associated with subsequent full publication.

Main Outcomes and Measures  Proportion of abstracts presented at eye and vision conferences subsequently published in full.

Results  There were 19 reports covering 12 261 abstracts presented at 11 unique eye and vision conferences. The overall risk of bias of the reports was low. The weighted proportion of abstracts published in full was 38.0% (95% CI, 31.7%-44.3%) and 54.9% (95% CI, 34.6%-73.7%) among reports restricted to abstracts describing randomized clinical trials. Nine reports (47.4%) investigated the proportion of abstracts subsequently published by ophthalmic subspecialties, ranging from 28.3% (oculoplastics: 95% CI, 17.2%-42.9%) to 42.7% (glaucoma: 95% CI, 34.7%-51.0%). Oral presentation (risk ratio, 1.45; 95% CI, 1.20-1.76) and basic science (risk ratio, 1.25; 95% CI, 1.05-1.47) were significantly associated with higher full publication; factors not significantly associated with full publication included positive results, randomized clinical trial vs other study design, multicenter study, and industry funding.

Conclusion and Relevance  More than 60% of abstracts presented at eye and vision conferences were not published in full within 2 years of conference presentation. Failure to disseminate research studies in peer-reviewed journals is not desired, especially when involving human participants.

Biomedical conferences are the primary venues to bring researchers together to present their work to the scientific community. Conferences are an integral part of the scientific process because they facilitate dissemination of research findings. Across all biomedical fields, more than half of all abstracts and one-third of randomized clinical trials (RCTs) initially presented at conference proceedings fail to be published as journal articles.^1,2 These unpublished results are often only located in conference proceedings or journal supplements and are not indexed or identifiable in commonly used electronic bibliographic databases (eg, PubMed). Although Embase began indexing abstracts from the Association for Research in Vision and Ophthalmology (ARVO) starting in 2014 and European Association for Vision and Eye Research Conference (EVER) in 2015,³ it does not index the American Academy of Ophthalmology (AAO), one of the largest international eye and vision conferences.³ Thus, abstracts from AAO must be retrieved from conference archives of society webpages. Because conference abstracts contain important study information that is unavailable elsewhere, systematic reviewers need to purchase society memberships or manually search conference proceedings to retrieve all relevant studies. This approach is cost-intensive and error-prone if many manual searches are required for systematic reviews.

Failure to include data from conference abstracts may lead to biased results in systematic reviews because positive results are published in full more often than negative or null results.¹ If results presented at conferences fail to be included in systematic reviews, these biased findings may be included in subsequent guidelines, resulting in misleading recommendations for clinical practice. On the other hand, insufficient data within conference abstracts leads to difficulties while incorporating and summarizing them in systematic reviews. Imposed limits on the number of words, tables, or figures constrain authors to describe the study design, participant characteristics, and methods adequately.^4-6 It is even more difficult to obtain the detailed outcomes and results that are required for meta-analyses⁷ or results may represent preliminary results.⁸ More than half of RCTs in ophthalmology and even more other study designs show inconsistencies between conference abstracts and corresponding journal publications.^9,10 Although usually minor in scope, these discrepancies leave systematic reviewers in danger of including incomplete results from conference abstracts or require substantial effort to contact abstract authors for complete and accurate data.

Prior research has investigated the proportion of abstracts subsequently published in full for various eye and vision conferences; however, there is no review to systematically identify and summarize these studies. Additionally, as systematic reviewers may overrely on fully published research to draw conclusions,¹¹ it is important to examine whether the presence of positive results in the abstracts is associated with more frequent publication of eye and vision research. To understand the volume of research in ophthalmology that remains unpublished and identify potential factors associated with subsequent full publication, we conducted this study to systematically review reports evaluating the proportion of abstracts presented at eye and vision conferences that are subsequently published in full and investigate factors associated with publication.

For this systematic review, we followed the systematic review protocol previously used and described by Scherer et al.¹ We refer to articles that examined the fate of abstracts that are included in this systematic review as reports and the abstracts analyzed within those reports as studies. Because the data used were free and publicly available, institutional review board approval was not necessary for this study.

Reports describing abstracts presented at eye and vision–related conferences were eligible. These reports also met the following criteria: identified those abstracts that were subsequently published in full and allowed an interval of at least 24 months between conference presentation and full publication to evaluate the proportion of abstracts published.

We searched MEDLINE, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, Web of Science, and reference lists on January 11, 2019, for reports that examined the proportion of abstracts presented at eye and vision conferences that were published as journal articles. We searched for references to include reports in the Science Citation Index and Google Scholar (see eMethods 1 in the Supplement for the full search strategies). We did not use any language or date restrictions in our searches.

Three reviewers (J.-Y.E., K.F., and R.W.S.) independently screened the titles and abstracts identified by the searches based on eligible criteria, classifying them as eligible, not eligible, or uncertain. Full-text articles classified as eligible or uncertain were retrieved and assessed independently by 2 reviewers (J.-Y.E. and R.W.S.) for eligibility. We resolved discrepancies in the classification of eligibility by discussion and consensus.

For each included report, 2 reviewers independently extracted information on the total number of abstracts, proportion subsequently published in full, and cumulative proportion published in full at 6-month intervals following conference presentation. We also extracted information on society, year, country of corresponding author, type sample (all abstracts, random or selected), and time between conference presentation and the search for publication. We extracted information on the proportion of abstracts subsequently published in full by the following factors: positive or statistically significant results, oral vs poster presentation, clinical vs basic science research, RCT vs other study design, multicenter vs single-center, language of the report author (English vs non-English), continent of report’s author, and funding sources (industry vs other).

We collated the proportions of abstracts subsequently published in full by ophthalmology subspecialties and in total. One author (P.Y.R.) classified subspecialties as described by report authors into one of the following subspecialties: glaucoma, cornea, retina, oculoplastics, pediatric ophthalmology and strabismus, uveitis, neuroophthalmology, and other.

We assessed risk of bias (RoB) in the reports by grading each of the following 5 domains as low, high, and unclear using the RoB assessment tool published elsewhere (eMethods 2 in the Supplement)¹: method used to determine the sample of abstracts to study; length of follow-up time between the date of the conference presentation and the date of the search of the electronic database or the author contact for corresponding full publications; method to identify full publication (search of electronic database or author contact); method to match the abstract to a corresponding full publication; and adjustment for factors associated with full publication. We graded reports as high overall RoB if 3 or more items were classified as high RoB or if 2 or more of 4 items were classified as high RoB when the fifth criterion was not applicable. We compared assessments and data extracted by 2 reviewers and resolved disagreements through discussion.

We calculated the overall proportion of abstracts subsequently published in full for all reports using a random-effects model through logit transformation¹² with 95% CIs using the Hartung-Knapp method.¹³ We also estimated the proportion of abstracts published in full by ophthalmology subspecialty. We ignored potential overlaps and analyzed all abstracts in the report if the same conference was reported in multiple reports.

We analyzed predetermined factors possibly associated with full publication using a random-effects model. We calculated risk ratios (RRs) and 95% CIs to express the strength of associations. We further assessed statistical heterogeneity by χ² and I² tests and looked for the sources of substantial heterogeneity; we conducted sensitivity analyses if I² was greater than 50%.

We conducted a survival analysis to determine the cumulative proportion of abstracts published over time. We used 6-month intervals to graph the percentage of abstracts published since conference presentation in a Kaplan-Meier curve, censoring abstracts that remained unpublished at the end of each follow-up period. We conducted all analyses in R, version 3.6.1 (R Foundation), and RevMan, version 5 (Cochrane). Statistical significance was set at P < .05.

We identified 1266 reports in the electronic searches. After removing duplicates, 1023 unique reports were screened based on titles and abstracts. Most of these reports clearly did not fulfill 1 or more eligibility criteria. We retrieved 22 reports (2.2%) for full-text screening, of which 18 (81.8%) were classified as eligible. We identified 1 additional report published after completion of searching, resulting in 19 reports included in this systematic review (Figure 1). All 9 reports (0.9%) describing abstracts presented at eye and vision conferences and included in the systematic review by Scherer et al¹ were included among the 19 included reports.

Within the 19 reports, 12 261 abstracts were presented at 11 unique eye and vision conferences, including ARVO, AAO, EVER, American Academy of Optometry, American College of Veterinary Ophthalmologists, Canadian Ophthalmological Society, Royal College of Ophthalmologists, European College of Veterinary Ophthalmologists, European Society of Ophthalmology, Asia-Pacific Academy of Ophthalmology, and the All India Ophthalmological Conference. The conferences were held from 1984 to 2014; 12 reports (63.2%) included multiple annual conferences. Twelve reports (63.2%) evaluated all abstracts presented at a conference: 4 (21.1%) selected a random sample of abstracts to study and 3 (15.8%) included a sample selected by study design (2 restricted to RCTs^9,14 and 1 to diagnostic test accuracy studies¹⁵). Seventeen (89.5%) reported a follow-up time between conference presentation and the search for publication of more than, and 1 (5.3%) of fewer than, 48 months. One report (5.3%) did not specify a time interval between conference presentation and the search for full-length publications but was included because the interval between conference presentation and the publication of the study was at least 3 years.¹⁶ Fourteen reports (73.7%) provided information on the cumulative proportion of abstracts published in full over time (Table 1^{9,14-17,19,20,22-24,26-29}).

Most (12 of 19 [63.2%]) reports had at least 1 design component with a high RoB. However, the overall RoB was low across reports, with 1 exception, which was considered as having a high RoB overall.³⁰ Commonly, the design components with a high RoB were inadequate methods to identify all possible full publications or failing to adjust analyses by known factors. The full RoB assessment is described in eFigure in the Supplement.

We found that 38.0% (95% CI, 31.7%-44.3%) of abstracts had been published in full but with substantial heterogeneity (I² = 98%). When we compared the 2 reports (10.5%) that selected RCTs with 17 reports (89.5%) that included all study designs, the proportion of abstracts published in full was 54.9% (95% CI, 34.6%-73.7%; I² = 93%) vs 36.0% (95% CI, 30.1%-43.2%; I² = 98) (Table 1). Because Scherer et al¹ found a difference between reports authored by non–native English speakers, we also calculated the proportion of abstracts published in full in 13 reports (68.4%) in which the corresponding authors were from native English-speaking countries (40%; 95% CI, 34%-46%; I² = 97) and in 6 reports (31.6%) in which they were not (34%; 95% CI, 17%-51%; I² = 99%). Similarly, reports classified by author’s continent of origin resulted in the proportion of abstracts published in full of 44% (9 [47.4%]; 95% CI, 34%-53%; I² = 97%) for reports from North America, 39% (5 [26.3%]; 95% CI, 27%-51%; I² = 98%) from Europe, and 28% (5 [26.3%]; 95% CI, 13%-42%; I² = 99%) from Asia. Nine reports (47.4%) examined the proportion of abstracts subsequently published by subspecialty, as shown in Table 2^{15,18,19,23,25-27,29,31}. The ophthalmology subspecialities are as follows: glaucoma (9 [47.4%]; 42.7%; 95% CI, 34.7%-51.0%]); pediatric ophthalmology and strabismus (7 [36.8%]; 37.9%; 95% CI, 24.3%-53.8%); uveitis (5 [26.3%]; 37.4%; 95% CI, 23.6%-53.7%); retina (8 [42.1%]; 35.8%; 95% CI, 26.8%-45.8%); neuroophthalmology (7 [36.8%]; 35.0%; 95% CI, 23.7%-48.4%); cornea (9 [47.4%]; 31.8%; 95% CI, 24.4%-40.4%); oculoplastics (6 [31.6%]; 28.3%; 95% CI, 17.2%-42.9%]); and across other areas (3 [15.8%]; 11.9%; 95% CI, 6.1%-21.9%).

Table 3 summarizes the results of the meta-analysis for factors associated with full publication. Twelve reports (63.2%) investigated the association between presentation type and full publication: abstracts presented orally were 45% more likely to be published compared with those presented as posters (RR, 1.45; 95% CI, 1.20-1.76; I² = 88%). Nine reports (47.4%) studied the association between research type and full publication: abstracts describing basic science research were 25% more likely to be published than those describing clinical research (RR, 1.25, 95% CI, 1.05-1.47; I² = 72%). The results of a sensitivity analysis that excluded reports that described only RCTs or were from non–English-speaking countries did not change the estimates or reduce heterogeneity.

Two reports (10.5%) emanating from RCTs examined the association between positive results, defined as statistically significant results, and subsequent full publication.^9,14 No association was observed between positive results and full publication (RR, 1.10; 95% CI, 0.91-1.32; I² = 0%). Other factors showing no association with full publication included RCTs vs other study designs, multicenter studies, and industry funding (Table 3).

Ten reports (52.6%) provided data to calculate cumulative time to publication using a survival analysis. We estimated that the proportion of abstracts published in full at 84 months was 39.0%. The rate of full publication following presentation was higher during the first 36 to 48 months and subsequently decreased (Figure 2).

We found that slightly more than one-third of abstracts presented at eye and vision conferences were subsequently published in full within 2 years of presentation. The proportion of abstracts published within 2 years of presentation was low across all subspecialties of ophthalmology, ranging from 28.3% for oculoplastics to 42.7% for glaucoma. Oral presentation and basic science research were strongly associated with more frequent publication. Factors not found to be associated with full publication included positive results, RCT vs other study designs, multicenter studies, and industry funding. These findings reinforce a key message from the other biomedical fields, that most research presented at conferences may not ultimately reach the public domain.

Our review found that only 38.0% of abstracts presented at eye and vision conferences resulted in full-length publication within 2 years, which might be an underestimate for 2 reasons. Full publication may occur after the time of the search for corresponding publication. However, a survival analysis that censors abstracts at the end of follow-up provides a similar estimate of full publication in that the estimated proportion of abstracts published in full at 84 months was 39.0%. Second, 9 reports identified full publication only by searching a single electronic database. This inadequate search might have missed publications that were not indexed in the database. Our findings are similar to those across all biomedical conferences, which indicated that full publication was unlikely to be different between ophthalmology and all biomedical fields. Although lack of time is often cited as the main reason of not publishing conference abstracts,³² the differences we found across ophthalmology subspecialties suggests that other factors contribute as well, such as disease prevalence or difficulty publishing specific types of data (ie, surgical techniques or case series).^26,33

We did not find evidence that positive results were more likely associated with more frequent publications (ie, publication bias).^9,14 Our finding differs from the results from 45 reports identified in the review by Scherer et al¹ that found that studies with positive results were more likely to be published when defined as statistically significant (RR, 1.31; 95% CI, 1.23-1.40), or favoring the experimental intervention (RR, 1.17; 95% CI, 1.07-1.28). Two possible reasons for our study findings are that RCTs are more likely to be published than other study designs regardless of positive or nonpositive results and our meta-analysis of 2 reports is underpowered. Prior research also has found abstracts with significant results were published sooner than abstracts with nonsignificant results.³⁴ Further research is needed to examine whether publication bias is present in ophthalmology research.

Our review suggested abstracts presented orally and describing basic science research were associated with more frequent publication. These findings are consistent with the results by Scherer et al¹ and suggest that abstract review committees are able to select those abstracts that are more interesting or effective. It is also possible that presenters are encouraged by oral presentation success to focus on completing and submitting full-text manuscripts for publication. However, we found that other factors (ie, RCT and multicenter studies) were not significantly associated with full publication. Again, it is possible the small sample of reports included in the meta-analysis limited the power to detect significant differences.

Our review has several strengths. In this report, we summarized all reports that examined abstracts presented at eye- and vision-related conferences, evaluated the RoB for each report, conducted a weighted meta-analysis to estimate the proportion of abstracts published in full, and investigated factors associated with full publication in ophthalmology research. Our review was limited by the number of available reports in the literature and our inability to examine other factors associated with publication, such as conference location. The amount of heterogeneity requires cautious interpretation. Future studies should investigate underlying reasons for nonpublication in the eye and vision field with the aim of improving research dissemination. Possible motives for submitting an abstract with little thought for future publication may include obtaining support to attend a conference or funding incentives from university departments or postgraduate programs.

A recognition of the few abstracts subsequently published within 2 years is important for the field of eye and vision research. Nonpublication and delay of full-length publication may reflect insufficient motivation to pursue publication and/or presentation of low-quality abstracts unsuitable for publication. The failure to pursue publication of quality data may reflect a waste of limited resources (ie, research funding, travel expenses, and/or time away from other clinical/research activities). Moreover, the failure to publish studies involving people who consent to undergo experimental treatments with potential harms can be unethical. Nonpublication has been considered a form of scientific misconduct.³⁵ Failure to publish may be more likely to affect researchers/practitioners who are not able to attend conferences (eg, those from rural areas or lower-income economies). Presentation of low-quality material dilutes the effect of higher-quality research (which may be harder to find within conferences) and can potentially mislead conference attendees with incorrect findings. While some abstracts may be published in full after 2 years of conference presentation, the failure to disseminate research studies in peer-reviewed journals is not desired, especially when involving people.

The low proportion of conference presentation abstracts converted to full publication calls for further investigation to determine the factors that could be addressed to mitigate this poor outcome. There is a potential for harm that may result from low-quality information being promulgated at conferences and it is in the interest of patients and professionals to reverse this trend.

Accepted for Publication: March 15, 2020.

Corresponding Author: Roberta W. Scherer, PhD, Center for Clinical Trials and Evidence Synthesis, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, E6138, Baltimore, MD 21205 (rschere1@jhu.edu).

Published Online: April 30, 2020. doi:10.1001/jamaophthalmol.2020.1264

Author Contributions: Drs Scherer and Jian-Yu E had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: E, Li, Scherer.

Acquisition, analysis, or interpretation of data: E, Ramulu.

Drafting of the manuscript: E.

Critical revision of the manuscript for important intellectual content: E, Ramulu, Li, Scherer.

Statistical analysis: E.

Obtained funding: Li.

Administrative, technical, or material support: E, Li.

Supervision: Ramulu, Scherer.

Other - Methodology, Validation, Visualization: E.

Conflict of Interest Disclosures: Dr Ramulu reported grants from the National Institutes of Health as well as personal fees from Aerie Pharmaceuticals, W. L. Gore Inc, and Ivantis Inc outside the submitted work. Dr Li reported grants from the National Eye Institute and National Institutes of Health during the conduct of the study. No other disclosures were reported.

Funding/Support: The research was supported by grant UG1 EY020522 from the National Institutes of Health.

Role of the Funder/Sponsor: The funding organization had no role in the design and conduct of this research.