Peer Review and Scientific Publication at a Crossroads: Call for Research for the 10th International Congress on Peer Review and Scientific Publication

The way science is assessed, published, and disseminated has markedly changed since 1986, when the launch of a new Congress focused on the science of peer review was first announced. There have been 9 International Peer Review Congresses since 1989, typically running on an every-4-year cycle, and most recently in 2022 after a 1-year delay due to the COVID-19 pandemic.¹ Here, we announce that the 10th International Congress on Peer Review and Scientific Publication will be held in Chicago, Illinois, on September 3-5, 2025.

The congresses have been enormously productive, incentivizing and publicizing important empirical work into how science is produced, evaluated, published, and disseminated.^2-4 However, peer review and scientific publication are currently at a crossroads and their future more difficult than ever to predict. After decades of experience and research in these fields, we have learned a lot about a wide range of aspects of peer review and scientific publication.^2-5 We have accumulated a large body of empirical evidence on how systems function and how they can malfunction. There is also growing evidence on how to make peer review, publication, and dissemination processes more efficient, fair, open, transparent, reliable, and equitable.^6-15 Experimental randomized evaluations of peer review practices are only a small part of the literature, but their numbers have been growing since the early trials of anonymized peer review.^16-22 Research has revealed a rapidly growing list of biases, inefficiencies, and threats to the trustworthiness of published research, some now well recognized, others deserving of more attention.^2,3 Moreover, practices continue to change and diversify in response to new needs, tools, and technologies as well as the persistent “publish or perish” pressures on scientists-as-authors.

With the continued evolution of electronic platforms and tools—most recently the emergence and use of large language models and artificial intelligence (AI)—peer review and scientific publication are rapidly evolving to address new opportunities and threats.^23,24 Moreover, a lot of money is at stake; scientific publishing is a huge market with one of the highest profit margins among all business enterprises, and it supports a massive biomedical and broader science economy. Many stakeholders try to profit from or influence the scientific literature in ways that do not necessarily serve science or enhance its benefits to society. The number of science journal titles and articles is steadily increasing²⁵; many millions of scientists coauthor scientific papers, and perverse reward systems do not help improve the quality of this burgeoning corpus. Furthermore, principled mandates for immediate and open access to research and data may not be fully understood, accepted, or funded. Many other new, often disruptive, ideas abound on how to improve dissemination of and access to science, some more speculative, utopian, or self-serving than others. In addition, deceptive, rogue actors, such as predatory and pirate publishers, fake reviewers, and paper mills continue to threaten the integrity of peer review and scientific publication. Careful testing of the many proposals to improve peer review and publication and of interventions and processes to address threats to their integrity in a rigorous and timely manner are essential to the future of science and the scholarly publishing enterprise.

Proposed remedies for several of the problems and biases have been evaluated,⁴ but many are untested or have inconclusive evidence for or against their use. New biases continue to appear (or at least to be recognized). In addition, there is tension about how exactly to correct the scientific literature, where a large share of what is published may not be replicable or is obviously false.²⁶ Even outright fraud may be becoming more common—or may simply be recognized and reported more frequently than before.^27,28

By their very nature, peer review and scientific publication practices are in a state of flux and may be unstable as they struggle to serve rapidly changing circumstances, technologies, and stakeholder needs and goals. Therefore, some unease would exist even in the absence of major perturbations, even if all the main stakeholders (authors, journals, publishers, funders) simply wanted to continue business as usual. However, the emergence of additional rapid changes further exacerbates the challenges, while also providing opportunities to improve the system at large. The COVID-19 crisis was one major quake that shook the way research is designed, conducted, evaluated, published, disseminated, and accessed.^29,30 Advances in AI and large language models may be another, potentially even larger, seismic force, with some viewing the challenge posed by these new developments as another hyped tempest in a teapot and others believing them to be an existential threat to truth and all of humanity. Scientific publication should fruitfully absorb this energy.^23,24 Research has never been needed more urgently to properly examine, test, and correct (in essence: peer review) scientific and nonscientific claims for the sake of humanity’s best interests. The premise of all Peer Review Congresses is that peer review and scientific publication must be properly examined, tested, and corrected in the same way the scientific method and its products are applied, vetted, weighted, and interpreted.²

The range of topics on which we encourage research to be conducted, presented, and discussed at the 10th International Congress on Peer Review and Scientific Publication expands what was covered by the 9 previous iterations of the congress (Box).^1,2,4 We understand that new topics may yet emerge; 2 years until September 2025 is a relatively long period, during which major changes are possible, and even likely. Therefore, we encourage research in any area of work that may be relevant to peer review and scientific publication, including novel empirical investigations of processes, biases, policies, and innovations. The congress has the ambitious goal to cover all branches and disciplines of science. It is increasingly recognized that much can be learned by comparing experiences in research and review practices across different disciplines. While biomedical sciences have had the lion’s share in empirical contributions to research on peer review in the past, we want to help correct this imbalance. Therefore, we strongly encourage the contribution of work from all scientific disciplines, including the natural and physical sciences, social sciences, psychological sciences, economics, computer science, mathematics, and new emerging disciplines. Interdisciplinary work is particularly welcome.

The congress is organized under the auspices of JAMA and the JAMA Network, The BMJ, and the Meta-research Innovation Center at Stanford (METRICS) and is guided by an international panel of advisors who represent diverse areas of science and of activities relevant to peer review and scientific publication.⁴ The abstract submission site is expected to open on December 1, 2024, with an anticipated deadline for abstract submission by January 31, 2025. Announcements will appear on the congress website (https://peerreviewcongress.org/).⁴

Corresponding Author: John P. A. Ioannidis, MD, DSc, Stanford Prevention Research Center, Stanford University, 1265 Welch Rd, MSOB X306, Stanford, CA 94305 (jioannid@stanford.edu).

Published Online: September 22, 2023. doi:10.1001/jama.2023.17607

Conflict of Interest Disclosures: All authors serve as directors or coordinators of the Peer Review Congress. Ms Flanagin reports serving as an unpaid board member for STM: International Association of Scientific, Technical, and Medical Publishers. Dr Bloom reports being a founder of medRxiv and a member of the Board of Managers of American Institute of Physics Publishing.

Additional Information: Drs Ioannidis and Berkwits are directors; Ms Flanagin, executive director; and Dr Bloom, European director and coordinator for the International Congress on Peer Review and Scientific Publication.

Note: This article is being published simultaneously in The BMJ and JAMA.