eLife Reviewed Preprints: Interview with Fiona Hutton
In October, the journal eLife announced that it will change how it handles peer review starting January 2023:
From next year, eLife is eliminating accept/reject decisions after peer review, instead focusing on public reviews and assessments of preprints.
To better understand what this change means for authors and reviewers, Upstream editor Martin Fenner asked Fiona Hutton, eLife's Head of Publishing, a few questions.
Can you briefly explain eLIfe’s new publishing model announced in October? eLife will be posting Reviewed Preprints (submitted preprints, alongside public peer review and an eLife assessment) on the eLife website; these Reviewed Preprints will contain a DOI and eLife citation. The eLife assessment details the reviewers’ and editors’ thoughts on the significance of the research and the strength of the evidence supporting the paper’s conclusions. These assessments will use common terminology that will be consistent for all articles in the journal that are assessed and the authors will be able to include a response to the assessment and reviews. We are also removing the binary accept/reject decision after peer review, with the focus now being on the public peer review and assessment of individual manuscripts. The author can then choose to revise their paper and receive updated reviews and/or an updated assessment, or choose to make their article the final version (Version of Record) with the original reviews and assessment. Authors will go through the new model in early 2023, and the publication fee will be reduced from $3,000 to $2,000.
What has been the feedback so far? We have had a huge amount of positive support for this model, from authors, funders, institutions and open science advocates. Many believe that although many organisations signed DORA, there has been a lack of movement and innovation in making those promises a reality and they see what we are doing as ground-breaking. Many are frustrated at the inertia in the current system and, at almost every open science event, everyone says the system is broken and we need an alternative. What we are doing at eLife is creating an alternative publishing model and providing an alternative output that can be used in research assessment. We are convinced that others will take up this model over time. Of course, we also know the current system is ingrained and that change can be difficult and cannot happen without strong support – we are in the fortunate position where our board is made up of funders that want the system to change and support eLife to lead that change. We also acknowledge that some people do not like or do not agree with the model and do not see the benefit of it. Instead of making judgements about a research paper based on the journal it was published in, we are asking the community to consider the substance of our reviews and editorial judgement, and that is a considerable step-change. It is the responsibility of eLife to show that the model can work and that there is a huge amount of value in it – and that’s where our priorities will be over the coming months.
What are the main problems you want to solve with this change? No one that supports science as an endeavour (including journal publishers), thinks that where something gets published is more important than what has been published, especially when getting into particular journals can often be the result of bias or chance. But where you get published has a huge influence on your academic research career and although there are a number of initiatives to try and address this problem, journal titles are often the quickest route to judge a candidate as part of any research assessment. If eLife can provide an alternate route so that the research is assessed on its individual merit (via a Reviewed Preprint and eLife assessment), rather than where it is published, and that output can be used as currency in research assessment, then we can focus on the production of genuinely impactful science. But there are other benefits – the existing publishing system is slow and although dissemination can be quick via preprints, the review of those preprints and subsequent acceptance into a journal can take time if subject to multiple rounds of revision. By quickly reviewing preprints, we can help researchers who require rapid evidence of their recent work in job, grant, or award applications. We can also provide rapid constructive assessments of that work for other researchers and readers. Crucially, by getting rid of that accept/reject decision, we are decoupling the scrutiny of the review process from the ability to get published. In doing so, we can stop turning recommendations from peer reviewers into requirements that authors must comply with in order to get accepted – a process that can also remove ideas and insights from scientific discourse. Our aim is to expose the nuanced and multidimensional nature of peer review, rather than reduce its function and output to a binary accept/reject decision.
What has been your experience only reviewing preprints since 2021? Very positive. In 2021, eLife began only reviewing preprints and asking our reviewers to write public versions of their peer reviews containing observations useful to readers. We have posted eLife reviews of more than 2,200 preprints to bioRxiv and medRxiv, along with a compact editorial assessment. We found that there were no changes in gender or geographic author demographic and it had no significant effect on reviewer recruitment and engagement. This initiative was the first step in the move to our new model and helped us develop the detail.
What proportion of submissions will be sent to external reviewers, and what will be the criteria? The criteria will change from 'is this suitable for publication in eLife' to ' is this an interesting article worth reviewing?'. Our purpose being to constructively review articles rather than fit them into a hierarchical journal construct. Our capacity however is limited by the size and interests of our editorial board, so the number of articles sent to review will naturally be limited. Currently, we send 30% of submissions to review but we assume this will change with the new model depending on how authors respond.
Will this new policy change the article types (https://reviewer.elifesciences.org/author-guide/types) that eLife publishes? No, we will continue to offer different article types.
Is content published as a Reviewed Preprint different from a Research Article? No, the content is the same. In this system, our editors and peer reviewers review the preprint, which is the version of the manuscript that has been deposited in a preprint server. The Reviewed Preprint will contain the public peer reviews and eLife assessment and will have an eLife citation and DOI. When our authors get to the end of the reviewing process and decide they want a final version (Version of Record), we carry out a series of additional checks to ensure that the data, methods, and code are made available, that appropriate reporting standards have been followed, that competing interest and ethics statements are complete, and that cell lines have been authenticated. It is the final Version of Record that we send to the indexers.
How is the new publishing model similar to or different from older publishing models based on preprints combined with peer review (e.g. Copernicus, F1000)? There are three main differences. 1) Peer review and assessment at eLife continues to be organised by an editorial team made up of academic experts and led by an Editor-in-Chief, Deputy Editors, Senior Editors, and a Board of Reviewing Editors via a consultative peer-review model already known as one of the most constructive for authors in the industry. 2) The addition of an eLife assessment is a further crucial part of our model, distinctive from what others are doing – it is a key addition to our public peer reviews and it enables readers to understand the context of the work, the significance of the research and the strength of the evidence. 3) We are no longer making accept/reject decisions based on peer review – authors will choose if and when to produce a Version of Record at any point following the review process.
What is your role at Life? My role is Head of Publishing. I manage the four publishing departments – Editorial, Production, Journal Development and Features – which function together to operate and publish the eLife journal. I also work with the technology, marketing, and communities teams to oversee the ongoing transition to the new model and work to drive innovation in publishing and in open initiatives. A further part of my role is to work with funders, libraries, external partners, and organisations in the open science community to ensure we strive in our mission to create real change in the practice of science and a future where a diverse, global community of researchers shares open results for the benefit of all.