How random is the review outcome? A systematic study of the impact of external factors on eLife peer review

Abstract

The advance of science rests on a robust peer review process. However whether or not a paper is accepted can depend on random external factors--e.g. the timing of the submission, the matching of editors and reviewers--that are beyond the quality of the work. This article systematically investigates the impact of these random factors independent of the papers quality on peer review outcomes in a major biomedical journal, eLife. We analyzed all of the submissions to eLife between 2016 to 2018, with 23,190 total submissions. We examined the effects of random factors at each decision point in the review process, from the gatekeeping senior editors who may desk-reject papers to review editors and reviewers who recommend the final outcome. Our results suggest that the peer-review process in eLife is robust overall and that random external factors have relatively little quantifiable bias.