Reproducibility in systems biology modelling

Abstract

The reproducibility crisis has emerged as an important concern across many fields of science including life science, since many published results failed to reproduce. Systems biology modelling, which involves mathematical representation of biological processes to study complex system behaviour, was expected to be least affected by this crisis. While lack of reproducibility of experimental results and computational analysis could be a repercussion of several compounded factors, it was not fully understood why systems biology models with well-defined mathematical expressions fail to reproduce and how prevalent it is. Hence, we systematically attempted to reproduce 455 kinetic models of biological processes published in peer-reviewed research articles from 152 journals; which is collectively a work of about 1400 scientists from 49 countries. Our investigation revealed that about half (49%) of the models could not be reproduced using the information provided in the published manuscripts. With further effort, an additional 12% of the models could be reproduced either by empirical correction or support from authors. The other 37% remained non-reproducible models due to missing parameter values, missing initial concentration, inconsistent model structure, or a combination of these factors. Among the corresponding authors of the non-reproducible model we contacted, less than 30% responded. Our analysis revealed that models published in journals across several fields of life science failed to reproduce, revealing a common problem in the peer-review process. Hence, we propose an 8-point reproducibility scorecard that can be used by authors, reviewers and journal editors to assess each model and address the reproducibility crisis.