Continuous Integration and Software Quality in Scientific Software: A Large-Scale Empirical Analysis of GitHub Repositories
Walker J Compton MelloniD, Makena SyiD, Navya GangradeiD, Tobias JakobiiD
Abstract
Bioinformatics software plays a central role in biological research, yet many academic projects lack robust testing and continuous integration (CI) practices. This absence can compromise reproducibility, accessibility, and long-term maintainability. In this study, we conduct an empirical analysis of GitHub repositories spanning bioinformatics and other scientific software domains to examine the relationship between CI, software testing, and software quality. Repository discovery was performed using domain-specific search terms via GitHub's Application Programming Interface (API). This was followed by a layered data extraction pipeline, which included a Large Language Model (LLM) validation step, that connected repositories to published works, institutions, authors, funding, and citation counts. By integrating repository activity, community engagement, and backlog health into a composite Repository Quality Index (RQI), we find that the presence of CI and software tests is significantly associated with higher repository quality. Additionally, repositories with higher academic citation counts tend to cluster in higher RQI tiers, suggesting that software quality, which is driven by proper testing and integration, is associated with academic impact and funding status. These results demonstrate that adopting industry-standard development practices, such as CI and software testing, is a tractable path toward more sustainable and impactful academic software and funding opportunities.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 0
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- A large-scale analysis of bioinformatics code on GitHub
Pamela H. Russell, Rachel L. Johnson, … Nichole E. Carlson·PLoS ONEPeer-reviewed·Oct 31, 2018
- Sustained software development, not number of citations or journal choice, is indicative of accurate bioinformatic software
Paul P. Gardner, James M. Paterson, … Michael A. Black·Genome biologyPeer-reviewed·Feb 16, 2022
- Adding software to package management systems can increase their citation by 280%
Vahid Jalili, Dave Clements, … Jeremy Goecks·bioRxiv·Nov 16, 2020
- A multi-agent platform for assessment and improvement of bioinformatics software documentation
Anjun Ma, Shaohong Feng, … Qin Ma·bioRxiv·Feb 10, 2026
- FAIRsoft - A practical implementation of FAIR principles for research software
Eva Martín del Pico, Josep Lluis Gelpí, Salvador Capella-Gutiérrez·bioRxiv·May 4, 2022
- A Bioinformatician, Computer Scientist, and Geneticist lead bioinformatic tool development - which one is better?
Paul P. Gardner·Bioinformatics AdvancesPeer-reviewed·Dec 26, 2024