Sign in
EcologybioRxiv
Preprint

Developing a modern data workflow for evolving data

Glenda M. YenniiD, Erica M. ChristenseniD, Ellen K. BledsoeiD, Sarah R. SuppiD, Renata M. DiaziD, Ethan P. WhiteiD, S.K. Morgan ErnestiD

Abstract

Data management and publication are core components of the research process. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. \"Evolving data\" present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a evolving data workflow for a long-term ecological study that addresses many of the challenges associated with managing this type of data. We do this by leveraging existing tools to: 1) perform quality assurance and control; 2) import, restructure, version, and archive data; 3) rapidly publish new data in ways that ensure appropriate credit to all contributors; and 4) automate most steps in the data pipeline to reduce the time and effort required by researchers. The workflow uses two tools from software development, version control and continuous integration, to create a modern data management system that automates the pipeline.

PDFDOI

02 /Specification & Classification

Click any tag to browse matching papers

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Similarity Map
SelectedSame fieldCross-field
THISReiter '21Griffin '17Darriba '18Kosmala '16Finak '18Novella '18
55% semantic matchBioinformatics
Taylor E. Reiter, Phillip Brooks, Luiz C. Irber+5 more·GigaScience·Jan 1, 2021· 60 citations