Developing a modern data workflow for evolving data
Glenda M. YenniiD, Erica M. ChristenseniD, Ellen K. BledsoeiD, Sarah R. SuppiD, Renata M. DiaziD, Ethan P. WhiteiD, S.K. Morgan ErnestiD
Abstract
Data management and publication are core components of the research process. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. \"Evolving data\" present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a evolving data workflow for a long-term ecological study that addresses many of the challenges associated with managing this type of data. We do this by leveraging existing tools to: 1) perform quality assurance and control; 2) import, restructure, version, and archive data; 3) rapidly publish new data in ways that ensure appropriate credit to all contributors; and 4) automate most steps in the data pipeline to reduce the time and effort required by researchers. The workflow uses two tools from software development, version control and continuous integration, to create a modern data management system that automates the pipeline.
02 /Specification & Classification
Click any tag to browse matching papersReproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Streamlining Data-Intensive Biology With Workflow Systems
Taylor E. Reiter, Phillip Brooks, … N. Tessa Pierce‐Ward·GigaSciencePeer-reviewed·Jan 1, 2021
- Best Practice Data Life Cycle Approaches for the Life Sciences
Philippa C. Griffin, Jyoti R. Khadake, … Maria Victoria Schneider·F1000ResearchPeer-reviewed·Aug 31, 2017
- The State of Software in Evolutionary Biology
Diego Darriba, Tomas Flouri, Alexandros Stamatakis·Molecular Biology and EvolutionPeer-reviewed·Jan 29, 2018
- Assessing data quality in citizen science - preprint
Margaret Kosmala, Andrea Wiggins, … Brooke Simmons·bioRxiv·Sep 8, 2016
- DataPackageR: Reproducible data preprocessing, standardization and sharing using R/Bioconductor for collaborative data analysis.
Greg Finak, Bryan T. Mayer, … Raphaël Gottardo·Gates Open ResearchPeer-reviewed·Jun 22, 2018
- Container-based bioinformatics with Pachyderm
Jon Ander Novella, Payam Emami Khoonsari, … Ola Spjuth·BioinformaticsPeer-reviewed·Aug 7, 2018