Bioinformatics Pipeline using JUDI: Just Do It

Abstract

Large-scale data analysis in Bioinformatics requires executing several software in a pipelined fashion. Generally each stage in a pipeline takes considerable computing resources and several workflow management systems (WMS), e.g., Snakemake and Nextflow, have been developed to ensure in case of multiple invocation of the pipeline, only the bare minimum stages that are affected by the changes across invocations get executed. However, when the pipeline needs to be executed with different settings of parameters, e.g., thresholds, underlying algorithms, etc. these WMS require significant scripting to ensure an optimal execution of the pipeline. We developed JUDI on top of a Python based WMS, DoIt, for a systematic handling of pipeline parameter settings based on the principles of DBMS that simplifies plug-and-play scripting. The effectiveness of JUDI is demonstrated in a pipeline for analyzing large scale HT-SELEX data for transcription factor and DNA binding where JUDI reduces scripting by a factor of five.\n\nAvailabilityhttps://github.com/ncbi/JUDI