pbcftools: parallel execution of bcftools for large variant call sets
Abstract
Summarybcftools is the standard toolkit for handling VCF and BCF variant files, but it processes records on a single core; its --threads option speeds up only compression of the output, not the work done on variant records. Processing large call sets is therefore slow, and users often divide the genome and reassemble the results by hand. We present pbcftools, a Perl wrapper that does this automatically: it splits the genome into chunks, runs an ordinary bcftools command on each in parallel, and reassembles the outputs by a method suited to the data type. Across Linux servers, Windows/WSL2 workstations and Apple laptops, with bcftools 1.21 to 1.24, parallel output was identical to serial output for every command tested. On 1000 Genomes Phase 3 data, operations writing compressed VCF ran 10.8 to 21.1 times faster with 32 cores and up to 35.4 times with 64, those writing text 3.7 to 12.8 times, and merging 100 VCF files 19.2 times. pbcftools also runs on LSF and Slurm clusters. Availability and implementationpbcftools is written in Perl (>= 5.16) and requires bcftools; local parallel execution also requires Perl module Parallel::ForkManager. It is released under the MIT license at https://github.com/zhangge-uc/pbcftools (DOI: 10.5281/zenodo.21780361).
02 /Specification & Classification
Click any tag to browse matching papersReproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- vcfilt: A Zero-Allocation Streaming Filter for High-Throughput VCF Processing
Muhammed Murshid KP·bioRxiv·Apr 16, 2026
- VariantFlow: a selective-execution engine for efficient population genomic computation on large variant datasets
Ehsan Estaji, Jian‐Feng Mao·bioRxiv·Sep 4, 2026
- vcfpp: a C++ API for rapid processing of the Variant Call Format
Zilong Li·BioinformaticsPeer-reviewed·Jan 23, 2024
- Vcfexpress: flexible, rapid user-expressions to filter and format VCFs
Brent S. Pedersen, Aaron R. Quinlan·BioinformaticsPeer-reviewed·Mar 3, 2024
- Analysis-ready VCF at Biobank scale using Zarr
Eric Czech, Will Tyler, … Jerome Kelleher·GigaSciencePeer-reviewed·Jan 1, 2025
- FM3VCF: A Software Library for Accelerating the Loading of Large VCF Files in Genotype Data Analyses
Zhen Zuo, Mingliang Li, … You Xi Tang·PLoS ONEPeer-reviewed·Jun 4, 2025