Byte-exact, format-aware compression for FASTA, FASTQ, BED and VCF files
Michail Patsakis, Alexandros MargarisiD, Theodore Chronopoulos, Ioannis MouratidisiD, Ilias Georgakopoulos‐SoaresiD
Abstract
Motivation Most omics data is stored by compressors that treat it as an unstructured byte stream. Format-aware compressors close that gap but are fragmented across formats and often expect a training pass over the user's own file. A general-purpose lossless compressor that is applicable to the most storage-intense omics data formats while beating the general codecs on ratio and speed would offer significant savings for large omics datasets storage and data retrieval. Results We present NYX, a compressor for FASTA, FASTQ, BED and VCF files. It leverages a reversible, format-specific transform optimised for the OpenZL compression algorithm to reach compression ratios up to 2.3 times those of the strongest general-purpose codecs (zstd -19, xz -9e and 7-Zip) while compressing up to two orders of magnitude faster. On the 1000 Genomes VCF dataset NYX reaches a compression ratio of 468.67x, 2.0 times that of xz -9e while compressing 1.8 times faster. On mammalian FASTA assemblies totalling 9.3 GB, NYX is within 3-4% of NAF, the strongest purpose-built FASTA codec, while compressing 57-73 times faster. Across all 54 inputs no single codec beats NYX on ratio and compression speed together: the tools that achieve higher compression are 1.2 to 73 times slower. NYX provides fully lossless, fast and efficient compression under one easy-to-use tool for the most common omics data types. Availability and implementation Source code, the shipped models, the benchmark harness and the scripts that regenerate every figure and table: https://github.com/Georgakopoulos-Soares-lab/compression
02 /Specification & Classification
Click any tag to browse matching papersReproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Nucleotide Archival Format (NAF) enables efficient lossless reference-free compression of DNA sequences
Kirill A. Kryukov, Mahoko Takahashi Ueda, … Tadashi Imanishi·BioinformaticsPeer-reviewed·Feb 22, 2019
- FASTR: Reimagining FASTQ via Compact Image-inspired Representation
Adrian Tkachenko, Sepehr Salem, … Mohammed Alser·bioRxiv·Jan 23, 2026
- GeneSqueeze: A Novel Lossless, Reference-Free Compression Algorithm for FASTQ/A Files
Foad Nazari, Sneh Patel, … Emma K. Murray·bioRxiv·Mar 21, 2024
- FaStore - a space-saving solution for raw sequencing data
Łukasz Roguski, Idoia Ochoa, … Sebastian Deorowicz·BioinformaticsPeer-reviewed·Mar 27, 2018
- Deep FASTQ and BAM co-compression in Genozip 15
Divon Mordechai Lan, Daniel Hughes, Bastien Llamas·bioRxiv·Jul 7, 2023
- Crumble: reference free lossy compression of sequence quality values
James Bonfield, Shane McCarthy, Richard Michael Durbin·BioinformaticsPeer-reviewed·Jul 9, 2018