Foldcomp: a library and format for compressing and indexing large protein structure sets

Abstract

Highly accurate protein structure predictors have generated hundreds of millions of protein structures; these pose a challenge in terms of storage and processing. Here we present Foldcomp, a novel lossy structure compression algorithm and indexing system to address this challenge. By using a combination of internal and cartesian coordinates and a bi-directional NeRF-based strategy, Foldcomp improves the compression ratio by a factor of 3 compared to the next best method. Its reconstruction error of 0.08[A] is comparable to the best lossy compressor. It is 5 times faster than the next fastest compressor and competes with the fastest decompressors. With its multi-threading implementation and a Python interface that allows for easy database downloads and efficient querying of protein structures by accession, Foldcomp is a powerful tool for managing and analyzing large collections of protein structures. AvailabilityFoldcomp is a free open-source library and command-line software available for Linux, macOS and Windows at https://foldcomp.foldseek.com. Foldcomp provides the AlphaFold Swiss-Prot (2.9GB), TrEMBL (1.1TB) and ESMatlas HQ (114GB) database ready-for-download.