FoldToken3: Fold Structures Worth 256 Words or Less

Abstract

Protein structure tokenization has attracted increasing attention in both protein representation learning and generation. While recent work, like FoldToken2 and ESM3, has achieved good reconstruction performance, the compressoin ratio is still limited. In this work, we propose FoldToken3, a novel protein structure tokenization method that can compress protein structures into 256 tokens or less and ensure the reconstruction quality comparable to FoldToken2. To the best of our knowledge, FoldToken3 is the most efficient, light-weight, and compression-friendly protein structure tokenization method. And it will benifit a wide range of protein structure-related tasks, such as protein structure alignment, generation, and representation learning. The work is still in progress and the code will be available upon acceptance.