Species-specific small models for cell type classification approach the performance of large single cell foundation models

Abstract

Accurate cross-species cell type classification remains a key evaluation task in single-cell transcriptomics. Recent foundation models trained on millions of single-cell profiles demonstrate great in-distribution and out-of-distribution performance on this task, but their large parameter counts and substantial computational costs limit accessibility and interpretability. Here, we introduce CytoType, a simple and interpretable model for cell type classification that leverages pre-trained ESM-2 protein embeddings of protein-coding transcripts. By learning linear, cell-type-specific weights over transcript embeddings, without relying on gene count information, CytoType achieves F1 scores comparable to or exceeding those of large-scale transformer-based models. We further developed ESM-Cell Embedding (ESM-CE), an even simpler variant that only averages ESM-2 embeddings across expressed genes, which also performs competitively against foundation models. Both CytoType and ESM-CE are trained on species-specific data, maintaining high accuracy when classifying cell types with orders of magnitude fewer parameters compared to larger foundation models. For example, for human tissues, the average performance gap between CytoType and the best foundation model was 0.053 F1 points while CytoType uses 10,000x fewer trainable parameters. Additionally, we quantified the contribution of ESM-2 embeddings to cell type classification tasks and demonstrated a three fold reduction in the performance gap between CytoType and the best foundation model for nine species. Finally, we show that CytoTypes learned gene weights are biologically interpretable.