How different AI models understand cells differently

Abstract

AI single-cell foundation models (scFMs) are believed to be able to learn essential relations in cell transcriptomics with the attention modules in Transformer, but there is no method to reveal what they actually learned. We observed that different models may grasp different aspects of relations. To unravel the mystery, we propose scGeneLens, a framework for dissecting how scFMs perceive cells. We employed a sparse block attention to replace the original attention mechanism to concentrate attentions into a few dominant gene-gene relations, used attention propagation to trace how the relations propagate across Transformer layers, and used integrated gradients to disentangle the relative contributions of gene identity and expression in cell representations. We applied it to scFoundation and scGPT and show that they exhibit pluralistic perceptions of cells: scFoundation emphasizes relations among cell-type marker genes, resulting in stronger cell-type separability, whereas scGPT focuses more on genes involved in shared cellular pathways and core biological activities, leading to representations that generalize across conditions. The framework provides a unified lens for probing what scFMs learn about cells and offers actionable insights for the design of future cellular foundation models. Our code can be seen in https://anonymous.4open.science/r/scGeneLens-B771/.