Probabilistic clustering of cells using single-cell RNA-seq data

Abstract

Single-cell RNA sequencing is a modern technology for analyzing cellular heterogeneity. A key challenge is to cluster a heterogeneous sample of different cell types into multiple different homogeneous groups. Although there exist a number of clustering methods, they do not perform well consistently across various datasets. Moreover, most of them are not based on probabilistic approaches making it difficult to assess uncertainties in their results. Therefore, in spite of having large cell atlases, it is often quite difficult to map cells to types. In addition, many of the methods require prior knowledge such as marker gene information for each type. Also due to technological limitations, dropouts of gene expressions may occur in the data which is not taken into account in other methods. Here we present a probabilistic method named CellHorizon for clustering scRNA-seq data that is based on a generative model, handles dropouts and works without any prior marker gene information. Experiments reveal that our method outperforms current state-of-the-art methods overall on six gold standard datasets.