Optimal transport modeling uncovers spatial domain dynamics in spatiotemporal transcriptomics studies

Spatiotemporal transcriptomics is an emerging and powerful approach that adds a temporal dimension to spatial transcriptomics, enabling the characterization of dynamic changes in tissue architecture during development or disease progression. Here, we present SpaDOT (Spatial Domain Transition detection), a computational method designed to identify spatial domains and infer their temporal dynamics across time points for spatiotemporal transcriptomics studies. SpaDOT employs a variational autoencoder (VAE) framework to capture a low-dimensional representation of the data, incorporating hidden clustering variables to define distinct spatial domains. In the process, SpaDOT integrates a Gaussian Process prior and graph neighbor information to explicitly model both global spatial continuity and local structural heterogeneity among spatial locations within each time point, while using optimal transport (OT) to derive time-varying embeddings and infer the relationships between spatial domains across time points. Through simulation and real data applications, we demonstrate the superiority of SpaDOT in spatial domain detection, latent space preservation, and domain transition tracking over time. In real data applications, SpaDOT accurately captures dynamic domain transitions, including the disappearance and re-emergence of domains due to technical variation. Notably, SpaDOT uncovers key aspects of valvulogenesis in the developing heart, revealing the splitting of a single valve structure into two distinct functional valves -- the atrioventricular valve and the semilunar valve.