A Space-Time Hidden Markov Model for In Vivo NanoScale Synaptic Plasticity Tracking

Abstract

Synapses are the fundamental unit of neural connectivity, exhibiting dynamic functional and structural changes that enable the brain to learn, adapt, and form memories. Recent advances in fluorescent labeling of endogenous proteins offer an opportunity to image synaptic strength in vivo and study mechanisms underlying adaptive neural computation. Studying synaptic dynamics requires tracking signals of small, densely packed synapses over days as they change in size, position, and intensity between imaging sessions, and may even appear or disappear. Associating >50,000 dynamic, submicrometer particles across time is difficult, even for state-of-the-art algorithms. Moreover, most algorithms assign equal weight to the lateral (XY) and noisier axial (Z) dimensions, reducing performance. To address these challenges and accurately track synapses in vivo, we developed SynTrack. We formulate tracking as a Maximum A Posteriori estimation problem that identifies the K most likely disjoint paths in a Hidden Markov Model, solved using min-cost circulation optimization. An anisotropic uncertainty model accounts for poorer axial resolution, and a fully temporally connected spatio-temporal graph overcomes long-term occlusions. SynTrack achieves a mean displacement of 0.50 {micro}m with a Multiple Object Tracking Accuracy (MOTA) score of 89.8%, on par with expert annotators but with substantially increased speed and scalability. In a large-scale volume imaged over two weeks, SynTrack reconstructed 74,000 synapse trajectories detected in 4.9 out of 8 imaging sessions on average, with 18,000 synapses tracked in at least seven sessions. We present a state-of-the-art algorithm capable of high-fidelity longitudinal tracking of individual synapses in behaving mice at an unprecedented scale.