The serotonin 1B receptor modulates striatal activity differentially based on behavioral context

The dorsomedial striatum (DMS) is critical for goal-directed behavior and has been implicated in both motivating and inhibiting behavioral responses. The DMS circuitry is complex as it integrates multiple inputs from the cortex, thalamus, and other subcortical structures including midbrain dopamine neurons. Though less studied, serotonin neurons from the dorsal raphe nucleus also richly innervate the DMS, which expresses the majority of the 14 receptor subtypes for serotonin. In particular, slice electrophysiology shows that the serotonin 1B receptor (5-HT1BR) impacts DMS physiology and plasticity, and behavioral experiments show that 5-HT1BR expression modulates impulsivity and other DMS-dependent reward-related behaviors. In these studies, our goal was to investigate the effects of 5-HT1BR on the DMS in vivo during goal-directed behavior in mice. Using a genetic 5-HT1BR loss-of-function model, we examined the calcium activity of individual medium spiny neurons (MSNs) in the DMS during operant tasks of responding and waiting. We found that knockout of 5-HT1BRs resulted in a significant reduction of excitatory calcium responses and an increase in the proportion of cells with inhibitory calcium responses following receipt of a reward. This suggests that serotonin may recruit MSN activity in response to reward via actions at 5-HT1BRs. On the other hand, in a behavioral paradigm designed to test impulsivity, we found that serotonin may inhibit DMS calcium activity through 5-HT1BRs. Specifically, mice lacking 5-HT1BR expression had a larger proportion of cells showing increased calcium responses during the waiting period of the trial, compared to controls. These results point to the importance of in vivo studies to understand the functional role of DMS serotonin in reward-related behavior. Overall our results demonstrate that serotonin can modulate the DMS in a behavioral state-specific manner, potentially providing a mechanism for how serotonin effects on behavior are context-dependent.