Task-similarity dependent reconfiguration of compositional modules and geometry in frontal cortex

Abstract

Task rules critically shape how neural populations encode information and generalize across cognitive demands, yet the mechanisms linking rule structure, neural dynamics, and behavior remain poorly understood. Here we investigate how rule congruence modulates neural-ensemble compositionality, geometry and cross-task generalizability in mouse anterior lateral motor cortex (ALM). Mice switched between tactile and auditory delayed-response tasks where sensory-motor contingencies were either aligned (congruent) or conflicting (incongruent). Recording from 11,000 ALM neurons revealed rule dependent compositional coding of stimulus, choice, and outcome. In contrast to congruent switching, incongruent rules reduced cross-task generalizability of sensory, choice and outcome representations, expanded population dimensionality, increased reliance on nonlinear mixed-selectivity, and produced more complex coding geometry. These neural changes paralleled behavioral costs during task transitions. Moreover, congruence dependent reconfiguration of sensory-choice subspaces in ALM predicted differences in transition costs, implicating ALM in adaptively resolving conflicting contingencies. Together, these results show that task congruence regulates neural coding geometry and population dynamics, linking rule-switching complexity to dimensionality of neural resource allocation and to cognitive flexibility.