Stochastic Evolutionary Control in Heterogeneous Populations

Abstract

Therapeutic resistance, which poses a central challenge in cancer and infectious disease treatment, arises from the evolutionary dynamics of heterogeneous populations as a natural consequence of the evolutionary tendency towards higher fitness. Here we introduce SHEPHERD (Stochastic Heterogeneity-informed Evolutionary Policy Hampering the Expansion of Resistance to Drugs), which integrates Wright-Fisher (WF) population genetics modeling with Markov Decision Processes (MDPs) to design optimal adaptive drug policies which reduce the fitness of evolving disease, mitigating resistance. While MDPs have been applied to evolutionary control under the strong-selection, weak-mutation (SSWM) assumption--where populations are effectively homogeneous--SHEPHERD operates beyond this regime, effectively capturing the full stochastic dynamics of genetically heterogeneous populations experiencing mutation and selection of a wide array of strengths. Using synthetic multi-drug fitness landscapes, we show the benefit of the optimized SHEPHERD protocol for in silico disease systems with three, four, and eight genotypes: the long-term mean fitness of the disease is reduced compared to any single-drug or two-drug periodic regimen. Although these examples specifically model drug resistance, we discuss the potential broad applicability of our approach to calculating strategies for controlling a wide class of evolving populations. We further analyze the sensitivity of SHEPHERD strategies to different levels of genotypic frequency discretization and temporal resolution, demonstrating robustness once moderate resolution is achieved, but strong dependence on the timing of drug updates. Although current results rely on synthetic landscapes due to the lack of empirical multi-genotype, multi-drug measurements outside the SSWM regime, our study establishes a foundation for applying MDP optimization to broad classes of evolutionary dynamics, highlighting both the opportunities and the computational challenges of controlling evolution.