Performance of Sequential Markovian Coalescence Methods when Populations are Structured

Abstract

Sequentially Markovian Coalescent (SMC) methods are widely used to reconstruct demographic histories from genomic data, yet their accuracy in the presence of population structure has not been systematically evaluated. Here we assessed the performance of two popular SMC based algorithms (PSMC and SMC++) in retrieving the inverse instantaneous coalescent rate (IICR) simulated under two structured equilibrium scenarios, namely the finite island (FIM) and the two-dimensional stepping-stone (2D-SST). Looking backward in time, the IICR of a couple of lineages collected in a deme is characterized by a short period of constant value (the scattering phase) followed by an increase over time (the transition phase) up to a plateau (the collecting phase). Both algorithms recovered the IICR at the collecting phase but significantly deviated from the values expected during the transition phase under specific parameters combination. Specifically, the total error increased with the number of demes and the abruptness of the transition phase, which in turn is related to the extent of between demes connectivity. Moreover, each algorithm displayed a specific bias, corresponding backward in time to an artificial expansion either at the beginning of the collecting phase (PSMC) or of the transition phase (SMC++). Our results demonstrate that the observed systematic biases carry information about metapopulation dynamics, rather than being simple artefacts. Moreover, combining the results obtained from the two algorithms will increase our understanding of the historical demography of the metapopulations and help to put forward specific evolutionary scenarios to be tested with model-based algorithms using SFS or LD statistics.