The effects of demography and genetics on the neutral distribution of quantitative traits

1Neutral models for quantitative trait evolution are useful for identifying phenotypes under selection in natural populations. Models of quantitative traits often assume phenotypes are normally distributed. This assumption may be violated when a trait is affected by relatively few genetic variants or when the effects of those variants arise from skewed or heavy-tailed distributions. Traits such as gene expression levels and other molecular phenotypes may have these properties. To accommodate deviations from normality, models making fewer assumptions about the underlying trait genetics and patterns of genetic variation are needed. Here, we develop a general neutral model for quantitative trait variation using a coalescent approach by extending the framework developed by SO_SCPLOWCHRAIBERC_SCPLOW and LO_SCPLOWANDISC_SCPLOW (2015). This model allows interpretation of trait distributions in terms of familiar population genetic parameters because it is based on the coalescent. We show how the normal distribution resulting from the infinitesimal limit, where the number of loci grows large as the effect size per mutation becomes small, depends only on expected pairwise coalescent times. We then demonstrate how deviations from normality depend on demography through the distribution of coalescence times as well as through genetic parameters. In particular, population growth events exacerbate deviations while bottlenecks reduce them. This model also has practical applications, which we demonstrate by designing an approach to simulate from the null distribution of QST, the ratio of the trait variance between subpopulations to that in the overall population. We further show that it is likely impossible to distinguish sparsity from skewed or heavy-tailed distributions of mutational effects using only trait values sampled from a population. The model analyzed here greatly expands the parameter space for which neutral trait models can be designed.