Assortative mating can impede or facilitate fixation of underdominant alleles

Abstract

Although underdominant mutations have undoubtedly fixed between divergent species, classical models of population genetics suggest underdominant alleles should be purged quickly, except in small or subdivided populations. Here we study the fixation of underdominant alleles at loci that also influence mate choice, such as loci encoding coloration patterns visible to mates and predators alike. We analyze a mechanistic model of positive assortative mating in which individuals have n chances to sample compatible mates. This one-parameter model naturally spans the two classical extremes of random mating (n = 1) and complete assortment (n [->] {infty}), and yet it produces a complex form of sexual selection that depends non-monotonically on the number of mating opportunities, n. The resulting interaction between viability selection and sexual selection can either inhibit or facilitate fixation of underdominant alleles, compared to random mating. As the number of mating opportunities increases, underdominant alleles can fix at rates that even approach the neutral substitution rate. This result is counterintuitive because sexual selection and underdominance each suppress rare alleles in this model, and yet in combination they can promote the fixation of rare alleles. This phenomenon constitutes a new mechanism for the fixation of underdominant alleles in large populations, and it illustrates how incorporating life history characteristics can alter the predictions of population-genetic models for evolutionary change.