Coadapted genomes and selection on hybrids: Fisher’s geometric model explains a variety of empirical patterns

Abstract

Natural selection plays a variety of roles in hybridization, speciation and admixture. Most research has focused on two extreme cases: crosses between closely-related inbred lines, where hybrids are fitter than their parents, or crosses between effectively isolated species, where hybrids suffer severe breakdown. Many natural populations must fall into intermediate regimes, with multiple types of gene interaction, but these are more difficult to study. Here, we develop a simple fitness landscape model, and show that it naturally interpolates between previous modeling approaches, involving mildly deleterious recessives, or discrete hybrid incompatibilities. The model yields several new predictions, which we test with genomic data from Mytilus mussels, and published data from plants (Zea, Populus and Senecio) and animals (Mus, Teleogryllus and Drosophila). The predictions are generally supported, and the model explains surprising empirical patterns that have been observed in both extreme regimes. Our approach enables novel and complementary uses of genome-wide datasets, which do not depend on identifying outlier loci, or \"speciation genes\" with anomalous effects. Given its simplicity and flexibility, and its predictive successes with a wide range of data, the approach should be readily extendable to other outstanding questions in the study of hybridization.