An explicit birth-death-reticulation model for studying the diversification of phylogenetic networks

Much of the history of life is reticulate and better represented by phylogenetic networks than by strictly bifurcating trees. Understanding the processes that generated that history thus requires models of diversification (speciation and extinction) that incorporate reticulation. However, we currently lack tractable reticulate diversification models. Here we develop a simple birth-death-reticulation model that includes unidirectional and bidirectional gene flow, homoploid hybrid speciation, and allopolyploidization, and derive a practical probability density function for networks under this model. We demonstrate that the model can extract information about reticulation processes from known phylogenetic networks. We also explore the empirical utility of the model using an allopolyploid network of ferns of the family Cystopteridaceae, revealing evidence in favor of the controversial hypothesis that polyploids have lower diversification rates than their diploid relatives. While the model represents an advance in our ability to learn about the diversification of reticulate lineages, we also identify significant statistical, computational, and empirical challenges that face this nascent framework.