Variational inference in coupled models of amino acid substitution

Abstract

We investigate the use of Expectation-Maximization (O_SCPLOWEMC_SCPLOW) and variational Bayes for inferring rates and interactions under models of molecular coevolution. We first review O_SCPLOWEMC_SCPLOW theory for continuous-time Markov chains (O_SCPLOWCTMCC_SCPLOWs) and develop it for coevolutionary models, exploiting exchangeability and reversibility symmetries to constrain the parameter dimension. We fit several paired amino-acid coevolutionary models to structural alignments and compare the results to previous work. Our richest model trained on pooled coevolutionary data has explanatory power comparable to CherryMLs Q2 matrix (also trained on pooled data), with one quarter the parameters. However, we observe that aggregation of training data can lead to a form of Simpsons Paradox: a mixture model, whose components are parameter-efficient continuous-time Bayes networks (O_SCPLOWCTBNC_SCPLOWs), resolves signals that wash out when a single model tries to capture everything. These signals include both correlated and anticorrelated hydropathy and volume-packing in coevolving amino-acid pairs, as well as the anticorrelated acid/base compensation that was detected by CherryMLs Q2. We next present a closed-form evidence lower bound (ELBO) for O_SCPLOWCTBNC_SCPLOWs using O_SCPLOWEMC_SCPLOW statistics. Compared to the state of the art in variational modeling of O_SCPLOWCTBNC_SCPLOWs, the Euler-Lagrange equations derived by Cohn et al (JMLR, 2010), our closed-form ELBO is competitive in accuracy, considerably more efficient, simpler, and more stable. We conclude by describing a covariant indel model: a Dirichlet process selecting coevolving sites within TKF92, yielding an Infinite Pair HMM over alignments and structures. This model slightly outperforms TKF92 on a structural-alignment benchmark. Code and data are at https://tkfdp.net/.