Dissecting oligogenic and polygenic indirect genetic effects through the lens of neighboring plant genotypes

Individual phenotypes often depend on the genotypes of other individuals within a group. These phenomena are termed "indirect genetic effects" (IGEs) and have been distinguished from direct genetic effects (DGEs) using quantitative genetic models. Recent studies have utilized high-resolution polymorphism data to enable genomic prediction (GP) and genome-wide association study (GWAS) of IGEs. To accelerate this practice, here we integrate polygenic and oligogenic IGEs using a multi-kernel mixed model. This model incorporates two random effects with a single covariance parameter for polygenic effects, while including neighbor allelic similarity to test SNP-marker effects for oligogenic effects. Our simulations demonstrated that the present model could infer a trade-off between DGEs and IGEs with a slight improvement in GWAS/GP performance. By applying our method to three species of woody plants, we found evidence for intergenotypic competition in aspen and apple trees, but limited evidence in climbing grapevines. Based on GWAS, we also detected significant variants associated with competitive IGEs on the apple trunk growth. Our study offers an alternative implementation for GWAS/GP of IGEs and thereby provides an effective tool to dissect the genetic architecture of mixing ability within a plant species.