Data Representation Bias and Conditional Distribution Shift Drive Predictive Performance Disparities in Multi-Population Machine Learning

Machine learning frequently encounters challenges when applied to population-stratified datasets, where data representation bias and data distribution shifts substantially impact model performance and generalizability across different population groups. These challenges are well illustrated in the context of polygenic prediction for diverse ancestry groups, and the underlying mechanisms are broadly applicable to machine learning with population-stratified data across domains. Using synthetic genotype-phenotype datasets representing five continental populations, we evaluate three approaches for utilizing population-stratified data, mixture learning, independent learning, and transfer learning, to systematically investigate how data representation bias and distribution shifts influence multi-population machine learning. Our results show that conditional distribution shifts, in combination with data representation bias, significantly influence machine learning performance across diverse populations and the effectiveness of transfer learning as a disparity mitigation strategy, while the effect of marginal distribution shifts is limited. The joint effects of data representation bias and distribution shifts demonstrate distinct patterns under different multi-population machine learning approaches, providing critical insights for the development of effective and equitable machine learning models for population-stratified data.