Binning-based Methods for Model Fitting and Data Splitting Improve Machine Learning Imbalanced Regression

Abstract

Modelling quantitative genomics data remains challenging due to imbalanced target distributions, non-independent observations, and substantial computational costs. This study examines the challenges and current methods, proposes new methods, applies these methods across two case studies: a controlled setting with synthetically generated data and on real genomics datasets. Effective modelling of imbalanced data starts with density estimation. Density estimation via binning facilitates downstream analytical steps. Minimum Transformation Binning is a novel method that improves current binning methods for density estimation. Data splitting of imbalanced and non-independent observations is best accomplished by objective data splitting which entails randomly generating splits, scoring them, and selecting the best scoring one. This study conducts an empirical comparison of scoring functions toward an optimal objective data splitting method, finding the use of all-against-one absolute relative difference to be the best scoring function. Model fitting methods can be divided into loss functions and data balancing. Two methods have been previously proposed based on weighting error functions by the reciprocal or complement of the density estimation. This study conducts a comparison of these methods on the case studies, finding that reciprocal density weighting significantly outperforms non-weighted and complement density weighting. This study also tests model fitting with reciprocal weighted sampling per epoch for imbalanced regression. While this method does not generally improve performance, it uses significantly less resources in model fitting which can be beneficial for hyperparameter optimization. Together, this study provides a unified framework for effective modelling of imbalanced regression, especially imbalanced genomics data.