Tensor Factorization-based Prediction with an Application to Estimating the Risk of Chronic Diseases

Abstract

Tensor factorization has emerged as a powerful method to address the challenges of high dimensionality regarding disease development and comorbidity. Chronic diseases have a high likelihood to co-occur, making patients suffering from one chronic disease to have an elevated risk for the other diseases in the course of aging. Individualized prediction of chronic diseases can help patients prevent new diseases and reduce the healthcare costs. Despite rich results of risk assessment models for chronic diseases, individualized risk prediction considering the complex mechanisms of disease development and comorbidity remains to be under-researched. This research aims to develop tensor factorization-based machine learning models to predict the onset of new chronic diseases for individual patients through incorporating the comorbidity patterns with the clinical and sequential factors revealed in the electronic health records (EHR) data. We propose two tensor factorization-based methods to incorporate the clinical and sequential factors to reveal the latent patterns of co-occurring chronic diseases. The efficacy of the proposed methods was validated through predicting the onset of new chronic diseases for individual patients using the EHR data for 23 years from a major hospital in Hong Kong. The proposed methods consistently outperform benchmark predictive models. The top 10 predictions of new chronic diseases have approximately 60% recall. Tensor factorization is an appropriate method for predicting the onset of chronic diseases at the individual level. The proposed predictive models could inform proactive health management programs for at-risk patients with different chronic conditions at discharge.\n\nAuthor summaryThe existing risk assessment models mainly focused on the prediction of single diseases in the population base. Chronic disease risk prediction considering the complex mechanisms of disease development and comorbidity is under-researched. To support and inform clinical decision making for healthcare professionals in the aging society, this study provides an innovative approach to mapping an interconnected web of chronic illnesses and investigated the performance of chronic disease prediction using 2 years worth of patient assessment records and 23 years admission history data from a major hospital in Hong Kong. We proposed matrix and tensor-based methods to represent the high-order interrelations of patients, chronic diseases and additional features, which can reveal the latent patterns of co-occurring chronic diseases to enable more effective prediction. The proposed methods exhibit state-of-the-art performance in predicting the onset of new chronic diseases for individual patients.