Clinically driven knowledge distillation for sparsifying high-dimensional multi-omics survival models

Recently, various methods have been proposed to integrate different heterogeneous high-dimensional genomic data sources to predict cancer survival, often in addition to widely available and highly predictive clinical data. Although clinical applications of survival models have high sparsity requirements, most state-of-the-art models do not naturally exhibit this sparsity, as they are based on random forests or deep learning. We propose to use[l] 1-penalized linear student models within a knowledge distillation framework to sparsify underlying multi-omics black-box teachers. We show that by excluding clinical variables from our[l] 1 penalty, we can effectively guide the knowledge distillation, reaching virtually identical discriminative performance to the teachers while using on average 140 features or less across the 17 cancer datasets from The Cancer Genome Atlas (TCGA) considered in our study.