Scalable subclonal reconstruction of cancer cells in DNA sequencing data using a penalized likelihood model

Background Characterizing intratumor heterogeneity and subclonal architecture from bulk DNA sequencing provides important insights into cancer evolution. However, existing subclonal reconstruction methods remain computationally demanding, hindering iterative quality control and reproducible application across large genomic cohorts. Methods Distinct from existing likelihood-based and Bayesian approaches, we developed CliPP (Clonal structure identification through Pairwise Penalization), utilizing machine learning to add pairwise fusion penalty to the full likelihood, in order to simultaneously estimate mutation cellular prevalence and fuse compatible estimates into clusters without prespecifying the number of clusters. We benchmarked CliPP in three simulation datasets comprising 5,256 tumors for accuracy, and further evaluated its concordance with consensus calls in 2,008 tumors profiled by whole-genome sequencing from the Pan-Cancer Analysis of Whole Genomes (PCAWG) study. We then applied a standardized, quality-controlled workflow to whole-exome sequencing data from The Cancer Genome Atlas (TCGA), yielding a high-confidence cohort of 5,424 tumors across 31 cancer types. Within this cohort, we mapped the clonality of 13,960 high-confidence driver mutations in 4,375 tumors. Results CliPP showed competitive performance across all datasets, as compared to PhylogicNDT and PyClone-VI, while gaining significant improvement in speed. In PCAWG, CliPP achieved a concordance correlation coefficient (CCC) of 0.93 with the published consensus, leading to a 96.3% agreement in driver-level clonal/subclonal assignments from the PCAWG study (Cohen's kappa = 0.81). It took approximately one hour for CliPP to complete the analysis of the entire PCAWG and TCGA cohorts, gaining noticeable time over existing methods. Applying CliPP to the TCGA cohort, we generate a complete annotation of driver clonality across TCGA samples, and observe broad patterns of driver timing that recapitulate known biology. The CliPP-on-Web app allows non-computational researchers to directly obtain subclonal annotations of their tumor samples. Conclusions Machine learning assisted CliPP combines competitive reconstruction accuracy, substantial computational efficiency, and standardized quality control to enable reproducible subclonal reconstruction at large scale. Its pan-cancer application characterizes context-dependent patterns of driver clonality, and its software web interface enables cancer evolution inquiries from the cancer research community.