The impact of systematized generation, evaluation, and incorporation of machine learning algorithms for clinical variant classification

Abstract

Variants of uncertain significance (VUS) pose a significant challenge for those undergoing genetic testing, leading to prolonged uncertainty and inappropriate medical care. VUS rate reduction is critical to fully realize the utility of genetic testing for all populations. With the growth of large-scale biological data sources and modern Machine Learning (ML) techniques, predictive modeling has enormous potential for VUS reduction. For this purpose, we developed the Invitae Evidence Modeling Platform (EMP), with key features designed to maximize the utility and confidence of predictive algorithms for variant classification. First, input data for a new model is curated to correspond to a single major evidence category within a variant classification framework. Second, gene-specific training and/or validation is performed for each model type. Third, accuracy thresholds are set to filter out gene-specific models that do not meet stringent accuracy metrics. Finally, prediction scores for variant pathogenicity are calibrated to ensure internally consistent evidence weighting within the classification framework. The EMP has accelerated the development of ML algorithms and greatly expanded the amount of evidence available for variant classification. EMP evidence has been applied to more than 800,000 variants across 1 million individuals, 42% of which would have been VUS without this evidence. Importantly, definitive classifications (P, LP, LB, B) made with EMP evidence have high prospective concordance (>99%) with ClinVar submissions. Finally, we demonstrate that further use and development of EMP evidence for variant classification has the potential to reduce the VUS disparity across race/ethnicity/ancestry (REA) groups.