Explainable Artificial Intelligence (XAI)

TorchLens: A Python package for extracting and visualizing hidden activations of PyTorch models

TorchLens extracts and visualizes hidden activations of PyTorch models, providing full computational graph records and validation for analysis.

Novel Methods for Elucidating Modality Importance in Multimodal Electrophysiology Classifiers

This study develops novel methods to assess modality importance in multimodal electrophysiology classifiers, finding EEG is globally most important for sleep stages except stage 1, with subject-level differences influenced by sex, medication, and age.

Towards Greater Neuroimaging Classification Transparency via the Integration of Explainability Methods and Confidence Estimation Approaches

This study integrates confidence estimation and explainability methods for schizophrenia diagnosis using rs-fMRI data, finding that MCBN is better suited for gradient-based explanations than MCD.

Towards Computing Attributions for DimensionalityReduction Techniques

The paper computes local feature attributions for t-SNE using gradient-based methods and validates them on synthetic and biological datasets.

Adversarial training improves model interpretability in single-cell RNA-seq analysis

This study shows adversarial training improves robustness and interpretability in single-cell RNA-seq cell type classification models.

Interpreting tree ensemble machine learning models with endoR

endoR interprets tree ensemble models in microbiome data by extracting feature importance and interactions, confirming known associations and revealing new microbial network insights.

CLARUS: An Interactive Explainable AI Platform for Manual Counterfactuals in Graph Neural Networks

CLARUS is an interactive explainable AI platform for GNNs that allows clinicians to validate, modify, and retrain models using patient-specific biological networks.

An Approach for Estimating Explanation Uncertainty in fMRI dFNC Classification

This study introduces a method to estimate uncertainty in gradient-based explanations for fMRI dFNC classification, finding significant differences in explanation relevance between schizophrenia and control groups.

Best Practices for Interpretable Machine Learning in Computational Biology

This paper introduces a workflow for best practices in interpretable machine learning to enhance knowledge discovery in computational biology.

Weakly supervised Unet: an image classifier which learns to explain itself

This paper introduces WSUnet, a weakly supervised Unet that generates voxel-level explanations for image-level predictions, outperforming existing methods in lung cancer detection.

fastISM: Performant in-silico saturation mutagenesis for convolutional neural networks

fastISM accelerates in-silico saturation mutagenesis for convolutional neural networks by 10x, making it feasible for large-scale sequence analysis.

Assessment and Optimization of the Interpretability of Machine Learning Models Applied to Transcriptomic Data

This study evaluates and optimizes the interpretability of machine learning models applied to transcriptomic data by assessing explainers on MLP and CNN architectures and identifying tissue-specific genes.

Positional SHAP (PoSHAP) for Interpretation of Machine Learning Models Trained from Biological Sequences

This paper introduces PoSHAP, a framework for interpreting machine learning models trained on biological sequences, demonstrating its ability to reproduce known peptide binding motifs and provide insights into amino acid interactions.

Interpreting Neural Networks for Biological Sequences by Learning Stochastic Masks

This paper introduces Scrambler networks, a method for interpreting sequence-based neural networks by learning stochastic masks to identify salient sequence positions and explain model predictions.

Towards More Realistic Simulated Datasets for Benchmarking Deep Learning Models in Regulatory Genomics

This paper benchmarks deep learning interpretation methods on more realistic simulated regulatory genomic data, finding that some methods perform poorly on complex datasets.

A Novel Activation Maximization-based Approach for Insight into Electrophysiology Classifiers

This paper introduces an activation maximization method for electrophysiology classifiers, showing it can identify spectral patterns in EEG sleep stages and reveal potential reliance on artifacts for awake classification.

A Novel Local Ablation Approach for Explaining Multimodal Classifiers

This paper introduces a local ablation method to explain multimodal classifiers, showing EEG's importance in sleep staging and identifying demographic effects on classification.

A Novel Local Explainability Approach for Spectral Insight into Raw EEG-Based Deep Learning Classifiers

This paper introduces a local spectral explainability method to analyze deep learning classifiers trained on raw EEG data for sleep stage classification, showing varying frequency band importance over time and within sleep stages.

A Gradient-based Spectral Explainability Method for EEG Deep Learning Classifiers

This study introduces a local spectral explainability method for EEG deep learning classifiers, showing that delta, beta, and gamma bands are most important and that age and sex affect classifier patterns.

Designing Interpretable Convolution-Based Hybrid Networks for Genomics

This study investigates how architectural choices in convolution-based hybrid networks affect motif representations and the interpretability of their attention maps in genomics.

Global Importance Analysis: An Interpretability Method to Quantify Importance of Genomic Features in Deep Neural Networks

This paper introduces Global Importance Analysis (GIA), a method to quantify the effect size of genomic features in deep neural networks, demonstrating its application in predicting RNA-protein interactions.

An Explainable Deep Learning Approach for Multimodal Electrophysiology Classification

The paper introduces a gradient-based explainability method for multimodal electrophysiology classifiers, showing EEG's dominance in sleep stage classification.

Explainable Feature Extraction Using a Neural Network with non-Synaptic Memory for Hand-Written Digit Classification

This paper introduces a brain-inspired neural network with non-synaptic memory for handwritten digit classification, achieving 86% accuracy using full training data.

Understanding binary classifier model structure based on Shapley feature interaction patterns

The paper examines how logical and additive interactions affect Shapley value decomposition in binary classifiers and shows their applicability in noisy models.

PlotMI: interpretation of pairwise dependencies and positional preferences learned by deep learning models from sequence data

PlotMI visualizes positional preferences and pairwise dependencies learned by sequence models, enabling comparison across models and datasets.

Explaining decisions of Graph Convolutional Neural Networks: patient-specific molecular subnetworks responsible for metastasis prediction in breast cancer

The paper introduces GLRP, a method to explain Graph-CNN decisions in breast cancer metastasis prediction, identifying patient-specific molecular subnetworks that align with clinical knowledge.

Improving representations of genomic sequence motifs in convolutional networks with exponential activations

This study shows that using exponential activations in the first layer of CNNs improves the interpretability and robustness of genomic sequence motif representations in both synthetic and real DNA data.

REM: An Integrative Rule Extraction Methodology for Explainable Data Analysis in Healthcare

REM extracts interpretable rules from deep learning and non-deep learning models to enhance explainability in healthcare decision-making.

Compression-based Network Interpretability Schemes

The paper investigates compression-based methods for interpreting deep learning models in cancer and psychiatric genomics, showing that compression quality can be assessed using generalization error and PAC-Bayes bounds.

Beyond generalization: Enhancing accurate interpretation of flexible models

The paper investigates whether accurate model interpretation can be achieved through good prediction in flexible models and proposes a method to identify correctly interpreted models.