Understanding binary classifier model structure based on Shapley feature interaction patterns

Abstract

There is increasing emphasis on the interpretability of machine learning models, including in understanding biological systems. The well-known Shapley value framework based on game theory works in principle with any models to attribute feature importance. While feature interactions are critical to understand and can be interpreted within this framework, much attention is paid in practice on global feature importance and general trends of interactions. The inter-relationships between underlying model structure and Shapley value and its decomposition is less clear. Here we use binary classifiers to systematically examine how logical and additive interactions affect marginal contributions. These decomposed main and interaction effects are reflected in resulting Shapley dependence plots. The directionality of inequalities or logical/additive operators influence independently the main and marginal/interaction effects. Lastly, we show that these principles are applicable for models with noise.