Challenges for Bayesian Model Selection of Dynamic Causal Models

Achieving a mechanistic explanation of brain function requires understanding causal relationships among regions. A relatively new technique to assess effective connectivity in fMRI data is Dynamic Causal Modeling (DCM). As DCM is more frequently used, it becomes increasingly important to further validate the technique and understand its limitations. With DCM, Bayesian Model Selection (BMS) is used to select the most likely causal model. We conducted simulations to test the degree to which BMS is robust to two types of challenges when applied to DCMs, those inherent to data (Category 1) and those inherent to model space (Category 2). Category 1 challenges tested properties of the data (low signal-to-noise, different response magnitudes and shapes across regions) that could either blur the distinction between models or potentially bias model selection. These challenges are impossible or difficult to measure and control in real data, so investigating their effect upon BMS through simulation is critical. Category 2 challenges tested properties of model space that create subsets of confusable models. Our results suggest that given data that conform to the prior assumptions of DCM, BMS is robust to challenges from Category 1. However, in the face of Category 2 challenges (when a more homogenous model space was tested) the false positive rate rose above an acceptable level. We show that such errors are neither trivial nor easily avoided with existing approaches. However, we argue that it is possible to detect Category 2 challenges, and avoid inappropriate interpretations by conducting simulations prior to applying DCM.\n\nAcronyms