In model-based fMRI significant is less than specific

Abstract

By comparing computational model output to BOLD signal changes model-based fMRI has the potential to offer profound insight into what neural computations occur when. If this potential is to be fully realized, statistically significant outcomes must imply specific outcomes. That is, we must have a clear idea of how often a model not present in the BOLD signal but present in the predictor set will reach significance. We ran Monte Carlo simulations of reinforcement learning to examine this kind of specificity, focusing in on two aspects. One, to what degree can we tell related but theoretically distinct predictors apart. About 40% of the time the studied predictors were indistinguishable. Two, how well can we separate out different parameterizations of the same reinforcement learning terms. Nearly all parameter settings were indistinguishable. The lack of specificity between models and between parameters suggests a uncertain relation between significance and specificity. Follow up analyses suggest the temporally slow and prototyped nature of the haemodynamic response (HRF) can substantially increase correlations, ranging from -0.16 to 0.73 with an average of 0.27. Though we focused on a single case study, i.e., reinforcement learning, specificity concerns are potentially present in any design which does not account for the slow prototyped nature of the HRF. We suggest more specific conclusions can be reached by moving from null hypothesis testing approach to a model selection or model comparison framework.