DNNBrain: a unifying toolbox for mapping deep neural networks and brains

Abstract

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks through an end-to-end deep learning strategy. Deep learning gives rise to data representations with multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Its success appeals to neuroscientists not only to apply DNNs to model biological neural systems, but also to adopt concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks such as PyTorch and TensorFlow could be used to allow such cross-disciplinary studies, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed for cognitive neuroscientists to map DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring internal representations in both DNNs and the brain. By integrating DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios, such as extracting DNN activation, probing DNN representations, mapping DNN representations onto the brain, and visualizing DNN representations. We expect that our toolbox will accelerate scientific research in applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.