Massively Parallel FPGA Hardware for Spike-By-Spike Networks

Abstract

While inspired by the brain, currently successful artificial neural networks lack key features of the biological original. In particular, the deep convolutional networks (DCNs) neither use pulses as signals exchanged among neurons, nor do they include recurrent connections which are both core properties of real neuronal networks. This not only puts to question the relevance of DCNs for explaining information processing in nervous systems but also limits their potential for modeling natural intelligence.\n\nSpike-By-Spike (SbS) networks are a promising new approach that combines the computational power of artificial networks with biological realism. Instead of separate neurons they consist of neuronal populations performing inference. Even though the underlying equations are rather simple implementations of such networks on currently available hardware are several orders of magnitude slower than for comparable non-spiking deep networks.\n\nHere, we develop and investigate a framework for SbS networks on chip. Thanks to the communication via spikes, already moderately sized deep networks based on the SbS approach allows a parallelization into thousands of simple and fully independent computational cores. We demonstrate the feasibility of our design on a Xilinx Virtex 6 FPGA while avoiding proprietary cores (except block memory) that cannot be realized on a custom-designed ASIC. We present memory access optimized circuits for updating the internal variables of the neurons based on incoming spikes as well as for learning the connections strength. The optimized computational circuits as well as the representation of variables fully exploit the non-negative properties of all data in the SbS approach. We compare the sizes of the arising circuits for floating and fixed point numbers. In addition we show how to minimize the number of components that are required for the computational cores by reusing their components for different functions.