BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology

Abstract

BigNeuron is an open community bench-testing platform combining the expertise of neuroscientists and computer scientists toward the goal of setting open standards for accurate and fast automatic neuron reconstruction. The project gathered a diverse set of image volumes across several species representative of the data obtained in most neuroscience laboratories interested in neuron reconstruction. Here we report generated gold standard manual annotations for a selected subset of the available imaging datasets and quantified reconstruction quality for 35 automatic reconstruction algorithms. Together with image quality features, the data were pooled in an interactive web application that allows users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and reconstruction data, and benchmarking of automatic reconstruction algorithms in user-defined data subsets. Our results show that image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. By benchmarking automatic reconstruction algorithms, we observed that diverse algorithms can provide complementary information toward obtaining accurate results and developed a novel algorithm to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms. Finally, to aid users in predicting the most accurate automatic reconstruction results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic reconstructions.