When do measured representational distances reflect the neural representational geometry?

Abstract

The representational geometry of a brain region can be characterized by the distances among neural activity patterns for a set of experimental conditions. Researchers routinely estimate representational distances from brain-activity measurements that either sparsely sample the underlying neural population (e.g. neural recordings) or pool across the activity of many neurons (e.g. fMRI voxels). Here we use theory and simulations to clarify under what circumstances representational distances estimated from brain-activity measurements reflect the representational geometry of the underlying neural population, and what distortions must be expected under other circumstances. We demonstrate that the estimated representational distances are undistorted if single neurons are sampled at random. For voxels that take non-negatively weighted linear combinations, the resulting geometry is linearly distorted, correctly reflecting the population-mean dimension, while downscaling all orthogonal dimensions, for which the averaging cancels a large portion of the signal. Surprisingly, removing the mean from voxel patterns recovers the underlying representational geometry exactly in expectation under idealized conditions. This explains why the correlation distance, the most popular measure of representational dissimilarity in neuroimaging studies, "works" so well, yielding geometries that can appear similar between fMRI and neural recordings. The Euclidean (or Mahalanobis) distance computed after removing the mean of each pattern (without normalizing its variance) is an attractive alternative to the correlation distance in that it corrects for the inflated relative contribution of the population-mean dimension, while avoiding the drawback of the correlation distance: it can be large for confusable low-norm patterns, failing to reflect decodability. Our results demonstrate that measured representational distances reflect the neural representational geometry when (1) single neurons are sampled at random or (2) the weights with which the measured responses sample the neurons are drawn i.i.d. and (2a) the weights are drawn from a zero-mean distribution or (2b) the population mean is the same for all conditions or (2c) the mean is removed from each estimated pattern. We discuss practical implications for analyses of neural representational geometries.