A battery of image classification challenges reveals shared and distinct object categorization behavior across monkeys, humans, and deep networks

Abstract

Humans categorize objects at multiple levels of abstraction--animate versus inanimate, big versus small, and many other attributes. Despite its apparent challenge, the advent of deep neural networks (DNNs) has demonstrated that complex visual processing alone can support such classification without language or human-specific knowledge. This raises a natural question: to what extent can non-human primates, without language, perform such categorization? Although basic object-recognition behavior in monkeys such as similarity judgment has been extensively studied, their ability to classify objects across diverse rules remains poorly characterized. Here, we developed a task paradigm that enabled us to train monkeys on a large battery of binary classification tasks using natural object images, spanning more than 10 rules, such as animate versus inanimate, natural versus man-made objects, and mammalian versus non-mammalian animals. We then evaluated their performance on novel, generalization stimuli. Monkeys acquired each rule in a few days, generalized it well, and exhibited error patterns consistent with human judgments. At the same time, their classification performance correlated more strongly with that of visual DNNs trained without language input, whereas human performance was better explained by language-informed DNNs. These results provide an important benchmark for the capacity of biological neural networks to perform image classification without language and human-specific knowledge.