Concurrent emergence of view invariance, sensitivity to critical features, and identity face classification through visual experience: Insights from deep learning algorithms

Visual experience is known to play a critical role in face recognition. This experience is believed to enable the formation of a view-invariant representation, by learning which features are critical for face identification across views. Discovering these critical features and the type of experience that is needed to uncover them is challenging. We have recently revealed a subset of facial features that are critical for human face recognition. We further revealed that deep convolutional neural networks (DCNNs) that are trained on face classification, but not on object categorization, are sensitive to these facial features, highlighting the importance of experience with faces for the system to reveal these critical features. These findings enable us now to ask what type of experience with faces is required for the network to become sensitive to these human-like critical features and whether it is associated with the formation of a view-invariant representation and face classification performance. To that end, we systematically manipulated the number of within-identity and between-identity face images and examined its effect on the network performance on face classification, view-invariant representation, and sensitivity to human-like critical facial features. Results show that increasing the number of images per identity as well as the number of identities were both required for the simultaneous development of a view-invariant representation, sensitivity to human-like critical features, and successful identity classification. The concurrent emergence of sensitivity to critical features, view invariance and classification performance through experience implies that they depend on similar features. Overall, we show how systematic manipulation of the training diet of DCNNs can shed light on the role of experience on the generation of human-like representations.