Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network

The features of an image can be represented at multiple levels - from its low-level visual properties to high-level meaning. What drives some images to be memorable while others are forgettable? We address this question across two behavioral experiments. In the first, different layers of a convolutional neural network (CNN), which represent progressively higher levels of features, were used to select the images that would be shown to 100 participants through a form of prospective assignment. Here, the discriminability/similarity of an image with others, according to different CNN layers dictated the images presented to different groups, who made a simple indoor vs. outdoor judgment for each scene. We find that participants remember more scene images that were selected based on their low-level discriminability or high-level similarity. A second experiment replicated these results in an independent sample of fifty participants, with a different order of post-encoding tasks. Together, these experiments provide evidence that both discriminability and similarity, at different visual levels, predict image memorability.