Humans compress order-based structure into short mental programs

Abstract

How do humans store sequences that far exceed working memory capacity? Previous work showed that visuo-spatial sequences with geometric regularities and binary auditory sequences are compressed into mental programs made of nested compositions of simple operations provided by the Language of Thought model. Here we ask whether this principle extends to a more abstract domain: order-based structure, such that sequences would be coded by how items repeat and in what order, leveraging ordinal positions in memory. For instance, imagine the sequence of tones CCDDEECCDDEE and the sequence of colors RRGGBBRRGGBB (R: red, G: green and B: blue). They share the same structure, which lets you store 12 items as '3 pairs, repeat'. In two experiments, participants memorized and reproduced 12-item sequences of spatial locations; in a third, they rated the complexity of those same sequences. The order-based Language of Thought description of the sequences predicted recall accuracy with high precision (r = .96), outperforming several alternative sequence complexity measures, but also participants' own subjective complexity ratings. Beyond accuracy, the inferred programs predicted the structure of motor production of sequences: inter-click intervals slowed down and errors increased at the constituent boundaries specified by the model. Finally, we identified a set of elementary operations (mirroring, interleaving) that are also building blocks of such mental programs. Together these findings indicate that when a sequence affords an order-based structure, it is encoded as a mental program in human memory. We thus extend the validity of the Language of Thought framework to an abstract code, in which structure relies on the order in which items appear rather than modality-specific regularities.