Initial signs of learning: Decoding newly-learned vocabulary from neural patterns in novice sign language learners

Abstract

How do novice language learners represent semantic information in their new language? Whether bilinguals neural representations of different languages are distinct or overlapping has been well-studied, but less is known about how new knowledge is integrated into established representational networks at the earliest stages of acquisition. In addition to being an understudied case in their own right, signed languages allow the study of language learning when the sensory features of the target language are maximally dissimilar from the learners prior experience. Using fMRI data from 40 hearing non-signers following their first lesson in American Sign Language (ASL), we show that group-level patterns of neural representation correlate with semantic features for studied (but not unstudied) ASL signs. Furthermore, cross-language neural similarity between semantically equivalent nouns reflects individual-level comprehension of ASL. Item-level comprehension was decodable in regions that represent semantic relationships within ASL, suggesting that representations of newly acquired ASL nouns become more similar to well-known English counterparts as a result of early-stage learning. These results demonstrate the role of distributed neural regions, including perisylvian language areas as well as right parietal areas, in representing semantic content across language and modality in novice learners. Research Transparency StatementThe authors declare no conflicts of interest. This research was supported by National Science Foundation Award #1822819. No artificial intelligence assisted technologies were used in the creation of this article. This research received approval from the Dartmouth Committee for the Protection of Human Subjects. No aspects of the studies were preregistered. Some stimulus videos cannot be shared due to copyright restrictions but can be viewed on the original website (www.asl-lex.org). Deidentified primary data and analysis scripts are available upon request to the corresponding author.