Thinking ahead: prediction in context as a keystone of language in humans and machines
Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Rose Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Se Catherine Kim, , Uri Hasson
In nine epilepsy patients, cortical activity carries next-word information up to 800 ms before onset, tracks GPT-2 surprise after onset, and fits GPT-2 contextual embeddings better than static ones.
bioRxiv · Dec 2, 2020 · PreprintNeuroscience✦ In-depth digest