Deficient Executive Control in Transformer Attention
Suketu Chandrakant Patel, Hongbin Wang, Jin Fan
This study evaluates the limitations of transformer attention mechanisms in large language models by testing their performance on a Stroop task, revealing degraded conflict resolution with longer sequences.