Weight initialization shapes task organization in recurrent neural networks
This study shows that low initial weight variance in recurrent neural networks leads to overlapping weight subspaces for subtasks, while high variance results in more segregated subspaces.