NeuroscienceNeuromorphic Computing and EngineeringPeer-reviewed
SNNs Are Not Transformers (Yet): The Architectural Problems for SNNs in Modeling Long-Range Dependencies
William Fishell, Gord Fishell, Suraj Honnuraiah
This paper analyzes the limitations of spiking neural networks (SNNs) in modeling long-range dependencies, showing their quadratic sample complexity compared to Transformers and RNNs.
DOI 10.1088/2634-4386/ae8626