Spiking is not differentiable¶
Limited gradients¶
- Julien Vitay lecture notes on reservoir computing
- Nicola and Clopath (2017) “Supervised learning in spiking neural networks with FORCE training”
- Schuman et al. (2020) “Evolutionary Optimization for Neuromorphic Systems”
- Mitchell et al. (2017) “NeoN: Neuromorphic control for autonomous robotic navigation”
- Hunsberger and Eliasmith (2015) “Spiking Deep Networks with LIF Neurons”
- Neural Engineering Framework
- Nengo
- Thorpe and Imbert (1989) “Biological constraints on connectionist modelling”
- Kheradpisheh and Masquelier (2020) “S4NN: temporal backpropagation for spiking neural networks with one spike per neuron”
Surrogate gradients¶
- Neftci et al. (2019) “Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks”
- Zenke and Neftci (2021) “Brain-Inspired Learning on Neuromorphic Substrates”
- Zenke and Vogels (2021) “The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks”
- SPyTorch tutorial (including video)
- Spiking Heidelberg Digits dataset
- Lillicrap and Santoro (2019) “Backpropagation through time and the brain”
- Rossbroich et al. (2022)
- Perez-Nieves et al. (2023) “Spiking Network Initialisation and Firing Rate Collapse”
- Perez-Nieves et al. (2021) “Neural heterogeneity promotes robust learning”
Approximate gradients¶
- Nicola, W., & Clopath, C. (2017). Supervised learning in spiking neural networks with FORCE training. Nature Communications, 8(1). 10.1038/s41467-017-01827-3
- Schuman, C. D., Mitchell, J. P., Patton, R. M., Potok, T. E., & Plank, J. S. (2020). Evolutionary Optimization for Neuromorphic Systems. Proceedings of the Neuro-Inspired Computational Elements Workshop, 1–9. 10.1145/3381755.3381758
- Mitchell, J. P., Bruer, G., Dean, M. E., Plank, J. S., Rose, G. S., & Schuman, C. D. (2017). NeoN: Neuromorphic control for autonomous robotic navigation. 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), 136–142. 10.1109/iris.2017.8250111
- Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks. IEEE Signal Processing Magazine, 36(6), 51–63. 10.1109/msp.2019.2931595
- Zenke, F., & Neftci, E. O. (2021). Brain-Inspired Learning on Neuromorphic Substrates. Proceedings of the IEEE, 109(5), 935–950. 10.1109/jproc.2020.3045625