📖 Reading material

Spiking is not differentiable¶

• Introduction to neural networks and backpropagation by 3Blue1Brown (excellent, easy to follow YouTube series)

Limited gradients¶

• Julien Vitay lecture notes on reservoir computing

• Nicola and Clopath (2017) “Supervised learning in spiking neural networks with FORCE training”

• Schuman et al. (2020) “Evolutionary Optimization for Neuromorphic Systems”

• Mitchell et al. (2017) “NeoN: Neuromorphic control for autonomous robotic navigation”

• Hunsberger and Eliasmith (2015) “Spiking Deep Networks with LIF Neurons”

• Neural Engineering Framework

• Nengo

• Thorpe and Imbert (1989) “Biological constraints on connectionist modelling”

• Kheradpisheh and Masquelier (2020) “S4NN: temporal backpropagation for spiking neural networks with one spike per neuron”

Surrogate gradients¶

• Neftci et al. (2019) “Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks”

• Zenke and Neftci (2021) “Brain-Inspired Learning on Neuromorphic Substrates”

• Zenke and Vogels (2021) “The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks”

• SPyTorch tutorial (including video)

• Spiking Heidelberg Digits dataset

• Lillicrap and Santoro (2019) “Backpropagation through time and the brain”

• Rossbroich et al. (2022)

• Perez-Nieves et al. (2023) “Spiking Network Initialisation and Firing Rate Collapse”

• Perez-Nieves et al. (2021) “Neural heterogeneity promotes robust learning”

Approximate gradients¶

• Marschall et al. (2019) “A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks”