Week 5 reading

Spiking is not differentiable

• Introduction to neural networks and backpropagation by 3Blue1Brown (excellent, easy to follow YouTube series)

Limited gradients

• Julien Vitay lecture notes on reservoir computing
• Nicola and Clopath (2017) “Supervised learning in spiking neural networks with FORCE training”
• Schuman et al. (2020) “Evolutionary Optimization for Neuromorphic Systems”
• Mitchell et al. (2017) “NeoN: Neuromorphic control for autonomous robotic navigation”
• Hunsberger and Eliasmith (2015) “Spiking Deep Networks with LIF Neurons”
• Neural Engineering Framework
• Nengo
• Thorpe and Imbert (1989) “Biological constraints on connectionist modelling”
• Kheradpisheh and Masquelier (2020) “S4NN: temporal backpropagation for spiking neural networks with one spike per neuron”

Surrogate gradients

• Neftci et al. (2019) “Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks”
• Zenke and Neftci (2021) “Brain-Inspired Learning on Neuromorphic Substrates”
• Zenke and Vogels (2021) “The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks”
• SPyTorch tutorial (including video)
• Spiking Heidelberg Digits dataset
• Lillicrap and Santoro (2019) “Backpropagation through time and the brain”
• Rossbroich et al. (2022) “Fluctuation-driven initialization for spiking neural network training”
• Perez-Nieves et al. (2023) “Spiking Network Initialisation and Firing Rate Collapse”
• Perez-Nieves et al. (2021) “Neural heterogeneity promotes robust learning”

Approximate gradients

• Marschall et al. (2019) “A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks”