A solution to the learning dilemma for recurrent networks of spiking neurons

This is the paper to introduce "e-prop," or "eligibility traces" for learning spike neural networks.

Protein markers are placed on to neurons to track recent firing activity. These markers are used with "learning signals" to instruct neurons to update themselves, replacing the need to store historical copies of the network as in Backpropagation Through Time.

Spike nets can be studied with gradient descent techniques by use of a "surrogate" derivative.

Surrogate gradients allow common calculus to analyze the spiking net, then backpropagation through time allows studying gradient flow over a period of time, which then allows seeking an approximate update formula that does not depend on strict historical records.

Essentially the partial derivatives are "folded up" with approximations and proofs for two elements:

• The internal history of the gate; ex. folding up the derivatives from backpropagation through time
• The spike's output influence on the error

First, neurons in the brain maintain traces of preceding activity on the molecular level, for example, in the form of calcium ions or activated CaMKII enzymes9. In particular, they maintain a fading memory of events where the presynaptic neuron fired before the postsynaptic neuron, which is known to induce synaptic plasticity if followed by a top–down learning signal10,11,12. Such traces are often referred to as eligibility traces.