surrogate gradients#

Heaviside#

class sinabs.activation.Heaviside(window: float = 1.0)#

Heaviside surrogat gradient with optional shift.

Parameters

window – Distance between step of Heaviside surrogate gradient and threshold, relative to threshold.

MultiGaussian#

class sinabs.activation.MultiGaussian(mu: float = 0.0, sigma: float = 0.5, grad_scale: float = 1.0)#

Surrogate gradient as defined in Yin et al., 2021. https://www.biorxiv.org/content/10.1101/2021.03.22.436372v2

SingleExponential#

class sinabs.activation.SingleExponential(grad_width: float = 0.5, grad_scale: float = 1.0)#

Surrogate gradient as defined in Shrestha and Orchard, 2018 https://papers.nips.cc/paper/2018/hash/82f2b308c3b01637c607ce05f52a2fed-Abstract.html