neuroptica
neuroptica copied to clipboard
Autograd activations
This introduces automatic computation of the backpropagation through activation functions.
Changes:
There is only one Nonlinearity
class. Real, complex, and non-holomorphic activations handled exactly the same way.
You only need to specify the forward_pass()
method of each nonlinearity. Autograd will take care of the backward pass for you.