nengo-dl icon indicating copy to clipboard operation
nengo-dl copied to clipboard

Warn if converter's scale_firing_rates would skew the nonlinearities

Open arvoelke opened this issue 3 years ago • 0 comments

Related to #206.

Currently the nengo_dl converter's scale_firing_rates will gladly change the effective response that you get from any nonlinearity except for the linear/ReLU activations. There is this warning:

https://github.com/nengo/nengo-dl/blob/e9b359a4778aa7812488d9a40a8118c0b92b18ee/nengo_dl/converter.py#L568-L573

But this is only emitted for neuron types that don't support amplitude. I'd expect a similar (or generalized) warning to be emitted if the neuron's activation function is going to be skewed by the rescaling, as this also changes the output of the neuron. Otherwise the model may perform worse, as the trained weights would be w.r.t. the wrong nonlinearities, which could be confusing for someone who misses this subtlty about scale_firing_rates in the converter's docstring.

arvoelke avatar Feb 24 '21 15:02 arvoelke