nengo-dl
nengo-dl copied to clipboard
Warn if converter's scale_firing_rates would skew the nonlinearities
Related to #206.
Currently the nengo_dl converter's scale_firing_rates
will gladly change the effective response that you get from any nonlinearity except for the linear/ReLU activations. There is this warning:
https://github.com/nengo/nengo-dl/blob/e9b359a4778aa7812488d9a40a8118c0b92b18ee/nengo_dl/converter.py#L568-L573
But this is only emitted for neuron types that don't support amplitude
. I'd expect a similar (or generalized) warning to be emitted if the neuron's activation function is going to be skewed by the rescaling, as this also changes the output of the neuron. Otherwise the model may perform worse, as the trained weights would be w.r.t. the wrong nonlinearities, which could be confusing for someone who misses this subtlty about scale_firing_rates
in the converter's docstring.