Vincent Roulet
Vincent Roulet
Closing since it was branched in #1497
Did you check this https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.lion ?
Ok, thank you for noting this. But is it an issue with the code or not? If the loss, as defined a priori, produces these instabilities, independently of the code,...
Thank you for the example! We will look into that
Ok, I was not considering the possibility of differentiating gamma. In the context of the user's issue, it seemed that mostly the behavior for the logits mattered. If gamma is...
Thanks @leochlon Can you **please** read my last comment? This function should be coded by keeping all possible computations in logarithm of probabilities to avoid numerical issues.
Logarithm is not a linear function. You cannot have `p_t = p * labels + (1 - p) * (1 - labels)` in the original code and `log(p_t) = labels...
Yes, if we constrain labels to be binary then this is great. I'm not familiar with this loss in particular so I don't know if some people would want to...
This looks pretty good to me. Again I don't think you will need clamping with such an implementation. In terms of tests, test for combinations of label and logits such...