Gamma_Negative goes to NaN and loss increases exponentially

[gssriram]

Hello,

Thanks for this innovative work. I tried to implement ASL in my multi-label binary classification model (20 classes with average positive-negative ratio of 0.06). While doing so, after Epoch-8, the gamma_neg value goes to NaN and at the same time the ASL raises exponentially and stays there forever. This trend remains same in both normal version and optimised version of ASL. This is with respect to adaptive ASL [starting gamma_neg=4, gamma_pos=0, other params are default]. [The same trend in loss is prevalent in non-adaptive ASL as well.]

I am sure there is something that I am missing out. It would be helpful, if someone clarifies this issue and suggests points to improve my training.