pytorch icon indicating copy to clipboard operation
pytorch copied to clipboard

optimizer loss becomes nan

Open MicheleBellomo opened this issue 9 months ago • 25 comments

I'm trying to use your implementation to faster optimize a problem that I've already trated using different optimizers and libraries. During the first iteration of LFBGS_B, the losses are in the first steps calculated correctly (and they are correctly decreasing), but then suddenly they become nan, and the same happen to the parameters being optimized. What can cause this behavior?

MicheleBellomo avatar May 27 '24 12:05 MicheleBellomo