LDU icon indicating copy to clipboard operation
LDU copied to clipboard

Published 20 hours ago • verified publisher icon ENSTA-U2IS-AI

Nan loss during training

Open Waynechen92 opened this issue 2 years ago • 3 comments

Has anyone ever met Nan lossing during training for bts_main? Any possible solution for that?

Waynechen92 avatar Nov 23 '22 02:11 Waynechen92

Hi, thank you for leaving the comments. Actually, it appears in the original bts training as well, you can check here to find some suggestions given by the authors of bts.

xuanlongORZ avatar Nov 23 '22 10:11 xuanlongORZ

Thanks for your reply xuanlong, I'm also wondering whether it is correct to have negative loss in training for bts_main

Waynechen92 avatar Nov 24 '22 05:11 Waynechen92

Several common loss functions are indeed non-negative such as MSELoss or L1Loss. But it is fine to have a negative loss value during training, not only in LDU case. You can check here for example.

xuanlongORZ avatar Nov 25 '22 20:11 xuanlongORZ