merlot_reserve icon indicating copy to clipboard operation
merlot_reserve copied to clipboard

Published 20 hours ago • verified publisher icon rowanz

Pretraining loss being negative

Open snat1505027 opened this issue 2 years ago • 1 comments

Is it possible to get negative loss for each task during pretraining? Also can you share the pretraining log file (mostly the loss of each task, i.e., audio2text, audio_text_matching etc.)?

snat1505027 avatar Sep 29 '22 18:09 snat1505027

hi Syeda - that shouldn't be possible! I think you probably have a bug somewhere in your setup. I do have logs somewhere on wandb, but I don't think they would be helpful in debugging this

rowanz avatar Sep 29 '22 21:09 rowanz