hadas

Hi Did you find a solution to the problem? I get the same decrease in loss. When uploading the open checkpoint released by meta it seems like the unmasked loss...

Hi, I ran into a similar issue with a very low loss and cluster collapse. Except for the batch size (4), I haven't changed anything in the base configuration, but...