FQ-GAN NaNs for C10

NaNs for C10

Open greeneggsandyaml opened this issue 3 years ago • 1 comments

Hello,

I ran the C10 script (literally nothing modified) and I get NaNs:

10/195 (  4.62%) (TE/ETA: 0:05 / 1:49) Deleting singular value logs...al : +nan, D_loss_fake : +nan, Quant_loss : +nan, Perplexity : +1.000

Is this to be expected?

I've tried on 1 GPU, 2 GPUs, and 4 GPUs. All of them give NaNs.

Nov 22 '20 20:11 greeneggsandyaml

that's weird. I just ran the experiment again. FID kept decreasing. What modifications did you make to the model or hyperpearameter settings?

Nov 23 '20 23:11 YangNaruto