nom57

check your stats.jsonl either diffaug doesnt work or you have a "NaN" value for your scores/fake , or you could have misconfigured batch size and learning rate.

I think what would be even more useful is logs.txt and stats.jsonl for every model provided.

https://github.com/autonomousvision/projected_gan/issues/87#issuecomment-1180581191 someone else facing the same issue , they just deleted act1 from projector.py but this is not recommended Should wait for @xl-sr 's reply on this issue in my...

I am having similar issues and its clearly visible in stats.jsonl that the discriminator is getting lower and lower loss score. I think this means diffaug is broken and it...

also @cleamm please share your stats.jsonl content so I can compare it to mine as I debugged my issues a lot with SGXL and it was always dead giveaway what...

I had this same issue and its caused either by : wrong plugin versions , your hardware being unstable, MKL running parallel , or Diffaug being broken in the model...

@mapengsen please share the text of your stats.jsonl file.

this method of training can make recall be 0.7+ instead of 0.5 on many datasets while still reaching the same FID (after also bottoming out FID on 128 batch size)...

for example the pokemon dataset can have a recall of 0.787 at 64x64 with this method @xl-sr so an -nkimg resume feature would be tremendously helpful

update : this seems to favor SG2-Ada way more than SGXL , SGXL can easily have collapses with low batch sizes , so its hard to tame , but still...