Visual-Feature-Attribution-Using-Wasserstein-GANs-Pytorch icon indicating copy to clipboard operation
Visual-Feature-Attribution-Using-Wasserstein-GANs-Pytorch copied to clipboard

Fixing bugs in data loading and cpu/gpu managment

Open baumgach opened this issue 2 years ago • 1 comments

This PR consists of three commits which roughly correspond to three bugs I found:

  • Remove requires_grad for network input: This was not required for anything, so I removed it, but it didn't affect the results of the model
  • Sample new batch for generator loss: The generator loss was calculated on the last critic batch rather than with a new batch. I changed the code to sample a new batch. But it turns out the length of the data loader happened to be exactly 100 which is equal the number of critic updates. Now that we need to sample 101 batches in total, this code was running out of samples. The fact that this is possible can be considered another bug. So I added some safe guards against this happening, and changed the dataset so that it generates data loaders of 101 batches instead.
  • Move images to cpu before saving: There was also an issue in the validation stage that is run every 50 epochs that was preventing any images from being written, because they were still on GPU. Not sure if this issue was introduced by a later PyTorch version (I am using 1.10.2+cu102), but the code was in fact not functional out of the box. Luckily that was a simple fix.

baumgach avatar Mar 09 '22 15:03 baumgach

Thank you very much @baumgach.

@lantiga could you check this out please? thanks!

dnlcrl avatar Mar 10 '22 11:03 dnlcrl