Cheating GAN?

[3DTOPO]

I am using the CycleGan model and it works perfectly for most images, but it sometimes creates garbage like the generator is cheating.

If I lower the learning rate enough, the problem goes away, but the effect it produces is too weak to be useful. I've also tried lowering the Adam momentum, but the issue still occurs. I've tried using lsgan, vanilla, and disabling gan loss. If I disable gan loss, the "garbage" issue goes away, but the effect is way too weak. I've lowered the learning rate to 0.00001 and ran it for 200 epochs but the effect was too weak. I've tried setting history to 500.

If I increase the number of generator and discriminator filters the issue is less severe, but I am trying to use as small model as possible to get the results I need (the generator needs to run on mobile).

Any suggestions? The training set is 41,000 images. Do I need more images? Do I need to use a very low learning rate and let it run for much longer (eg 500 epochs at 0.00001)?

Each setting I try takes days (on the GeForce 3090), so suggestions would be very much appreciated.

[3DTOPO]

Here's another example of "garbage".

[yipengliu1993]

Hi, just wondering are you using ubuntu or windows when running cycle gan? My 3090 is not working at all

[3DTOPO]

Ubuntu.

[junyanz]

It looks like that the generator is hiding information for the reconstruction purpose. See this study for more details. It is hard to know what's going on without seeing the entire dataset/task and trying multiple models. If you haven't tried it, you could train a model on smaller patches rather than the entire image.