SimGAN

SimGAN copied to clipboard

HI~ Grateful for your work! I run the code, setting the epoch to 6000, but the performance is seemed to be poor. The refined images has no difference with the synthesized images. Have you ever tried a long training and got a good result? Thank you.

Hi @USTClj , I tried to train this model on some other dataset for 20000 epoch and got the results almost the same as the inputs, and also noticed that the refiner loss was extremely big and was never reduced during the training. There was no improvement even if I increased the update times of refiner network in an epoch.

See https://github.com/soumith/ganhacks for tips to stabilize GAN training. GANs are notoriously unstable and tough to get to converge. If refiner loss is extremely high and discriminator is near 0, this could mean discriminator is much more powerful than refiner and you should increase the number of steps you train refiner https://github.com/wayaai/SimGAN/blob/master/sim-gan.py#L46

@mjdietzx Hi, thank you for you replying. I have tried to set the kg(=50) bigger which means that the refiner is trained more step in one loop, but the result is still very similar to the synthesized image. I trained the model on the 50000 Unity Eyes: https://www.kaggle.com/4quant/eye-gaze.

I just found that apple has a journal about their SimGAN paper. Check out Improving the Realism of Synthetic Images. It has some suggestions about setting the hyper-parameters, which might be helpful.

I read the post mentioned by @zhengkang86 , and found some tips about training:

1. Initialization of G First we initialized G only with the self-regularization loss so that it can start producing blurry version of the synthetic input. Typically it took 500-2,000 steps of G (without training D).
2. Different steps of G and D for each training iteration We used different number of steps for the generator and discriminator in each training iteration. For hand-pose estimation using depth, we used 2 steps of G for each D step, and for eye gaze estimation experiment we ended up using 50 steps of G for each D step.

Changing preprocessing function worked for me.

(Before) datagen = image.ImageDataGenerator(preprocessing_function=applications.xception.preprocess_input, data_format="channels_last")

↓

(After) datagen = image.ImageDataGenerator(samplewise_center=True, samplewise_std_normalization=True, data_format="channels_last")