fraction_loss_gan

[houssem-tum]

Hello, in GAN.py fraction_loss_gan default value is 0,95. This means that the binary cross entropy loss has a bigger weight compared to vae loss. In the paper, an other alpha is defined, which has the value of 0.99. This means that the vae loss has a bigger weight compared to the binary cross entropy loss. Can someone please explain that for me?

[tawheeler]

Thanks for your interest @houssem-tum! The ratio between the losses will have to be tuned based on your particular dataset. We found that they have significantly different magnitudes. I recommend computing both loss contributions on your own data and seeing what ratio is most appropriate for you.

[houssem-tum]

Thank you for your answer @tawheeler. I have an other question: How could you evaluate the performance of your network during and at the end of the training? The generator at the end has to be more powerful than the discriminator, right?

[tawheeler]

That is a deep question that I am sure is still being actively researched. You don't want either the generator or the discriminator to get too good - if the generator is always fooling the discriminator, then your output is probably overfit on the training data - if the generator is never fooling the discriminator, then your output is clearly separable.

The simplest way to juggle this is to alternate between which model is trained. In our training script we simply alternate between 10 steps on each:

for step in range(1,args.nsteps):
		step = step + 1
		loss_D = train_discriminator(10)
		loss_G = train_generator(10)

I am sure there are more clever ways to go about it, but this worked reasonably well for the purpose of the research paper. Googling "how to train GANs" can probably get you a bunch of additional info.