Has anyone succeeded in reproducing the results?

[nashory]

trafficstars

I am still struggling with training VQ-GAN in the first stage, not even the conditional transformer which is a second stage. The result looks fine before the discriminator loss is injected. BUT using the discriminator loss suddenly ruins the reconstructed images. disc_loss remains 1.0 during the training. Why??

[richcmwang]

The loss goes up with the default parameters/set up very early in the training even before the discriminator kicked in. I am puzzled by this.

[mhh0318]

The same, using discriminator loss makes my result worse.

[nicolasfischoeder]

once the discriminator kicks in everything goes crazy, does somebody have an advice?

[function2-llx]

once the discriminator kicks in everything goes crazy, does somebody have an advice?

Perhaps because the discriminator is not trained at all until the g_loss is introduced to the total loss. According to the code, both the training of discriminator and the introduction of g_loss starts at the self.discriminator_iter_start step. Perhaps we can try training discriminator from the very beginning.

https://github.com/CompVis/taming-transformers/blob/3ba01b241669f5ade541ce990f7650a3b8f65318/taming/modules/losses/vqperceptual.py#L106-L107

https://github.com/CompVis/taming-transformers/blob/3ba01b241669f5ade541ce990f7650a3b8f65318/taming/modules/losses/vqperceptual.py#L129-L130