Trick 11: How to schedule thresholds A and B?

[benstaf]

I want to use trick 11, because I have a collapsed GAN problem: discriminator is too strong (accuracy=1, loss =0). I want to use the trick:

while lossD > A: train D while lossG > B: train G

But how to schedule A and B at each step? Do you have a principled approach? For example, can I choose (A_{n+1}, B_{n+1})= 0.5 * (A_n, B_n) ? What do you recommend?

Do you have other suggestions? New references on this topic?

[danielkaifeng]

I tried this by adjust different learning rate to G and D, but the results is even worse.

[florian-boehm]

I want to propose a slightly different approach of avoiding the collapse that might be easier to formulate:

while lossD > A: train D while lossD < B: train G

The idea behind the changed part is the following: If the discriminator is too bad, then lossD >= B and the generator should not be trained anymore because the generator could start outperforming the discriminator ("helvetica scenario" if I am correct?).

Setting the threshold on the discriminator loss was more comprehensable for us, but this will be open for discussion. Even though we are currently using this approach I am very sorry that I can not provide any proof that we successfully avoided the collapse thanks to it.