How should the normal losses like?

[WayneCho]

Hi, Can you give me an example how should the normal losses look like in the training process? I am training my model and after several epochs the loss became: (epoch: 6, iters: 3128, time: 0.511) G_GAN: 1.974 G_GAN_Feat: 1.775 G_VGG: 0.991 D_real: 0.009 D_fake: 0.008 is the model collapsed?

[kex243]

Not always ot means that model has collapsed. Collapsed model is ususaly blask screen or some kinda light spot artefact (actualy not the worst thing) .If image quality visualy is being improved slowly you can try to decrease learning rate even more to let the model apply small improvments. Or increase and than decrease it for some epochs. Loss will be close to 0.001 to 0.000 and its okay if quality is getting better. IMHO some traning takes weeks to perform significant gap in quality for some features.

[WayneCho]

Not always ot means that model has collapsed. Collapsed model is ususaly blask screen or some kinda light spot artefact (actualy not the worst thing) .If image quality visualy is being improved slowly you can try to decrease learning rate even more to let the model apply small improvments. Or increase and than decrease it for some epochs. Loss will be close to 0.001 to 0.000 and its okay if quality is getting better. IMHO some traning takes weeks to perform significant gap in quality for some features.

Thank you for your reply. I have already given a very small learning rate, e.g., 1e-6 at the very beginning, the D_fake and D_real still drop to very small values after several steps. The image quality seems still improving, but does this mean the discriminator can't tell the real and fake images, the quality improvement relies on the feature matching loss and perception loss?

[zhongtao93]

"the D_fake and D_real still drop to very small values after several steps", I get this training results too. But I thick that should mean the discriminator D is too strong instead of too fool. Have you tried to decrease the learning_rate of D?