latent-diffusion icon indicating copy to clipboard operation
latent-diffusion copied to clipboard

How to set the base_learning_rate in finetune autoencoderKL?

Open haooxia opened this issue 1 year ago • 2 comments

How to set the base_learning_rate in finetune autoencoder? Should I modify the default base_learning_rate: 1.6e-7

It can be seen that the learning rate changes with num_gpus, batchsize and base_learning_rate: image

haooxia avatar Aug 30 '23 08:08 haooxia

Set --scale_lr=False, then the learning rate will not be modified.

OwalnutO avatar Sep 01 '23 07:09 OwalnutO

Set --scale_lr=False, then the learning rate will not be modified.

Thanks for the reply, the base_learning_rate given by the repository is 4.5e-6, so what value should I shrink to when finetune on my own dataset?

haooxia avatar Nov 02 '23 14:11 haooxia