improved-diffusion icon indicating copy to clipboard operation
improved-diffusion copied to clipboard

Published 20 hours ago • verified publisher icon openai

About model training

Open xinyuz8270 opened this issue 2 years ago • 4 comments

Do I need to take any other steps during the training process? After training for a few days, the loss did not continue to decrease and the sampling results obtained were also very blurry. Do anyone have any training suggestions? Here are some of my sampling results. I trained it for 250K iterations. real_data

xinyuz8270 avatar Oct 31 '23 01:10 xinyuz8270

Hi there. Did you solve your problem? I had the same issue. How many images do u have for the training?

cuonghoang29 avatar Feb 29 '24 07:02 cuonghoang29

I had the same issue. How many images do u have for the training?

1zeryu avatar Mar 10 '24 14:03 1zeryu

My dataset is only 1000 images, but after checking the code, I found that we should use random crop instead of using center crop (because only 1k images). In my case, If I train from scratch (4 GPUs A100), after 100k steps the generated images will be clearer. If I use a pre-trained model to train, after about 50k steps the generated would be better.

cuonghoang29 avatar Mar 13 '24 01:03 cuonghoang29

Hi there. Did you solve your problem? I had the same issue. How many images do u have for the training?

Sorry for not replying in time. The ImageNet dataset I used for training has about 1.2 million images. After checking my code, I found that my problem was caused by a coding error in the sampling process. So I'm sorry I can't answer your question.

xinyuz8270 avatar Apr 17 '24 11:04 xinyuz8270