finetuning with caption_cn_large

Open dinglei8908 opened this issue 3 years ago • 2 comments

I wonder the mini batch size when finetuning with caption_cn_large model on a singe GPU like V100/A100(32G/40G) for caption task. I found that the max batch size is 2/4 on V100/A100. Is there any ways to increase the batch size? fp16 is already set

Oct 19 '22 02:10 dinglei8908

Try gradient accumulation with --update-freq

Oct 19 '22 06:10 JustinLin610

Try gradient accumulation with --update-freq

thanks for suggestion. max mini batch size is 2/4 on V100/A100 for caption task, does this make sense?

Oct 19 '22 08:10 dinglei8908