simple-llm-finetuner icon indicating copy to clipboard operation
simple-llm-finetuner copied to clipboard

Published 20 hours ago • verified publisher icon lxe

Can Nivdia 3090 with 24G video memory support finetune?

Open pczzy opened this issue 1 year ago • 4 comments

pczzy avatar Mar 22 '23 13:03 pczzy

Yes! You can go even lower if you use shorter sequence length.

lxe avatar Mar 22 '23 14:03 lxe

How about 3060?

wizztjh avatar Mar 23 '23 02:03 wizztjh

I was able to use 15 batch size with 200 max length on 4090. 'train_samples_per_second': 3.713. Couldn't get higher batch size for some reason, even though only 17 gb was taken, and 7 was available.

Shiro836 avatar Mar 24 '23 23:03 Shiro836

I hope finetuning will work on my RTX 3080 10GB when deepspeed will be implemented...

leszekhanusz avatar Mar 26 '23 16:03 leszekhanusz