stanford_alpaca Finetune with A100 40G

Finetune with A100 40G

Open jianchaoji opened this issue 2 years ago • 4 comments

Can we use A100 40G to finetune llama-7B? Is there anyone try that?

Jun 14 '23 00:06 jianchaoji

I try 8 A100 40g to finetune llama-7B with FSDP offload, it works fine for me.

Jun 20 '23 07:06 GasolSun36

Thank you so much for the response! Did you try 4 A100 40G as well?

Jun 20 '23 16:06 jianchaoji

I tried 4 A100 40GB with FSDP offload, but had to reduce the eval and train batch size from 3 to 2 in order to avoid OOM. Took 58 hours.

Jun 30 '23 02:06 ffohturk

I tried 4 A100 40GB with FSDP offload, but had to reduce the eval and train batch size from 3 to 2 in order to avoid OOM. Took 58 hours.

I tried the same configuration 4 A100 40G, but it still OOM. Can you publish your parameter settings? Thanks! @ffohturk

Jan 31 '24 02:01 hychaochao