DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Can I train a opt-6.7B model on 4x4090 gpus?

Open eggqq007 opened this issue 1 year ago • 4 comments

I get 4 4090 gpus, and I want to train the opt-6.7B by using DeepSpeed Chat. Is that possible? I mean I have no idea if I should get a GPU that has enough VRAM or maybe I can use 4 small gpus instead.

eggqq007 avatar Apr 14 '23 05:04 eggqq007

I have the same question

Benstime avatar Apr 18 '23 03:04 Benstime

How large memory does 4090 have? We have basic instruction in the tutorial/readme of step 3

yaozhewei avatar Apr 18 '23 16:04 yaozhewei

How large memory does 4090 have? We have basic instruction in the tutorial/readme of step 3

One single 4090 has 24Gb VRAM, so 4 4090s have a total of 96Gb. Is it possible to train the opt-6.7B?

JimEverest avatar Apr 22 '23 14:04 JimEverest

Looks like the memory is enough. Can you share the training script you are using for now?

yaozhewei avatar Apr 24 '23 03:04 yaozhewei

Close as no followup

yaozhewei avatar May 05 '23 18:05 yaozhewei