CodeGen icon indicating copy to clipboard operation
CodeGen copied to clipboard

Published 20 hours ago • verified publisher icon salesforce

memory out of error. Hardware requirements

Open Kushalamummigatti opened this issue 2 years ago • 1 comments

I have tried to finetune Codegen 2B Mono on 40GB GPU (single card) with sequence length set as 256. It gave CUDA memory out of error. What is the GPU memory required to finetune 2B and above models?

Kushalamummigatti avatar Jun 16 '23 17:06 Kushalamummigatti

Seems like an issue on your side. A 7Billion parameter model fits on a 12GB GPU (the 7B 4-Bit one a 8GB GPU).

The 2B model should fit on a 6GB card just fine.

Fusseldieb avatar Jul 24 '23 21:07 Fusseldieb