Out of memory with RTX3090

[PyxAI]

Hi, I'm trying to train gpt2xl, but keep getting OOM, even when I set batch size to 1 and gradient_accumulation to 8\16\512, contigous_gradients false and allgather_bucket_size \ reduce_bucket_size 2e2. I can see in nvidia-smi that I'm only reaching half the memory capacity - around 12GB My system is as stated - 3090 with 24GB memory 80 GB RAM 5600x cpu if that matters running WSL2 on windows 10 Thanks.

[PyxAI]

So working with WSL is just a no-go I installed dual boot ubuntu and now the problem disappeared

[BrandonKoerner]

dual boot only, huh... that sucks. I was really hoping I could use this on win10 or wsl(2)

[PyxAI]

I was however, able to run the model under WSL2 windows 11 Didn't check training, it's worth a shot @ReewassSquared

[uahmad235]

Hi @PyxAI . Which ubuntu version did you run this code on?