Co-DETR icon indicating copy to clipboard operation
Co-DETR copied to clipboard
verified publisher icon Sense-X

How much VRAM is needed to finetune co_dino_5scale_vit_large_coco?

Open hee-dongdong opened this issue 1 year ago • 1 comments

I have 6 4090 GPUs (VRAM = 120GB). However, when I try to finetune the model, it shows "CUDA out of memory" error. How much VRAM is needed to train the ViT backbone model? I want to know how many GPUs you had when you pretrain the model.

hee-dongdong avatar Sep 13 '24 13:09 hee-dongdong

We use 56 A100 80G GPUs to pretrain the model. FSDP and deepspeed can help you reduce training memory consumption, please refer to this issue.

TempleX98 avatar Sep 20 '24 10:09 TempleX98