Chinese-Llama-2-7b icon indicating copy to clipboard operation
Chinese-Llama-2-7b copied to clipboard

Published 20 hours ago • verified publisher icon LinkSoul-AI

4张p40的显卡，装载模型报错CUDA out of memory，但是每张卡24G呀，这个该怎么办

Open GzRichChen opened this issue 1 year ago • 1 comments

：torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 22.38 GiB total capacity; 22.18 GiB already allocated; 16.81 MiB free; 22.20 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

GzRichChen avatar Sep 17 '23 13:09 GzRichChen

请问下，你是怎么放到4张显卡上的呢

zozoteacher avatar Oct 20 '23 08:10 zozoteacher