Firefly icon indicating copy to clipboard operation
Firefly copied to clipboard
Published 1 year ago • verified publisher icon yangjianxin1

多卡load模型OOM

Open TonyUSTC opened this issue 1 year ago • 0 comments

遇到一个诡异的情况，qlora微调llama3-8b模型，单卡可以加载模型运行，多卡在load权重的时候就OOM了，use_unsloth设置为false， 观察显存占用，发现只有gpu0显存一直在涨，直到OOM. image image

TonyUSTC avatar Aug 19 '24 02:08 TonyUSTC