DeepSeek-V2 icon indicating copy to clipboard operation
DeepSeek-V2 copied to clipboard

Published 20 hours ago • verified publisher icon deepseek-ai

模型部署困惑

Open ylhou opened this issue 8 months ago • 2 comments

在A800 8卡机上按照Chat Completion的示例代码加载模型 设置 max_memory = {i: "75GB" for i in range(8)} device_map="sequential" 启动后显存占用集中在0卡~6卡，7卡被完全放空，然后测试的上下文稍长一点，就报显存无法分配了 这是什么原因呢，有人遇到过吗

ylhou avatar May 30 '24 02:05 ylhou