Qwen3-Coder icon indicating copy to clipboard operation
Qwen3-Coder copied to clipboard
verified publisher icon QwenLM

Qwen3-Coder-480B-A35B-Instruct-FP8，6卡部署报错

Open deepblacksky opened this issue 5 months ago • 1 comments

Qwen3-Coder-480B-A35B-Instruct-FP8，6卡部署报错

GPU：H20e-141G*6

vllm部署命令

vllm server --model xxxx --port xxx--host 0.0.0.0 --trust-remote-code --served-model-name Qwen3-Coder-480B-A35B-Instruct --gpu_memory_utilization 0.9 --tensor-parallel-size 6 --enable-prefix-caching --enable-chunked-prefill --enable-expert-parallel --max_num_batched_token 2048

报错信息：

Image

deepblacksky avatar Jul 25 '25 03:07 deepblacksky

用8卡试试，151936无法被6整除

NAKONAKO4 avatar Jul 25 '25 04:07 NAKONAKO4