why i use vllm inference deepseek v2 ,speed is low

[ZzzybEric]

i use vllm to inference deepspeed, use flask to deploy model. When the problem enters the model, it always gets stuck for a long time in the processd prompt step，the code i use is your example code

[luofuli]

https://huggingface.co/deepseek-ai/DeepSeek-V2/discussions/1 @ZzzybEric

[ran130683]

whats your gpu type？