vllm
vllm copied to clipboard
[Feature]: Deepseek R1 GGUF 4bit(Q4KM) support
🚀 The feature, motivation and pitch
Now I run vllm on deepseek R1 GGUF 4bit(Q4KM), but it shows not supported. Can you support this?
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.