inference icon indicating copy to clipboard operation
inference copied to clipboard
Published 1 month ago • verified publisher icon xorbitsai

chat template test failed when I chose DeepSeek-R1-Distill-Qwen in model family

Open Nie-Yingying opened this issue 11 months ago • 3 comments

Image

xinference 1.2.1 vllm 0.6.1.post2 vllm-flash-attn 2.6.1 transformers 4.48.3 torch 2.4.0

Nie-Yingying avatar Feb 12 '25 07:02 Nie-Yingying

You can just choose model family deepseek-r1-distill-qwen, you don't need to specify chat template on your own.

qinxuye avatar Feb 12 '25 12:02 qinxuye

I don't change the chat template after I choose model family deepseek-r1-distill-qwen.

Nie-Yingying avatar Feb 13 '25 07:02 Nie-Yingying

I encountered the same issue while using Docker. Has it been resolved?

SharkSyl avatar Feb 19 '25 01:02 SharkSyl