Nie-Yingying
Results
2
comments of
Nie-Yingying
> this will load the model with half its memory and should solve your problem. I'll integrate this soon sorry to tell you and it's still oom 
I don't change the chat template after I choose model family deepseek-r1-distill-qwen.