Does it support other DeepSeek models?

[wuooo339]

I want to inference other DeepSeek models in V100 GPU.Does it support?Such as deepseek-ai's DeepSeek-R1-Distill-Llama-70B or DeepSeek-R1-Distill-Qwen-32B?

[drunkcoding]

These two models are llama and qwen by themselves but distilled using R1, we plan to support Qwen later

[wuooo339]

@drunkcoding Dose it support python3.12?

[drunkcoding]

We haven't tested yet, but given Python3 is is backward compatible, it should work. You might need to buiild wheel by yourself form source using BUILD_OPS=1 python3 -m build

[TheLogan6]

Dose it support deepseek-moe-16b-chat? Thanks