Azure comments

Repositories
Issues
Comments

Results 22 comments of


                                            Azure

[Bug] 部署unsloth/DeepSeek-R1-0528-GGUF/UD-Q8_K_XL 报错

> 试了一下，unsloth/DeepSeek-R1-0528-GGUF:Q4_K_M 的模型不报错，是对动态量化的版本没有比较好的支持吗您似乎使用了错误的yaml。跑多并发需要使用带名字中带有‘serve’的yaml，您可以参照`DeepSeek-V3-Chat-serve.yaml`来修改8卡的规则yaml

[Feature Request]: please add an embedding model "BAAI/bge-m3"

> You can add API key for NVIDIA/PerfXCloud/SILICONFLOW to apply bge-m3. hi, can you show more details about how to apply this? Thanks!@KevinHuSh