BUJIDAOVS comments

Repositories
Issues
Comments

Results 4 comments of


                                            BUJIDAOVS

trafficstars

Qwen-72B-Chat-Int4推理时间

一样，docker部署的72b-int4模型，单卡和双卡推理都非常慢

[Bug] ImportError: DLL load failed while importing _turbomind: 找不到指定的模块。

windows支持这么差吗，从安装到推理量化部署，报错没停过。有在win上跑通的案例吗

希望把 api 地址改为可配置的环境变量

+1，希望能支持本地部署的模型，我的本地模型兼容支持openapi

support qwen3 /think & /no_think & enable_thinking parameter

> Thanks again for your dedicated contributions. > > As I was testing the functionality, how are we expected to use this feature? Currently, I use the following commands after...