njzfw1024

Inference too slow

2

inference mmseg in win10, one line code costs 600s , but with the same code and config in ubuntu, it costs just a little .The same situlation in dataload. I...

worker_generate_stream接口在100并发量下响应非常慢

1

请问有哪里可以改进的地方嘛？ `@app.post('/worker_generate_stream') async def generate_stream(request: Request): global model_semaphore, global_counter global_counter += 1 params = await request.json() if model_semaphore is None: model_semaphore = asyncio.Semaphore(args.limit_model_concurrency) await model_semaphore.acquire() worker.send_heart_beat() generator = worker.generate_stream_gate(params) background_tasks...

[FEATURE] Qwen-agent 支持stream

1

请问是否有该计划

[BUG] 流式请求-前几个content内容为空的情况

2


api_v2.py 启动报错-HPU device module is not loaded

1

启动命令： `python api_v2.py -a 0.0.0.0 -p 7860 -c GPT_SoVITS/configs/tts_infer.yaml` tts_infer.yaml: ` custom: bert_base_path: GPT_SoVITS/pretrained_models/chinese-roberta-wwm-ext-large cnhuhbert_base_path: GPT_SoVITS/pretrained_models/chinese-hubert-base device: gpu is_half: false t2s_weights_path: GPT_SoVITS/pretrained_models/gsv-v2final-pretrained/s1bert25hz-5kh-longer-epoch=12-step=369668.ckpt version: v2 vits_weights_path: GPT_SoVITS/pretrained_models/gsv-v2final-pretrained/s2G2333k.pth ` 报错信息： ` Loading...

添加ollma渠道，需求访问api/chat而不是v1/chat/completions

5

**例行检查** [//]: # (方框内删除已有的空格，填 x 号) + [X] 我已确认目前没有类似 issue + [X] 我已确认我已升级到最新版本 + [X] 我已完整查看过项目 README，尤其是常见问题部分 + [X] 我理解并愿意跟进此 issue，协助测试和提供反馈 + [ X] 我理解并认可上述内容，并理解项目维护者精力有限，**不遵循规则的 issue 可能会被无视或直接关闭** **问题描述** 我使用渠道添加ollama, 然后post请求，返回的参数和渠道openai兼容一致。...