流式处理结束标识不应该和回复内容合并

[lizt666888]

System Info / 系統信息

golang 1.21

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• [ ] docker / docker
• [ ] pip install / 通过 pip install 安装
• [ ] installation from source / 从源码安装

Version info / 版本信息

xinference, version 0.14.0.post1

The command used to start Xinference / 用以启动 xinference 的命令

docker 部署

Reproduction / 复现过程

{"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "1"}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " + 1"}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " 等"}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "于"}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "2"}, "finish_reason": null}]} {"id": "chat0e6df3fe-557d-11ef-ab44-0242ac110002", "model": "qwen1.5-chat", "created": 1723118107, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "。这是最基本的数学加法。"}, "finish_reason": "stop"}]}

Expected behavior / 期待表现

最后结束返回 我看openai是单独返回的一个标识 不会和内容合并在一起 这样有可能会导致内容截断

openai的例子 {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}]}

....

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

最后结束 只是结束 内容为空