higress icon indicating copy to clipboard operation
higress copied to clipboard

使用AI-Proxy插件代理 qwen 模型进行流式返回时因字段不全造成 golang sdk 解析失败

Open daixijun opened this issue 1 month ago • 0 comments

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

  • [x] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

使用 Higress 配置 ai-proxy 插件来代理 dashscope / openai 等模型,在使用社区 golang sdk 进行流式返回的时候发现使用 qwen-plus 模型时, golang 侧解析出来的response结构体为空,但使用 gpt-4o-mini 模型时并没有问题

通过在终端使用 curl 测试两者区别发现使用 qwen 系列模型时返回结构中缺少部分值为 null 的字段

qwen系列返回数据 image

openai 系列返回数据 image

在 sdk 的仓库里看到结构体定义中, system_fingerprint 、finish_reason 等字段是必须存在的(没有添加标签omitempty),导致了结构体解析失败 https://github.com/sashabaranov/go-openai/blob/2a0ff5ac63e460cbe44cccd0d4199d51bf8682a4/chat_stream.go#L38 https://github.com/sashabaranov/go-openai/blob/2a0ff5ac63e460cbe44cccd0d4199d51bf8682a4/chat_stream.go#L53

Ⅱ. Describe what happened

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

希望在返回的结构中与 Openai 保持一致,补全缺失的字段信息

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

  • Higress version:
  • OS :
  • Others:

daixijun avatar Jan 09 '25 07:01 daixijun