langgenius
Non-standard model outputs: Optimizing handling of 'data: ' fields
Self Checks
- [x] I have searched for existing issues search for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
- [x] Please do not modify this template :) and fill in all the required fields.
1. Is this request related to a challenge you're experiencing? Tell me about your story.
When DIFY processes model outputs, some models with non-standard output formats may generate errors, such as the "Tianyi Cloud Xirang Model." By default, DIFY expects the output to start with "data: ", but the output format of the "Tianyi Cloud Xirang Model" is "data:" (note the absence of a space after "data:"). The standard format "data: " includes a space, whereas non-standard formats may omit it. In versions prior to 1.0, I manually modified the code in llm_generator.py to ensure compatibility. However, in versions after 1.0, the official release removed this part of the code and moved it to a plugin written in Go. Below is the trial address for the non-standard format model: https://www.ctyun.cn/act/xirang/deepseek. If possible, we hope that compatibility can be maintained for such cases. Looking forward to your response.
2. Additional context or comments
No response
3. Can you help us with this feature?
- [ ] I am interested in contributing to this feature.
@starweavingdream I think you can submit a PR to this repo https://github.com/langgenius/dify-official-plugins/tree/main/models/openai_api_compatible
Please do not forget to bump the version in the manifest. :)
@starweavingdream I have resolved this issue in https://github.com/langgenius/dify-plugin-sdks/pull/54. Could you please merge the PR? If there are any issues with the code, would you kindly give me a reply?
@bxfxf Sorry, I saw the changes you submitted. I made modifications in the same way in version 0.x, but now I don't know how to apply them. I'd like to know where the dify-plugin-sdk is referenced or how to make changes so that it can be applied to my project. Or do I need to develop a model plugin separately?
@crazywoola Hello, could you please merge bxfxf submission into the project?
@crazywoola Hello, could you please merge
bxfxfsubmission into the project?
I guess you need to wait for the official new versions of dify-plugin-sdks and dify-official-plugins to build, openai_api_compatible plugin have references to dify-plugin-sdks.
@bxfxf Okay, thank you for your help.
Apologies for the confusion earlier. This pull request still requires approval from the official developers. I submitted it several days ago, but there has been no review or any feedback regarding potential issues with the PR. @Yeuoly Hello，could you please review this PR https://github.com/langgenius/dify-plugin-sdks/pull/54?
Great! I've merged it, the changes will be released in few days.
Thank you very much!
Self Checks
- [x] I have searched for existing issues search for existing issues, including closed ones.[x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）[x] Please do not modify this template :) and fill in all the required fields.
1. Is this request related to a challenge you're experiencing? Tell me about your story.
When DIFY processes model outputs, some models with non-standard output formats may generate errors, such as the "Tianyi Cloud Xirang Model." By default, DIFY expects the output to start with "data: ", but the output format of the "Tianyi Cloud Xirang Model" is "data:" (note the absence of a space after "data:"). The standard format "data: " includes a space, whereas non-standard formats may omit it. In versions prior to 1.0, I manually modified the code in llm_generator.py to ensure compatibility. However, in versions after 1.0, the official release removed this part of the code and moved it to a plugin written in Go. Below is the trial address for the non-standard format model: https://www.ctyun.cn/act/xirang/deepseek. If possible, we hope that compatibility can be maintained for such cases. Looking forward to your response.
2. Additional context or comments
No response
3. Can you help us with this feature?
- [ ] I am interested in contributing to this feature.
Hi， how do you use the xirang model? I can not get any output
@Yeuoly Hello，When the dify-plugin-sdk version upgraded, do I need to manually update the dify_plugin version in the requirements.txt file under openai_api_compatible, or will it automatically reference the new SDK version?
@Yeuoly Hello，When the dify-plugin-sdk version upgraded, do I need to manually update the dify_plugin version in the requirements.txt file under openai_api_compatible, or will it automatically reference the new SDK version?
hmmm, you need to update it manually if you have specific the plugin sdk version