LLM Visual Issues

[llx020727]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When LLM enabled the visual function and uploaded an image, calling LLM resulted in an error

✔️ Expected Behavior

Understand the content of the image

❌ Actual Behavior

ModelInvokeLLMRequest error

[dosubot[bot]]

I found a similar closed issue that addresses the "ValidationError" related to input type mismatch when using the "doubiao-vision" model in a self-hosted Dify version 1.1.3 setup. The solution is to upgrade your plugins to the following minimum versions:

• Azure OpenAI: version 0.0.11 or higher
• OpenAI: version 0.0.14 or higher
• OpenRouter: version 0.0.6 or higher

These updates address the validation error that occurs due to type checking issues with multimodal prompt messages in SDK version 0.0.1b74 ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[llx020727]

@dosu 我是Doubao-vision-pro-32k版本 如何解决

[dosubot[bot]]

I couldn't find specific information about the 'Doubao-vision-pro-32k' model in the Dify repository. However, if you're encountering a similar "ValidationError" related to input type mismatch, you might want to try updating your plugins to the recommended versions for the 'doubiao-vision' model, as this might also resolve issues with the 'Doubao-vision-pro-32k' model:

• Azure OpenAI: version 0.0.11 or higher
• OpenAI: version 0.0.14 or higher
• OpenRouter: version 0.0.6 or higher

These updates address validation errors related to type checking issues with multimodal prompt messages ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Jimmy-L99]

请问解决了吗？我用zhipu也是这种情况

[llx020727]

@Jimmy-L99 目前我测试新版本1.2.0好像还是存在该问题

[llx020727]

@Jimmy-L99 我测试该问题已在火山方舟模型新版本0.0.10已修复，zhipu问题你可能需要单独提出issues进行修复

[Jimmy-L99]

@Jimmy-L99 我测试该问题已在火山方舟模型新版本0.0.10已修复，zhipu问题你可能需要单独提出issues进行修复

是的，我现在使用的也是火山0.0.10。