[Feature Request] Support Gemini’s New Multimodal Output

[laipz8200]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Background

Google recently released the Gemini 2.0 Flash Experimental API, which includes support for generating images. This is an important step toward broader multimodal capabilities, and we anticipate future support for audio, video, and documents as well.

To keep Dify aligned with these advancements, we propose updating the backend to support Gemini’s new output formats while rendering them in Markdown format on the frontend.

Feature Scope

• Backend Update Only: No UI/UX changes are required; the frontend will render responses using Markdown.
• Support for Image Generation (Initial Focus): Since Gemini 2.0 Flash Exp already supports image generation, we should prioritize integrating this feature first.
• Extendable for Future Media Outputs: The implementation should be flexible enough to accommodate audio, video, and documents once Gemini enables them.

2. Additional context or comments

No response

3. Can you help us with this feature?

• [x] I am interested in contributing to this feature.

[zhangever]

I need this feature, too. How's the cr going?

[laipz8200]

@zhangever Thanks for your comment! We're currently testing this feature, and it shouldn't be too long before it's ready for you. 😌

[JacoBezuidenhout]

Heyy everyone! Thanks sooo much for the effort you guys put into this! Going to be amazing!