Canner
Can DeepSeek be integrated as a base into wrenAI?
Does the official team intend to integrate DeepSeek into the project as a supported option, or is there currently no plan, requiring users to manually adapt and use it themselves?
@figo003 I think you could already use DeepSeek models now. Wren AI now uses Litellm under the hood to serve as LLM provider.
please refer to these two docs to setup DeepSeek model
- https://docs.litellm.ai/docs/providers/deepseek
- https://docs.getwren.ai/oss/installation/custom_llm#running-wren-ai-with-your-custom-llm-embedder-or-document-store
basically you need to add a new model entry for litellm_llm provider and change llm names in the section of pipeline definition
@cyyeh The deepseek API does not support the json_schema response format, but wrenai uses the json_schema response format by default in many places. Using deepseek directly will result in an error, as follows:
2025-01-09 08:22:01,859 - wren-ai-service - ERROR - An error occurred during question recommendation generation: litellm.BadRequestError: DeepseekException - Failed to deserialize the JSON body into the target type: response_format: response_format.type json_schema is unavailable now at line 1 column 37554 (question_recommendation.py:60)
@sdw777 so you mean it doesn't support structured output, right? if so, I think as of now we don't support DeepSeek. We could fix that I think.
@cyyeh DeepSeek doesn't support structured output.
Ok, we can fix that and support json_object
@figo003 I think you could already use DeepSeek models now. Wren AI now uses Litellm under the hood to serve as LLM provider.
please refer to these two docs to setup DeepSeek model
- https://docs.litellm.ai/docs/providers/deepseek
- https://docs.getwren.ai/oss/installation/custom_llm#running-wren-ai-with-your-custom-llm-embedder-or-document-store
basically you need to add a new model entry for
litellm_llmprovider and change llm names in the section of pipeline definition
As mentioned above, the support from DeepSeek is indeed not very good. Support for structured data is still needed and it's anticipated.
@cyyeh DeepSeek doesn't support structured output.
Ok, we can fix that and support json_object @cyyeh I'm also having the same problem, has this problem been solved?
@cyyeh DeepSeek doesn't support structured output.
Ok, we can fix that and support json_object @cyyeh I'm also having the same problem, has this problem been solved?
@Yuezhengming Not yet, would u like to contribute on this?
@figo003 @Yuezhengming @sdw777 Deepseek is supported now, please check the latest version: https://github.com/Canner/WrenAI/releases/tag/0.15.2 Also the deepseek config examples: https://github.com/Canner/WrenAI/blob/main/wren-ai-service/docs/config_examples/config.deepseek.yaml
