Canner
It is so difficult to setup the Ai Provider
I think we need a setting page to test if the LLM and Embedder works easily, now it is so wired, always show "failed to create ask"
Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like A clear and concise description of what you want to happen.
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.
@paopa , @cyyeh we might need to think about how to make AI provider integration easier. Seems a lot barrier for users.
Yes please, i have been trying to integrate Deepseek with ollama, for the past 4 days, and still it says Failed to create task, unable to deploy. Even tho I am not a tech sound person, I am still trying yet failing miserably.
Hey @ravenizzed, @realcarlos, could you guys share your thoughts with us? What kind of option would work better for you—setting up an independent page on the UI, a more dedicated document, or something else? Also, could you share which part of the configuration setup is the most challenging for you? Your feedback will really help us improve the setup flow. Thanks a lot!
Hi @paopa , thank you for your prompt reply. Usually we choose "custom" when init the setup, and there is no visual UI for us to test if the LLM and Embedder config works. I think if there is a Ai setting page in the "Setting" section will help. Just replace the config.yaml with a visual web page , and add a "Test connection" button.
If you want to make it more perfect, you can refer to the model configuration module of Dify and add all the existing AI inference services.
WrenAi looks good ,I hope I can experience it locally soon. Thank you very much.
Well, @paopa Let me share my recent experience with WrenAI. A quick background: I am not technical and can read okayish. When you access it through Openai, the process: OpenAI -> Key -> Model -> Data selection -> All good ( I did not have to look into any config files, or docker files) FYI: I made Gemini custom model run after 1 day of tweaks, which is an achievement for me. ^_^ But Custom -> read config files, change config files, replace config files, more changing config files. The option I would like to have:
- Custom -> which custom model am I using? as you already have config files for the custom models WrenAI supports.
- When in custom feature -> change the selected config right then and there with the user, Testing and editing and finalizing before moving ahead. 2.1 - I took me 3 days just to find out my local host was the problem and my deepseek-coder llm had to have the word Latest or it would not work. and the errors information is limited. 2.2 Custom -> Gemini -> Config file (you already have on github) -> make required changes, config, docker, env -> Test connection -> this step would have updated me to check my connection -> I worked eventually. 2.3 Things got interesting when I tried to use Ollama service with deepseek-coder and I have been stuck since 3 days (I actually had the connection problem here using ollama not gemini)
The point is, the GUI we have is fair enough for a starter piece, the user needs to be updated along while connecting, because if the connection works, then its just the data we have to worry.
i hope this minor step would change so much, because you already have this with OpenAI setup.
and thankyou for the quick response. Edit: i will share or open up a discussion with my current problems. (some json issue)
@paopa @qdrddr Let me explain more about why it is confused:
in .env ,there are some key strings:
LLM_OPENAI_API_KEY= EMBEDDER_OPENAI_API_KEY= OPENAI_API_KEY=
in config.yaml:
type: embedder provider: litellm_embedder models:
define OPENAI_API_KEY=<api_key> in ~/.wrenai/.env if you are using openai embedding model
please refer to LiteLLM documentation for more details: https://docs.litellm.ai/docs/providers
I wonder which key is the right one.
Hi @ravenizzed and @realcarlos, thanks so much for your feedback! We’ll chat in the team about how to make the config step even easier. If you have any more thoughts on the config setup, I think we can keep commenting on this issue. It’ll be a great option for us to make it easier!
@Nikita23526 Well, let's just discuss what is happening with you. I remembered there is a lot of updated information since I used Gemini and might share an irrelevant file that is already outdated, the config file in the example of GitHub is pretty self-explanatory but it is confusing nonetheless.
why don't you share your files, env, config, and docker so the devs can help you out as well.
I will resume with gemini in a couple of days, trying to setup dB via postgresSql to Redshift. So I can guide as best I can with the limited knowledge I have.
@ravenizzed I have used ollama mistral but i was unable to understand that do i need to put mistral api key ....2025-03-19 11:27:32 wren-ai-service-1 | File "/src/globals.py", line 49, in create_service_container
2025-03-19 11:27:32 wren-ai-service-1 | **pipe_components["semantics_description"],
2025-03-19 11:27:32 wren-ai-service-1 | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-19 11:27:32 wren-ai-service-1 | KeyError: 'semantics_description'
2025-03-19 11:27:32 wren-ai-service-1 |
2025-03-19 11:27:32 wren-ai-service-1 | ERROR: Application startup failed. Exiting. and i am getting this error although i have ollama.mistral in semantics_description inside pipeline
My Config file
LLM Configuration (Ollama Mistral)
type: llm provider: ollama timeout: 120 models:
model: mistral api_base: http://host.docker.internal:11434/ # Use http://localhost:11434/ if running outside Docker kwargs: temperature: 0 n: 1 max_tokens: 4096 seed: 0
Embedding Model Configuration
type: embedder provider: ollama models:
model: mxbai-embed-large api_base: http://host.docker.internal:11434/ timeout: 120
Wren Engine Configuration
type: engine provider: wren_ui endpoint: http://wren-ui:3000/
Document Store Configuration
type: document_store provider: qdrant location: http://qdrant:6333/ embedding_model_dim: 3072 timeout: 120 recreate_index: true
AI Pipeline Configuration
type: pipeline pipes:
name: db_schema_indexing embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: historical_question_indexing embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: table_description_indexing embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: db_schema_retrieval llm: ollama.mistral embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: historical_question_retrieval embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: sql_generation llm: ollama.mistral engine: wren_ui name: sql_correction llm: ollama.mistral engine: wren_ui name: followup_sql_generation llm: ollama.mistral engine: wren_ui name: sql_summary llm: ollama.mistral name: sql_answer llm: ollama.mistral name: sql_breakdown llm: ollama.mistral engine: wren_ui name: sql_expansion llm: ollama.mistral engine: wren_ui name: semantics_description llm: ollama.mistral name: relationship_recommendation llm: ollama.mistral engine: wren_ui name: question_recommendation llm: ollama.mistral name: question_recommendation_db_schema_retrieval llm: ollama.mistral embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: question_recommendation_sql_generation llm: ollama.mistral engine: wren_ui name: intent_classification llm: ollama.mistral embedder: ollama_embedder.text-embedding-3-large document_store: qdrant name: data_assistance llm: ollama.mistral name: sql_pairs_indexing document_store: qdrant embedder: ollama_embedder.text-embedding-3-large name: sql_pairs_retrieval document_store: qdrant embedder: ollama_embedder.text-embedding-3-large llm: ollama.mistral name: preprocess_sql_data llm: ollama.mistral name: sql_executor engine: wren_ui name: chart_generation llm: ollama.mistral name: chart_adjustment llm: ollama.mistral name: sql_question_generation llm: ollama.mistral name: sql_generation_reasoning llm: ollama.mistral name: sql_regeneration llm: ollama.mistral engine: wren_ui
General Settings
settings: engine_timeout: 30 column_indexing_batch_size: 50 table_retrieval_size: 10 table_column_retrieval_size: 100 allow_using_db_schemas_without_pruning: false query_cache_maxsize: 1000 query_cache_ttl: 3600 langfuse_host: https://cloud.langfuse.com/ langfuse_enable: true logging_level: DEBUG development: false
@Nikita23526, well the first thing i see is the embeddings, try using phi4 as stated in the config logs, check if that works. secondly, let me try mistrial myself to check the issue. Did you download the most updated config file for ollama.
try using nomic embeder. and the chat model phi4, if it works let me know, if it doesn't share the logs on discord, you can get quick responses from the devs logs for wren-ai-server, engine, ibis-server, ui
@ravenizzed As mistral is supported so i think we can use it and yes i have used most updated config file yet facing the issue
@Nikita23526 hey, sorry was a bit occupied, I still yet to test mistrial. Do you have any update if Gemini is working for you? From the updated config files you have to change a few things which I will share
The link of Google /v1/... Is not supported by litellm, if you can check litellms website to verify the link.
Second picture check the dim numbers for the embedding model you have chosen.
Type the exact name of the model from Google ai studio