🚨 wren-ai-service crashes with Bus error (core dumped) when using custom LLM provider (Groq / OpenAI-compatible API)

Open anshulpwappgo opened this issue 4 months ago • 1 comments

Hi team 👋,

I’m trying to run Wren AI with Groq’s OpenAI-compatible API as the LLM provider instead of OpenAI. I configured my .env like this:

COMPOSE_PROJECT_NAME=wrenai
PLATFORM=linux/amd64

PROJECT_DIR=.

# service port
WREN_ENGINE_PORT=8080
WREN_ENGINE_SQL_PORT=7432
WREN_AI_SERVICE_PORT=5555
WREN_UI_PORT=3000
IBIS_SERVER_PORT=8000
WREN_UI_ENDPOINT=http://wren-ui:${WREN_UI_PORT}

# ai service settings
QDRANT_HOST=qdrant
SHOULD_FORCE_DEPLOY=1

# vendor keys
OPENAI_API_KEY=
GROQ_API_KEY=gsk_*******************************************************************

# version
# CHANGE THIS TO THE LATEST VERSION
WREN_PRODUCT_VERSION=0.27.0
WREN_ENGINE_VERSION=0.18.3
WREN_AI_SERVICE_VERSION=0.27.1
IBIS_SERVER_VERSION=0.18.3
WREN_UI_VERSION=0.31.1
WREN_BOOTSTRAP_VERSION=0.1.5

# user id (uuid v4)
USER_UUID=

# for other services
POSTHOG_API_KEY=phc_nhF32aj4xHXOZb0oqr2cn4Oy9uiWzz6CCP4KZmRq9aE
POSTHOG_HOST=https://app.posthog.com
TELEMETRY_ENABLED=true
# this is for telemetry to know the model, i think ai-service might be able to provide a endpoint to get the information
GENERATION_MODEL=llama3-70b-8192
LANGFUSE_SECRET_KEY=
LANGFUSE_PUBLIC_KEY=

# the port exposes to the host
# OPTIONAL: change the port if you have a conflict
HOST_PORT=3000
AI_SERVICE_FORWARD_PORT=5555

# Wren UI
EXPERIMENTAL_ENGINE_RUST_VERSION=false

# Wren Engine
# OPTIONAL: set if you want to use local storage for the Wren Engine
LOCAL_STORAGE=.

config.yaml:

type: llm
provider: litellm_llm
timeout: 120
models:
  - alias: default
    model: groq/llama3-70b-8192
    context_window_size: 8192
    kwargs:
      max_tokens: 4096
      n: 1
      seed: 0
      temperature: 0
  - model: groq/mixtral-8x7b-32768
    context_window_size: 32768
    kwargs:
      max_tokens: 4096
      n: 1
      seed: 0
      temperature: 0
  - model: gpt-4.1-2025-04-14
    context_window_size: 1000000
    kwargs:
      max_tokens: 4096
      n: 1
      seed: 0
      temperature: 0
  - model: gpt-5-nano-2025-08-07
    context_window_size: 380000
    kwargs:
      max_completion_tokens: 4096
      n: 1
      seed: 0
      reasoning_effort: minimal
  - model: gpt-5-mini-2025-08-07
    context_window_size: 380000
    kwargs:
      max_completion_tokens: 4096
      n: 1
      seed: 0
      reasoning_effort: minimal
  - model: gpt-5-2025-08-07
    context_window_size: 380000
    kwargs:
      max_completion_tokens: 4096
      n: 1
      seed: 0
      reasoning_effort: minimal
---
type: embedder
provider: litellm_embedder
models:
  - model: text-embedding-3-large
    alias: default
    timeout: 120

---
type: engine
provider: wren_ui
endpoint: http://wren-ui:3000

---
type: engine
provider: wren_ibis
endpoint: http://ibis-server:8000

---
type: document_store
provider: qdrant
location: http://qdrant:6333
embedding_model_dim: 3072
timeout: 120
recreate_index: true

---
type: pipeline
pipes:
  - name: db_schema_indexing
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: historical_question_indexing
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: table_description_indexing
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: db_schema_retrieval
    llm: litellm_llm.default
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: historical_question_retrieval
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: sql_generation
    llm: litellm_llm.default
    engine: wren_ui
    document_store: qdrant
  - name: sql_correction
    llm: litellm_llm.default
    engine: wren_ui
    document_store: qdrant
  - name: followup_sql_generation
    llm: litellm_llm.default
    engine: wren_ui
    document_store: qdrant
  - name: sql_answer
    llm: litellm_llm.default
  - name: semantics_description
    llm: litellm_llm.default
  - name: relationship_recommendation
    llm: litellm_llm.default
    engine: wren_ui
  - name: question_recommendation
    llm: litellm_llm.default
  - name: question_recommendation_db_schema_retrieval
    llm: litellm_llm.default
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: question_recommendation_sql_generation
    llm: litellm_llm.default
    engine: wren_ui
    document_store: qdrant
  - name: intent_classification
    llm: litellm_llm.default
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: misleading_assistance
    llm: litellm_llm.default
  - name: data_assistance
    llm: litellm_llm.default
  - name: sql_pairs_indexing
    document_store: qdrant
    embedder: litellm_embedder.default
  - name: sql_pairs_retrieval
    document_store: qdrant
    embedder: litellm_embedder.default
    llm: litellm_llm.default
  - name: preprocess_sql_data
    llm: litellm_llm.default
  - name: sql_executor
    engine: wren_ui
  - name: chart_generation
    llm: litellm_llm.default
  - name: chart_adjustment
    llm: litellm_llm.default
  - name: user_guide_assistance
    llm: litellm_llm.default
  - name: sql_question_generation
    llm: litellm_llm.default
  - name: sql_generation_reasoning
    llm: litellm_llm.default
  - name: followup_sql_generation_reasoning
    llm: litellm_llm.default
  - name: sql_regeneration
    llm: litellm_llm.default
    engine: wren_ui
  - name: instructions_indexing
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: instructions_retrieval
    embedder: litellm_embedder.default
    document_store: qdrant
  - name: sql_functions_retrieval
    engine: wren_ibis
    document_store: qdrant
  - name: project_meta_indexing
    document_store: qdrant
  - name: sql_tables_extraction
    llm: litellm_llm.default

---
settings:
  doc_endpoint: https://docs.getwren.ai
  is_oss: true
  engine_timeout: 30
  column_indexing_batch_size: 50
  table_retrieval_size: 10
  table_column_retrieval_size: 100
  allow_intent_classification: true
  allow_sql_generation_reasoning: true
  allow_sql_functions_retrieval: true
  enable_column_pruning: false
  max_sql_correction_retries: 3
  query_cache_maxsize: 1000
  query_cache_ttl: 3600
  langfuse_host: https://cloud.langfuse.com
  langfuse_enable: true
  logging_level: DEBUG
  development: false
  historical_question_retrieval_similarity_threshold: 0.9
  sql_pairs_similarity_threshold: 0.7
  sql_pairs_retrieval_max_size: 10
  instructions_similarity_threshold: 0.7
  instructions_top_k: 10

docker-compose.yaml file:

version: "3"

volumes:
  data:

networks:
  wren:
    driver: bridge

services:
  bootstrap:
    image: ghcr.io/canner/wren-bootstrap:${WREN_BOOTSTRAP_VERSION}
    restart: on-failure
    platform: ${PLATFORM}
    environment:
      DATA_PATH: /app/data
    volumes:
      - data:/app/data
    command: /bin/sh /app/init.sh

  wren-engine:
    image: ghcr.io/canner/wren-engine:${WREN_ENGINE_VERSION}
    restart: on-failure
    platform: ${PLATFORM}
    expose:
      - ${WREN_ENGINE_PORT}
      - ${WREN_ENGINE_SQL_PORT}
    volumes:
      - data:/usr/src/app/etc
      - ${PROJECT_DIR}/data:/usr/src/app/data
    networks:
      - wren
    depends_on:
      - bootstrap

  ibis-server:
    image: ghcr.io/canner/wren-engine-ibis:${IBIS_SERVER_VERSION}
    restart: on-failure
    platform: ${PLATFORM}
    expose:
      - ${IBIS_SERVER_PORT}
    environment:
      WREN_ENGINE_ENDPOINT: http://wren-engine:${WREN_ENGINE_PORT}
    volumes:
      - ${LOCAL_STORAGE:-.}:/usr/src/app/data
    networks:
      - wren

  wren-ai-service:
    image: ghcr.io/canner/wren-ai-service:${WREN_AI_SERVICE_VERSION}
    restart: on-failure
    platform: ${PLATFORM}
    expose:
      - ${WREN_AI_SERVICE_PORT}
    ports:
      - ${AI_SERVICE_FORWARD_PORT}:${WREN_AI_SERVICE_PORT}
    environment:
      # sometimes the console won't show print messages,
      # using PYTHONUNBUFFERED: 1 can fix this
      PYTHONUNBUFFERED: 1
      CONFIG_PATH: /app/config.yaml
    env_file:
      - ${PROJECT_DIR}/.env
    volumes:
      - ${PROJECT_DIR}/config.yaml:/app/config.yaml:ro
      - ${PROJECT_DIR}/data:/app/data:ro
    networks:
      - wren
    depends_on:
      - qdrant

  qdrant:
    image: qdrant/qdrant:v1.11.0
    restart: on-failure
    expose:
      - 6333
      - 6334
    volumes:
      - data:/qdrant/storage
    networks:
      - wren

  wren-ui:
    image: ghcr.io/canner/wren-ui:${WREN_UI_VERSION}
    restart: on-failure
    platform: ${PLATFORM}
    environment:
      DB_TYPE: sqlite
      # /app is the working directory in the container
      SQLITE_FILE: /app/data/db.sqlite3
      WREN_ENGINE_ENDPOINT: http://wren-engine:${WREN_ENGINE_PORT}
      WREN_AI_ENDPOINT: http://wren-ai-service:${WREN_AI_SERVICE_PORT}
      IBIS_SERVER_ENDPOINT: http://ibis-server:${IBIS_SERVER_PORT}
      # this is for telemetry to know the model, i think ai-service might be able to provide a endpoint to get the information
      GENERATION_MODEL: ${GENERATION_MODEL}
      # telemetry
      WREN_ENGINE_PORT: ${WREN_ENGINE_PORT}
      WREN_AI_SERVICE_VERSION: ${WREN_AI_SERVICE_VERSION}
      WREN_UI_VERSION: ${WREN_UI_VERSION}
      WREN_ENGINE_VERSION: ${WREN_ENGINE_VERSION}
      USER_UUID: ${USER_UUID}
      POSTHOG_API_KEY: ${POSTHOG_API_KEY}
      POSTHOG_HOST: ${POSTHOG_HOST}
      TELEMETRY_ENABLED: ${TELEMETRY_ENABLED}
      # client side
      NEXT_PUBLIC_USER_UUID: ${USER_UUID}
      NEXT_PUBLIC_POSTHOG_API_KEY: ${POSTHOG_API_KEY}
      NEXT_PUBLIC_POSTHOG_HOST: ${POSTHOG_HOST}
      NEXT_PUBLIC_TELEMETRY_ENABLED: ${TELEMETRY_ENABLED}
      EXPERIMENTAL_ENGINE_RUST_VERSION: ${EXPERIMENTAL_ENGINE_RUST_VERSION}
      # configs
      WREN_PRODUCT_VERSION: ${WREN_PRODUCT_VERSION}
    ports:
      # HOST_PORT is the port you want to expose to the host machine
      - ${HOST_PORT}:3000
    volumes:
      - data:/app/data
    networks:
      - wren
    depends_on:
      - wren-ai-service
      - wren-engine

However, when starting via Docker (docker compose up), the wren-ai-service container repeatedly fails with the following logs:

Waiting for qdrant to start... qdrant has started. Waiting for wren-ai-service to start... /app/entrypoint.sh: line 56: 7 Bus error (core dumped) uvicorn src.main:app --host 0.0.0.0 --port $WREN_AI_SERVICE_PORT --loop uvloop --http httptools Timeout: wren-ai-service did not start within 60 seconds

I also noticed that when running wren CLI setup and choosing "Custom", the generated docker-compose.yaml seems to wipe out any custom command: overrides for uvicorn, which makes it hard to debug.

Steps to Reproduce

Install Wren AI on Windows 10 (Docker Desktop, WSL2 backend, platform linux/amd64).

Configure .env with Groq endpoint as shown above.

Run wren up → observe wren-ai-service container crash with Bus error (core dumped).

Expected Behavior:->

wren-ai-service should start and route requests properly to any OpenAI-compatible API (Groq, Azure OpenAI, etc.), not just OpenAI.

Actual Behavior:->

Container crashes immediately with Bus error before serving requests.

Questions:->

Does wren-ai-service currently support non-OpenAI OpenAI-compatible APIs (Groq, Azure)?

Is the crash related to uvicorn + uvloop/httptools on Windows + WSL2?

Should we instead configure Wren UI to call Groq directly, bypassing wren-ai-service?

Aug 20 '25 06:08 anshulpwappgo

@anshulpwappgo are you using our launcher to start Wren AI?

Sep 08 '25 03:09 cyyeh