[Bug]: ollama not wroking

[kittenvr]

trafficstars

Is there an existing issue for the same bug?

• [X] I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• [X] I have checked the existing issues.

Describe the bug

when i start openhands with ollama it just doesnt work

Current OpenHands version

ghcr.io/all-hands-ai/openhands

Installation and Configuration

ollama serve

# The directory you want OpenHands to modify. MUST be an absolute path!
export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://host.docker.internal:11434" \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:main

Model and Agent

model: deepseek-coder-v2:latest

Operating System

linux chachyos

Reproduction Steps

i dont know how to reproduce on other installs

Logs, Errors, Screenshots, and Additional Context

https://discord.com/channels/1222935860639563850/1281641914210455643

[enyst]

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

[kittenvr]

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

https://www.logpasta.com/paste/99927887-c724-425d-9ae5-07580bc64e13

[enyst]

I think the docker problem is the main issue there, but first, please also try to simplify the command a bit, and pull the 0.9 tag instead of main.

export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:0.9

About the docker issue: can you try to check if docker networking is working? Could start by restarting Docker and checking if it's up to date. That error doesn't seem related to openhands, but docker installation or configuration, I would say.

[ParkerrDev]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

[kittenvr]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

thanks but im not using ollama anymore, could you check this one out tho https://github.com/All-Hands-AI/OpenHands/issues/4586#issuecomment-2445365390

[jimmyzhao-57blocks]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

when I add --network host, I can't open the localhost:3000 in the browser.

[BridgerB]

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

This is not working for me. I am able to use

ollama run llama3.1

or

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "2+2?",             
  "stream": false
}'

But when I try your command I get this:

INFO:     Started server process [10]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO:     127.0.0.1:55928 - "GET /api/options/models HTTP/1.1" 200 OK
03:41:36 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55932) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed
INFO:     127.0.0.1:55928 - "GET /api/options/agents HTTP/1.1" 200 OK
INFO:     127.0.0.1:55928 - "GET /api/options/security-analyzers HTTP/1.1" 200 OK
03:41:39 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55940) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed

[skylarmb]

This is basic docker networking configuration. localhost and 127.0.0.1 refer to addresses within the docker network, not addresses on the host machine. This works fine with the docker command in the docs as long as you correctly specify the docker host as the location of your ollama server.

ollama serve

# from the docs, no modifications required
docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.13-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    -e LOG_ALL_EVENTS=true \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.13

Now open the UI and configure your provider using host.docker.internal to reference the docker host machine network. You don't need to specify an API key.

Works fine for me!

[BioCruz-sudo]

This is basic docker networking configuration. localhost and 127.0.0.1 refer to addresses within the docker network, not addresses on the host machine. This works fine with the docker command in the docs as long as you correctly specify the docker host as the location of your ollama server.

ollama serve

# from the docs, no modifications required
docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.13-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    -e LOG_ALL_EVENTS=true \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.13

Now open the UI and configure your provider using host.docker.internal to reference the docker host machine network. You don't need to specify an API key.

Works fine for me!

You are AMAZING! Thanks for helping me get this figured out, I got it working on my PC. A bit slow but hey at least it's free! Thanks a bunch :)

[enyst]

@mamoodi what do you think, could we add @skylarmb 's screenshot to our docs? You know that saying, an image is worth a thousand words!

[mamoodi]

That's great. I've queued this in my todo list and see in incorporating it. Thank you @skylarmb!!!

[TomLucidor]

@skylarmb should this type of Advanced usage be documented then, since the network configs are mostly trivial? (and yes, that includes adding custom Ollama models like Qwen and Phi that are usually not in the default selector)

[enyst]

@skylarmb should this type of Advanced usage be documented then, since the network configs are mostly trivial? (and yes, that includes adding custom Ollama models like Qwen and Phi that are usually not in the default selector)

You may want to check this out: https://docs.all-hands.dev/modules/usage/llms/local-llms

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for over 30 days with no activity.