access api from docker container

[robertsmaoui]

Hello, docker containers cannot access to http://127.0.0.1:11434/api/chat

so i installed docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

it works using docker exec -it ollama ollama run llama2

but i want using as API, it is possible , base url ??

Thanks

[jimscard]

Hi, it would be better to ask questions like this in the discord.

It looks like your docker run command is constructed incorrectly. Did it not throw an error?

The first argument of the -v flag is not in the correct format. It should be in the format of "host-path:container-path". In this case, it seems like you want to map a local folder called "ollama" to the container's "/root/.ollama" folder. If that's the case, the correct format would be:

-v /path/to/local/ollama:/root/.ollama

Also, you may need to pull the image first, e.g.

docker pull ollama/ollama

Then, you're going to need a model to use. I started with an empty "ollama" folder, so I connected to the terminal session in the running container via Docker Dashboard, and pulled a model, e.g.,

ollama pull tinyllama

Then I could access and use the API from my laptop's CLI, I tested using curl, e.g.,

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "tinyllama",
  "prompt":"Here is a story about llamas eating grass"
 }'

[jinnabaalu]

• I created a docker-compose.yml Created the PR with the same docker-compose

• Running as the container in the daemon mode with docker-compose up -d

• Post the model with API

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "tinyllama",
  "prompt":"Here is a story about llamas eating grass"
 }'

[mxyng]

@robertsmaoui I'm not sure what issues you're experiencing. The commands you provided should work as you'd expect.

$ docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
a28f0d7934d3c96066a70937fc1b99d280b37653b423d6e45e31f82ce0951087
$ curl -v localhost:11434/api/version
*   Trying [::1]:11434...
* Connected to localhost (::1) port 11434
> GET /api/version HTTP/1.1
> Host: localhost:11434
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Date: Mon, 08 Jan 2024 19:10:37 GMT
< Content-Length: 20
<
* Connection #0 to host localhost left intact
{"version":"0.1.18"}
$ ollama --version
ollama version is 0.1.18