RageLtMan

Can't use `127.0.0.1:50052` with docker - has to be the host IP/gateway name/etc. Strangely didnt seem to work with another docker hostname. that said: ``` time=2025-05-26T16:08:06.568Z level=INFO source=types.go:139 msg="inference compute"...

Apologies, to clarify: what is the execution pattern? Are the `ollama serve` instances connected to the RPC server dispatching work for it to execute or is the RPC server functioning...

@gkpln3 thanks, so i tried that but the RPC servers only come up in CPU compute mode. They dont detect the GPUs :-(

Yeah, same effect: ``` docker run --rm --name ollama_test --gpus=all -p 50052:50052 ollama:mnt rpc --host 0.0.0.0 --port 50052 Command 'ollama' not found, but can be installed with: snap install ollama...

To clarify: same exact container built from this PR starts with GPUs just fine in its default mode but in `rpc ...` it only presents CPU resources. The same container...

@gkpln3 (and probably @rick-github + team given the potential visibility of this track) - the team at [WhiteFiber](https://www.whitefiber.com/) have graciously offered a couple of 8-way B200 hosts and possibly some...

It is not happy doing that :-\\ ``` docker run --rm --name ollama_test --gpus=all -p 50053:50053 ollama:mnt rpc --device list --port 50053 2025/05/26 18:26:03 rpc_server.go:25: Starting RPC server on 0.0.0.0:50053...

> > @gkpln3 (and probably @rick-github + team given the potential visibility of this track) - the team at [WhiteFiber](https://www.whitefiber.com/) have graciously offered a couple of 8-way B200 hosts and...

Rebuilding :-)

No dice on 84aa6d0 unfortunately @gkpln3: ```sh $ docker run --rm --name ollama_test --gpus=all -p 50053:50053 ollama:mnt rpc --device list --port 50053 2025/05/26 19:26:07 rpc_server.go:26: Starting RPC server on 0.0.0.0:50053...