Insecure usage recommendations

[joakim]

Exposing ollama on a public port is a bad idea.

https://thehackernews.com/2024/11/critical-flaws-in-ollama-ai-framework.html

Oligo said it found 9,831 unique internet-facing instances that run Ollama, with a majority of them located in China, the U.S., Germany, South Korea, Taiwan, France, the U.K., India, Singapore, and Hong Kong. One out of four internet-facing servers has been deemed vulnerable to the identified flaws.

"Exposing Ollama to the internet without authorization is the equivalent to exposing the Docker socket to the public internet, because it can upload files and has model pull and push capabilities (that can be abused by attackers)," Lumelsky noted.

Why can't Enchanted access ollama on 127.0.0.1?

[Josh-Voyles]

Would the dev be open to someone (or me) compiling instructions for how to setup with Tailscale? With Tailscale, you should be able to access your home instance with any of your devices connected to your tailnet.

[joakim]

If privacy is the reason for running ollama locally, I wouldn't use a privately owned VPN company. But with regards to security, it might be good enough for most.

I just want to access ollama on 127.0.0.1:11434. Why go to the trouble of using VPNs and online services when it's right there on the same machine? Am I missing something?

[Josh-Voyles]

If privacy is the reason for running ollama locally, I wouldn't use a privately owned VPN company. But with regards to security, it might be good enough for most.

I just want to access ollama on 127.0.0.1:11434. Why go to the trouble of using VPNs and online services when it's right there on the same machine? Am I missing something?

If I'm traveling, having a dedicated home server is going to be much more powerful than running locally. For the VPN, with Tailscale, you're the VPN provider. Tailscale just initiates the connection.

[joakim]

I see. I admit I only took a cursory look at Tailscale. I do see the usefulness of a setup like that for mobile use, I'm just naturally suspicious of venture capital funded companies.

[Josh-Voyles]

I see. I admit I only took a cursory look at Tailscale. I do see the usefulness of a setup like that for mobile use, I'm just naturally suspicious of venture capital funded companies.

For what's it's worth, you could also run the open-source fork Headscale.

[joakim]

Thanks for the tip, that's more down my alley :)

[arnab]

I have the same question.

Opening up my Ollama server to the world seems like a bad idea. I am on my Mac, and what I want to do is:

1. Run ollama locally
2. Run Enchanted
3. Connect Enchanted to the local ollama server (I don't want to open up my server to the world, and I don't need mobile access etc. I only need to access it from the machine I am on).

Is this possible? How?

[joakim]

I had a look at the code, and it does actually support running a local Ollama server! No ngrok needed.

You just have to set the server URI to http://127.0.0.1:11434 in Enchanted's settings. Make sure you use a firewall so that the Ollama server isn't publicly available.

[Josh-Voyles]

I have the same question.

Opening up my Ollama server to the world seems like a bad idea. I am on my Mac, and what I want to do is:

1. Run ollama locally
2. Run Enchanted
3. Connect Enchanted to the local ollama server (I don't want to open up my server to the world, and I don't need mobile access etc. I only need to access it from the machine I am on).

Is this possible? How?

Let's assume your running Ollama on a local Ubuntu server and not running in a container.

You can use a reverse proxy to access your server on your local network using caddy. You will need to edit the caddy file: (/etc/caddy/Caddyfile)

<your-server-ip>:11435 {
    reverse_proxy 127.0.0.1:11434
}

However, the tailscale approach is still easier since you can access everywhere and you don't have to keep switching addresses in enchanted. Just replace with your servers Tailscale IP. Caddy will also provision TLS certificates for a secure connection.

sudo systemctl restart caddy after editing the caddy file.

[adutta98]

I was trying for a similar approach. I have domain based hosting on my server. And I wanted to see if there was a way to secure this say using Oauth.

I have Authelia setup with a lot of my services and it works really well. Wondering if something like this could be done and that way my local instance could be shared between friends and family securely.

[rcanand]

My browser on iOS can connect to http://:11434 - why can't Enchanted? I agree with others here that exposing a public url is risky.