kubectl-ai icon indicating copy to clipboard operation
kubectl-ai copied to clipboard

Published 20 hours ago • verified publisher icon GoogleCloudPlatform

LLM Self Signed Certificate

Open RGanor opened this issue 6 months ago • 3 comments

Hi Everybody!,

I'm using kubectl-ai to connect to a self-hosted OpenWebUI (with Ollama backend) which uses a self-signed certificate that I can't change.

Setup:

Client: kubectl-ai on windows 10 Server: Self-hosted OpenWebUI/Ollama with a self-signed certificate. Endpoint: https://<webui-address>/ollama Provider Type (in client): openai

Issue:

Running commands results in: OpenAI ChatCompletion API error: Post "https://<webui-address>/ollama/chat/completions:" tls: failed to verify certificate: x509: certificate signed by unknown authority

Feature Request:

Could you please add support for environments with self-signed certificates? This would be very helpful for self-hosted setups. Options could include:

Ignore TLS Verification: A flag like --insecure or --skip-tls-verify. Custom CA Path: A flag like --ca-bundle <path-to-ca.pem> or an environment variable to specify a trusted CA. This would enable users to connect to self-hosted services like Ollama/OpenWebUI in private or testing environments.

Thanks!

RGanor avatar May 11 '25 08:05 RGanor

Thanks @RGanor for sharing the use-case with details. We will try to get this addressed it this week.

droot avatar May 11 '25 17:05 droot

The company has its own internally developed LLM Proxy, which uses a private certificate. Therefore, it requires a skip-verify SSL approach; otherwise, the following error message will occur:

Error: simulated streaming failed during non-streaming call: OpenAI chat completion failed: Post "https://example.com/v1/chat/completions": tls: failed to verify certificate: x509: “example.com” certificate is not standards compliant

appleboy avatar May 12 '25 00:05 appleboy

@droot I created a new PR to fix the issue.

Image

appleboy avatar May 12 '25 02:05 appleboy

Thanks @appleboy for the fix. @RGanor would be great if you can try out this https://github.com/GoogleCloudPlatform/kubectl-ai/pull/217 and confirm if it fixes the issue for you.

droot avatar May 12 '25 22:05 droot

hey @droot ! ihve tried using --skip-verify-ssl but it still doesnt work

RGanor avatar May 15 '25 09:05 RGanor

@tuannvm @droot i think #207 override the changes made #217 PR. can you have a look on it?

RGanor avatar May 15 '25 10:05 RGanor

@RGanor, I created a new PR to resolve the OpenAI provider. See the https://github.com/GoogleCloudPlatform/kubectl-ai/pull/243

appleboy avatar May 18 '25 04:05 appleboy