[BUG]: SSL Certificate Verification Issue When Hosting Ollama on Vast.ai [BUG or FEATURE req?]

[yerbaMatte]

Describe the bug

I am hosting Ollama on Vast.ai and encountered an SSL certificate verification issue and unexpected error.

Steps to reproduce

LLM_MODEL_TYPE = 'ollama' LLM_MODEL = 'mistral:v0.3' LLM_API_URL = "https://<VAST-IP>:PORT"

Branch

Current

Python version

3.12

LLM Used

ollama

Model used

mistral

Additional context

I am hosting Ollama on Vast.ai and encountered an SSL certificate verification issue. The specific error message I received is as follows:

ai_hawk.llm.llm_manager:__call__:292 - Unexpected error occurred: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

At first, the problem appeared when attempting to make a simple curl request from the terminal to the API, resulting in an SSL error. This error was related to the inability to verify the certificate due to missing issuer certificates.

Curl Issue: The issue persisted even when I tried curl from the terminal. The certificate verification failed with a similar error.

Installation of Vast.ai Certificate: After installing the certificate from Vast.ai to my system's trusted certificates store, the curl issue was resolved, and the SSL error no longer appeared in the terminal.

Python Still Affected: Despite resolving the issue with curl, the error still appears when making SSL requests within Python.

Edit. I've manually added SSL cert directly to the console before running script and right now this is the error i get, hard to say, what's going on... Has some tried before hosting Ollama on some cloud or that feature is not accessible at this point?

2024-11-17 12:09:53.822 | ERROR | ai_hawk.llm.llm_manager:__call__:380 - Unexpected error occurred: ... empty

Maybe its not a bug actually, but request about Remote Ollama feature? 😅 Not sure, can't wait for your replies guys!