caBundlePath and verifySsl ignored for openai model

[stephanbertl]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Windows 11
- Continue: Continue.continue-0.9.71@win32-x64
- IDE: VSCode 1.86.2

Description

I am using a self-hosted Mistral on FastChat OpenAI API. The HTTPS certificate uses a custom CA.

Config:

"models": [
    {
      "title": "FastChat",
      "provider": "openai",
      "model": "Mistral-7B-Instruct-v0.2",
      "apiKey": "EMPTY",
      "apiBase": "https://openaiserver-private.local/v1",
      "requestOptions": {
        "caBundlePath": [
          "C:\\dev\\certs\\RootCA.crt",
          "C:\\dev\\certs\\SubCA.crt"
        ],
        "verifySsl": false,
      }
    }
  ],

This yields the following:

caBundlePath and verifySsl are both ignored.

To reproduce

No response

Log output

[Extension Host] FetchError2: request to https://openaiserver-private.local/v1/chat/completions failed, reason: unable to verify the first certificate
	at ClientRequest.<anonymous> (c:\Users\operatorstb\.vscode\extensions\continue.continue-0.9.71\out\extension.js:164601:14)
	at ClientRequest.emit (node:events:526:35)
	at TLSSocket.socketErrorListener (node:_http_client:501:9)
	at TLSSocket.emit (node:events:514:28)
	at emitErrorNT (node:internal/streams/destroy:151:8)
	at emitErrorCloseNT (node:internal/streams/destroy:116:3)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

[hbeaster]

Having the same issue. It works just fine in VScode but not in JetBrain's Rider

[stephanbertl]

@hbeaster how did you make it work in VS Code?

[stephanbertl]

Only way to get this to work is setting the environment variable NODE_TLS_REJECT_UNAUTHORIZED to 0. But this is terrible and shouldnt be done, because its system-wide.

Could someone please reproduce our issue? It makes continue unusable with local models behind HTTPS with own ca cerficate

[devwearsprada]

Same problem, can't use LocalAI instance with self signed certificate. Did you figure this out?

[stephanbertl]

Only workaround is setting NODE_EXTRA_CA_CERTS on the pc and installing another VS code extension that makes it available

https://marketplace.visualstudio.com/items?itemName=pharndt.node-extra-ca-certs-vscode

But this is a lot of pain.

[fm-knopfler]

Somehow, the option to disable TLS verification is ignored when set.

"models": [
    {
        "title": "codellama" ,
        "provider":"together",
        "model": "codellama/CodeLlama-70b-Instruct-hf",
        "apiKey": "api-key",
        "apiBase": "https://api.example.org/llm/v1",
        "template": "codellama-70b",
        "requestOptions": {
          "verifySsl": false
        }
    }
]

[jannikstdl]

We are facing the same issue with a local OpenAI compatible inference server.

Related #1474 #1423

[bdeb1337]

a considered temporary workaround would be to trust the certificate in the machine truststore.

for macos I did: sudo security add-trusted-cert -d -r trustRoot -p ssl -k /Library/Keychains/System.keychain /path/to/cert.crt