[FEATURE] api key for ollama

[somera]

Hi, as we know, Ollama does not have an API key to secure it. Nothing like that is planned (https://github.com/ollama/ollama/issues/849).

It would be nice If you could set for Ollama

Bearer ${OLLAMA_SECRET_API_KEY}

Then you can check something like that with a reverse proxy.

See https://github.com/kesor/ollama-proxy/blob/main/nginx-default.conf.template

Regards, Rafal

[fcorneli]

We run an Ollama instance behind a reverse proxy, basic HTTP authentication enabled. Hence it would be nice if you could define the following header somehow:

Authorization: Basic ...

The Continue plugin within Visual Studio Code already allows this via:

- name: Llama 3.1 8B (remote)
    provider: ollama
    model: llama3.1:8b
    apiBase: https://ai......be/ollama
    requestOptions:
      headers:
        Authorization: "Basic ..." 
    roles:
      - chat
      - edit
      - apply

[mydeveloperplanet]

This should not be that difficult to implement In the LocalChatModelFactory (and GUI settings of course)

Map<String, String> customHeaders = new HashMap<>();
customHeaders.put("Authorization", "Bearer your_token");
OpenAiChatModel model = OpenAiChatModel.builder()
    .baseUrl(baseUrl)
    .modelName("gpt-4o")
    .apiKey("your_api_key")
    .customHeaders(customHeaders)
    .build();

[fcorneli]

Unfortunately, not every Ollama builder exposes the method customHeaders. For example, OllamaModels does not have this method. I solved this by means of a custom HttpClientBuilder. Usage:

MyHttpClientBuilder httpClientBuilder = new MyHttpClientBuilder();
        httpClientBuilder.authenticate(USERNAME, PASSWORD);
        OllamaModels ollamaModels = OllamaModels.builder()
                .baseUrl(BASE_URL)
                .logRequests(true)
                .logResponses(true)
                .httpClientBuilder(httpClientBuilder)
                .build();

The implementation:

import dev.langchain4j.http.client.HttpRequest;
import dev.langchain4j.http.client.jdk.JdkHttpClient;
import dev.langchain4j.http.client.jdk.JdkHttpClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Base64;

public class MyHttpClientBuilder extends JdkHttpClientBuilder {

    private static final Logger LOGGER = LoggerFactory.getLogger(MyHttpClientBuilder.class);

    private String username;

    private String password;

    @Override
    public JdkHttpClient build() {
        LOGGER.debug("build");
        return new MyHttpClient(this);
    }

    public MyHttpClientBuilder authenticate(String username, String password) {
        this.username = username;
        this.password = password;
        return this;
    }

    public void config(HttpRequest.Builder httpRequestBuilder) {
        if (this.username == null) {
            return;
        }
        String auth = Base64.getEncoder().encodeToString((this.username + ":" + this.password).getBytes());
        httpRequestBuilder.addHeader("Authorization", "Basic " + auth);
    }
}

and

import dev.langchain4j.exception.HttpException;
import dev.langchain4j.http.client.HttpRequest;
import dev.langchain4j.http.client.SuccessfulHttpResponse;
import dev.langchain4j.http.client.jdk.JdkHttpClient;
import dev.langchain4j.http.client.jdk.JdkHttpClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MyHttpClient extends JdkHttpClient {

    private static final Logger LOGGER = LoggerFactory.getLogger(MyHttpClient.class);

    private final MyHttpClientBuilder builder;

    public MyHttpClient(JdkHttpClientBuilder builder) {
        super(builder);
        this.builder = (MyHttpClientBuilder) builder;
    }

    @Override
    public SuccessfulHttpResponse execute(HttpRequest request) throws HttpException {
        LOGGER.debug("execute");
        HttpRequest.Builder httpRequestBuilder = HttpRequest.builder()
                .url(request.url())
                .body(request.body())
                .method(request.method());
        this.builder.config(httpRequestBuilder);
        HttpRequest newRequest = httpRequestBuilder
                .build();
        return super.execute(newRequest);
    }
}

[fcorneli]

For streaming chat, you also need to override the following method of JdkHttpClient:

@Override
    public void execute(HttpRequest request, ServerSentEventParser parser, ServerSentEventListener listener) {
        LOGGER.debug("execute listener");
        HttpRequest.Builder httpRequestBuilder = HttpRequest.builder()
                .url(request.url())
                .body(request.body())
                .method(request.method());
        this.builder.config(httpRequestBuilder);
        HttpRequest newRequest = httpRequestBuilder
                .build();
        super.execute(newRequest, parser, listener);
    }