Support dynamic apiKey

[AlexanderYastrebov]

Validations

• [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that requests the same enhancement

Problem

In our environment we'd like to use custom model served via proxy. This proxy requires API key that user may obtain via pre-installed authentication tool (get_token). The proposal is to support dynamic API key by using standard output of the tool.

Solution

Possible solution could be to modify config.ts such that it executes the tool:

const { execSync } = require("child_process");

export function modifyConfig(config: Config): Config {
  config.models
    .filter(model => model.apiKey && model.apiKey.startsWith("$"))
    .forEach((model) => {
      const cmd = model.apiKey.substr(1).trim();
      Object.defineProperty(model, "apiKey", {
        get() {
          return execSync(cmd, { encoding: "utf8" }).trim();
        },
      });
    });
  return config;
}

and config.json

{
  "models": [
    {
      "title": "myLLM",
      "provider": "openai",
      "model": "bar/baz",
      "apiBase": "https://example.com/v1",
      "apiKey": "$ get_token"
    }
  ],
...

Another option is to allow apiKey to be a function:

const { execSync } = require("child_process");

export function modifyConfig(config: Config): Config {
  config.models
    .filter(model => model.apiKey && model.apiKey.startsWith("$"))
    .forEach((model) => {
      const cmd = model.apiKey.substr(1).trim();
      model.apiKey = () => {
        return execSync(cmd, { encoding: "utf8" }).trim();
      };
    });
  return config;
}

The challenge is to ensure that apiKey property getter is invoked each time its needed to refresh the value.