Allow add custom AI assistant provider

[Loner1024]

Check for existing issues

• [X] Completed

Describe the feature

zed currently offers support for a limited number of AI assistants, but many model providers offer OpenAI-compatible interfaces, e.g., DeepSeek, and there are a number of projects that offer OpenAI-compatible interfaces for a wider range of models. There are also projects that provide compatibility layers for more models, e.g. simple-one-api. Users in some regions may need to use the OpenAI interface through a proxy service provider, e.g. users in China cannot pay for the OpenAI interface. Therefore, it is necessary to provide a configurable generic OpenAI interface access, which will bring more powerful AI assistant capabilities to zed. In this feature, users can customize the endpoint and API key of the OpenAI API, and users can configure multiple custom model providers. I'd love to be involved in zed's development of this feature!

If applicable, add mockups / screenshots to help present your vision of the feature

[dalpatsingh-jabsainfotech]

I was trying to add Groq but could not, guess still this feature is not available.

[susl]

Hi! I wanted to test Groq and Cerebras in Zed (both offer OpenAI Compatible API) and I assumed that it would be easy, since OpenAI is already supported. However, there's no configuration option for an OpenAI connection string as far as I can tell. Could you please add this configuration option?

[venables]

@susl we were able to get it set up for our OpenAI compatible endpoint at Crosshatch. We put instructions here, which you can tweak to make it point to Groq, etc.:

https://docs.crosshatch.app/integrations/using-crosshatch-in-zed

But essentially something like this:

"language_models": {
  "openai": {
    "version": "1",
    "api_url": "<BASE_URL>",
    "available_models": [
      {
        "name": "<MODEL_NAME>",
        "max_tokens": 10000
      }
    ]
  }
},
"assistant": {
  "provider": null,
  "default_model": {
    "provider": "openai",
    "model": "<MODEL-NAME>"
  },
  "version": "2"
}

[dalpatsingh-jabsainfotech]

@venables I was able to make Groq work using this method.

[susl]

@venables Thanks! That worked for me. Will try Crosshatch now too :) Switching between multiple OpenAI API compatible models is a mess, but it works for now.

[pchalasani]

But essentially something like this:

just adding the "language_models" setting suffices; in fact, having the "assistant" setting disables using any other model. When I remove that, then I can switch between many models, including "OpenAI" models pointing to groq:

 
  "language_models": {
    "openai": {
      "version": "1",
      "api_url": "https://api.groq.com/openai/v1",
      "low_speed_timeout_in_seconds": 600,
      "available_models": [
        {
          "name": "llama-3.1-8b-instant",
          "max_tokens": 10000
        },        
        {
          "name": "llama-3.1-70b-versatile",
          "max_tokens": 32000
        },
        {
          "name": "gemma2-9b-it",
          "max_tokens": 6000
        }        
      ]
    }
  }

[nirfse]

Is there a discussion thread or other issue to follow progress? I already use a few local APIs and about five third-party GPU compute platforms, and I imagine the number will only grow with time. The configurations of these APIs clash with each other, it's just not possible to use them under one custom "OpenAI" API key umbrella as all of them require different keys. This UI/configuration problem needs a solution.

[itsaphel]

Is there a discussion thread or other issue to follow progress?

I think you're looking for this? https://github.com/zed-industries/zed/discussions/23426

[dukebarman]

Some LLM installations use custom headers. For example, authorization headers. It would be great if I could specify these headers in settings for custom AI providers.

[westykid]

Hi! I wanted to test Groq and Cerebras in Zed (both offer OpenAI Compatible API) and I assumed that it would be easy, since OpenAI is already supported. However, there's no configuration option for an OpenAI connection string as far as I can tell. Could you please add this configuration option?

I found a way around this. Create a proxy for LMStudio. Annoying, I know, if you use that tool. The added benefit of this is the list of supported models is provided by your own implementation so now I add every model I want to try to the proxy.

It's a bit rough IMO - I got AI to help with Groq (which was a real pain) - but if you like I'll make the repo public so you can try it.

A screen shot of the 'LMStudio' choices - which are not LMStudio models (of course).

Here's a snippet from the configuration:

const MODEL_MAP = {
    "grok-2-1212": {
        url: "https://api.x.ai/v1",
        api_key: process.env.GROK_API_KEY,
        publisher: "x.AI",
    },
    "qwen-qwq-32b": {
        url: "https://api.groq.com/openai/v1",
        api_key: process.env.GROQ_API_KEY,
        publisher: "Alibaba Cloud",
    },

Finally, my settings.json:

    "assistant": {
        "default_model": {
            "provider": "lmstudio",
            "model": "qwen-qwq-32b"
        },
        "version": "2"
    },
    "language_models": {
        "lmstudio": {
            "api_url": "http://localhost:1234/v1"
        }
    },

BTW - groq is awesome with this tool!

[Loner1024]

It seems that modifying the configuration file can resolve this issue now. Consider closing this issue.

[HeavySnowJakarta]

It seems that modifying the configuration file can resolve this issue now. Consider closing this issue.

man before you close the issue i think we might configure an openai-compatible url directly rather than install an extra program. for non-openai-compatible providers, maybe the extension API could do the job

[HeavySnowJakarta]

The providers natively available with Zed seems to be a mess. If there is only ChatGPT or ChatGPT along with Claude on the world, it's great, but there are actually many, many providers

[awoimbee]

It seems that modifying the configuration file can resolve this issue now. Consider closing this issue.

The current workaround is very limited as we can only have one "OpenAI" provider.

E.g. I would like to setup 2 OpenAI compatible providers:

• My deployment of Open WebUI: where I can send it any data but the models are limited
• OpenRouter: to get access to huge models (but I can't send restricted/secret data there)

---

My current config is:

"language_models": {
    // "openai": {
    //   "api_url": "https://openrouter.ai/api/v1",
    //   "available_models": [
    //     {
    //       "name": "anthropic/claude-3.7-sonnet",
    //       "display_name": "Claude 3.7 Sonnet",
    //       "max_tokens": 128000
    //     }
    //   ],
    //   "version": "1"
    // }
    "openai": {
      "api_url": "https://my-super-server.example.com/api",
      "version": "1",
      "available_models": [
        {
          "name": "qwen2.5-coder:32b",
          "max_tokens": 128000,
          "display_name": "qwen-coder 2.5"
        }
      ]
    }
  },

[jessearmand]

My workaround is to use OpenRouter to replace openai, as they support almost every provider and open weights models available. In any case, how do we extend Zed editor so we can add our own configuration, any guides or documentation on that?

[iwinux]

What I want is:

{
    "openai-compatible-provider-1": { "api_url": "..." },
    "openai-compatible-provider-2": { "api_url": "..." },
    "...": {}
}

[excieve]

Just my two cents to this request.

In my day to day usage I need to use two language model "profiles": for work and for personal projects. Currently I don't think this is possible in Zed as you can configure only one provider of any given type and only one API key for it. But I can't use e.g. work config and API key for my personal projects and vice versa.

This means each time I need to "switch" I've got to change the configs, which is less than ideal. I've been using Roo Code recently and while it's glitchy and has a quirky UX it provides this functionality right away by letting the users set up their LLM profiles and switching easily between them. Going back to Zed I'm missing this very much.

[abdessalaam]

I've been using Roo Code recently and while it's glitchy and has a quirky UX it provides this functionality right away by letting the users set up their LLM profiles and switching easily between them. Going back to Zed I'm missing this very much.

Same... I am just trying Zed, and generally feel very positive, but I miss roo or continue... The built in AI is kind of ok if you use the standard paid models, but I would like to connect OpneRouter (I think there's WIP on this), be able to switch models easier, specify my own autocomplete (or edit predictions), for example from Ollama etc.... :(

[notpeter]

Zed supports configuring OpenAI compatible APIs. Docs: https://zed.dev/docs/ai/configuration#openai-api-compatible

Ideally at some point in the future Zed will better support multiple configured implementations of a provider (e.g. two OpenAI compatible APIs, work/personal API keys for Anthropic, etc) but have nothing to announce at this time.

Thanks for reporting.

[iwinux]

The response above basically says "too bad you want to use OpenAI AND other provider at the same time LOL".

[ar-jan]

It would be better to keep this open with a label like postponed or something since it's not completed.