k8sgpt icon indicating copy to clipboard operation
k8sgpt copied to clipboard

Published 20 hours ago • verified publisher icon k8sgpt-ai

Allow for support for Azure OpenAI Services

Open msandbu opened this issue 1 year ago • 9 comments

Since many (including myself) with organizations will be using ChatGPT as part of Azure OpenAI Service. The feature should also allow the ability to point to ChatGPT running on Azure OpenAI Services.

It is using the same API but different endpoint https://openaichatgpt.openai.azure.com, could for instance be a featureflag to point it to another endpoint?

msandbu avatar Mar 28 '23 12:03 msandbu

We have the ability to switch the --backend and an IAI interface which should make implementing it pretty easily. Do you know if there is a golang based SDK? The current SDK we use is tightly coupled to OpenAI public endpoints though I could look at whether we can modify that.

AlexsJones avatar Mar 28 '23 13:03 AlexsJones

Are you using this? https://github.com/sashabaranov/go-openai (Do not believe there is Perhaps it would be better if I get support for the Azure OpenAI endpoint there first make it easier to add the new backend here?

msandbu avatar Mar 28 '23 13:03 msandbu

I'm looking into this going forward. The SDK we are using has an open issue on support for Azure OpenAI, but I will look into several solutions for this.

roberthstrand avatar Mar 30 '23 07:03 roberthstrand

I feel we might want to build our own SDK to control our destiny a bit more.

AlexsJones avatar Mar 30 '23 08:03 AlexsJones

I was thinking of that as well. Let's have a discussion about it tomorrow, when I got more time :)

roberthstrand avatar Mar 30 '23 08:03 roberthstrand

This is a solution：https://github.com/stulzq/azure-openai-proxy

stulzq avatar Mar 30 '23 08:03 stulzq

Are you using this? https://github.com/sashabaranov/go-openai (Do not believe there is Perhaps it would be better if I get support for the Azure OpenAI endpoint there first make it easier to add the new backend here?

Hi! It seems that sashabaranov/go-openai already supports Azure OpenAI. https://github.com/sashabaranov/go-openai/releases/tag/v1.6.0

jnytnai0613 avatar Apr 17 '23 09:04 jnytnai0613

I have created a draft PR to introduce azure open ai provider, bear with us a little longer as it also includes other changes that need to test first before merging it

arbreezy avatar Apr 19 '23 01:04 arbreezy

@msandbu , would you be able to beta-test the feature since I don't have an azure openai deployment up and running?

arbreezy avatar Apr 20 '23 15:04 arbreezy