microsoft
KernelConfig.CompletionBackends is closed, cannot add custom ai backend to the kernel.
Can we open CompletionBackends so we can use custom AI backends, my team has a service layer before OpenAI/AzureOpenAI so we need to inject a custom client.
@matiazo without disclosing private info, could you provide the technical changes you would make? we can try adding support for more scenarios
Goal is not add more implementation but open Kernel to accept new implementations of IBackendConfig and ITextCompletionClient. We also need to review usage of IEmbeddingGenerator implementations.
I'm working in a PR but the overall changes so far:
- add KernelConfig.AddCompletionBackend(IBackendConfig)
- add public ITextCompletionClient CreateAIBackend() to IBackendConfig
- Kernel.CreateSemanticFunction - replace a swtich on concrete implementation of IBackendConfig
right, that's in line with current work in progress:
add KernelConfig.AddCompletionBackend(IBackendConfig)
spot on. Currently the list is hardcoded, we're removing that and simply work with interfaces 👍
add public ITextCompletionClient CreateAIBackend() to IBackendConfig
that could be a temporary stop gap, but we're removing this hard coded assumptions about what kind of AI is used.
Kernel.CreateSemanticFunction - replace a swtich on concrete implementation of IBackendConfig
yup makes sense, in fact I would suggest limiting the changes to a minimum and only in this area, without worrying too much about hardcoding a new option. In other words it's ok following the current approach and hardcoding a new option, because we're taking care of that and it's a rather big change
@matiazo I have a PR almost ready addressing this, thanks for raising this issue
@matiazo this should help https://github.com/microsoft/semantic-kernel/pull/161 - Here's an example showing how to plug in custom backends: https://github.com/dluc/semantic-kernel/blob/dluc99-backends/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs
Updating thread with the link that is now in main https://github.com/microsoft/semantic-kernel/blob/main/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs
