default_backends is missing from .net PromptTemplateConfig

[Woland2k]

I want to be able to specify backend to use for skill inside config.json like this, so it overwrites default one used when creating kernel:

"default_backends": [ "text-davinci-003" ]

It was possible and included in example a few months ago. It also seem to be available in python implementation but not in .net.

[evchaki]

@Woland2k , thanks for raising this. This should already work for .net. Try it out and let me know if it isn't working. You will need to setup the other models in the kernel along with the default model.

[Woland2k]

@evchaki is that what I put in PromptTemplateConfig.DefaultServices? If it is, I don't see it being used anywhere in the .net code besides sample that shows how to import skills from the cloud storage:

DefaultServices = new List<string> { "text-davinci-003" }

Changing default model in kernel config makes all imported semantic skills to be registered with new model.

kernel.Config.SetDefaultTextCompletionService("text-babbage-001");

You can see in Kernel.CreateSemanticFunction DefaultServices is not used in anyway and the call is made like this:

func.SetAIService(() => this.GetService<ITextCompletion>());

I believe GetService here should pass name of the service configured for the function for the custom per function model to work.

[shawncal]

You can see in Kernel.CreateSemanticFunction DefaultServices is not used in anyway and the call is made like this:

func.SetAIService(() => this.GetService<ITextCompletion>());

I believe GetService here should pass name of the service configured for the function for the custom per function model to work.

@Woland2k Yes, you are correct. That is indeed a bug. I came across this one myself earlier this week, when attempting to refactor the AI services registry. https://github.com/microsoft/semantic-kernel/pull/632#discussion_r1177203413

Note, as mentioned in that comment, the solution I had in that PR only causes it to respect the intended delay load -- it doesn't fix the effect you've called out here.

[Woland2k]

Yes, I think it is a powerful capability to allow each prompt to have the model that fits it the best. Hopefully can be addressed in the timely manner.

[Woland2k]

@evchaki any update on this issue?

[dluc]

@Woland2k did you try replacing default_backends with default_services?

[Woland2k]

@dluc Yes, you can see the implementation doesn't take this into account when setting AI service to be used by Semantic function https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/Kernel.cs#L325

[dluc]

Thanks, so, yeah, this is part of a major change we need to make, inverting the way dependencies are used by functions. We need to store in functions a reference to the kernel, and then at runtime retrieve ITextCompletion from the kernel. This will also affect the definition of context, which currently offers the same functionality, but due to several edge cases it's not always available. For now, I would suggest trying to invoke SetAIService manually, until we sort this out.

[Woland2k]

can I invoke SetAIService? it might actually be fine in my case

[Woland2k]

So, I tried it and it seem to work, here is the code if anyone interested:

var functionConfig = new SemanticFunctionConfig(config, template);
var func = kernel.RegisterSemanticFunction(prompt.SkillName, prompt.FunctionName, functionConfig);
var serviceName = config.DefaultServices?.FirstOrDefault();
func.SetAIService(() => kernel.GetService<ITextCompletion>(serviceName));