extensions
extensions copied to clipboard
[API Proposal]: ChatToolMode.RequireAnyOf
Background and motivation
MEAI currently supports 3 mode to control the output of the LLM:
- Auto: return tools or natural language
- RequireAny: only return tools
- RequireSpecific: only return this single tool
Providers also support a "RequireSpecific" but with a list of tools:
- tool_choice.tools for OpenAI
- toolConfig.functionCallingConfig.allowedFunctionNames for Google
- Not supported with Mistral. Should fallback on ANY
- Not supported with Claude. Should fallback on ANY
This proposal aims to support that.
API Proposal
public class ChatToolMode
{
+ public static RequiredChatToolMode RequireAnyOf(IList<string> functionNames);
}
public sealed class RequiredChatToolMode : ChatToolMode
{
/// <summary>
/// Gets the name of a specific tool that must be called.
/// </summary>
/// <remarks>
/// If the value is <see langword="null"/>, any available tool can be selected (but at least one must be).
+ /// If multiple functions are required, this property returns <see langword="null"/>.
/// </remarks>
public string? RequiredFunctionName { get; }
+
+ /// <summary>
+ /// Gets the names of specific tools that must be called.
+ /// </summary>
+ /// <remarks>
+ /// If the value is <see langword="null"/>, any available tool can be selected (but at least one must be).
+ /// </remarks>
+ public IList<string>? RequiredFunctionNames { get; }
public RequiredChatToolMode(string? requiredFunctionName) { }
+ public RequiredChatToolMode(IList<string> requiredFunctionNames) { }
}
I think it makes sense to set RequiredFunctionName to null and not the first tool if multiple tools are specified, that would make providers that don't support multiple tools to fallback on the ANY mode.
API Usage
var getCurrentLocationFunc = AIFunctionFactory.Create(() => "Paris", name: "get_current_location");
var getTemperatureFunc = AIFunctionFactory.Create((string location) => 15f, name: "get_temperature");
var response = await chatClient.GetResponseAsync(
"What's the weather like today?",
new ChatOptions
{
Tools = [getCurrentLocationFunc, getTemperatureFunc],
ToolMode = ChatToolMode.RequireAnyOf([getCurrentLocationFunc.Name, getTemperatureFunc.Name])
});
Alternative Designs
- Rename RequireAnyOf to RequireSpecific as it would be closer to the provider implementations but I think the former is clearer
- Use a list of AITool instead of a list of string because of https://github.com/dotnet/extensions/issues/6288
- Not providing this feature at all. It seems like this feature is not very different from ChatToolMode.RequireAny and filtering the tools.
Risks
- Some providers don't support the feature
- OpenAI .NET doesn't seem to support it in their client even if it's supported by OpenAI
- The two properties RequiredFunctionName + RequiredFunctionNames can be confusing.