openai-openapi icon indicating copy to clipboard operation
openai-openapi copied to clipboard

ChatCompletionToolChoice and ChatCompletionToolChoiceOption are unnecessarily complex

Open the-gigi opened this issue 9 months ago • 4 comments

The ChatCompletionToolChoice forces the model to call a specific tool. let's call it X. There are 3 options:

  1. The tools list doesn't contain a tool called X
  2. The tools list contains just the tool X
  3. The tools list contains X and additional tools

Option 1 is invalid. There is no reason to allow it. Option 2 is valid, but why specify the tool X twice? once in the list of tools and second time in the ChatCompletionToolChoice. Option 3 is valid but wasteful/confusing. If the goal is to call X, why even pass additional tools?

I suggest to completely get rid of ChatCompletionToolChoice. The same effect can be accomplished by passing a tool list that contains just X and the ChatCompletionToolChoiceOption "required".

Now, let's consider the ChatCompletionToolChoiceOption. "auto" and "required" are necessary and useful. But, "none" doesn't provide any value. If you pass "none" it tells the model to avoid calling any tool. If this is the case, then just pass an empty list of tools.

If following both recommendations the complex specification for tool choice (can be either ChatCompletionToolChoice or ChatCompletionToolChoiceOption) will collapse to super simple single enum ChatCompletionToolChoiceOption with just two values: "auto" and "enum". That's it.

This will considerably simply the life of OpenAI API users and OpenAI client libraries (especially if using strongly typed languages).

Actually, we can take it further. If the Choice is only "auto" or "required". We don't even need an enum. We can specify everything with a simple boolean "toolRequired" field. If it's "true" then a tool call is required. If it's not true (missing or "false") then it's auto (model decides which tool to call if any).

the-gigi avatar May 14 '24 08:05 the-gigi