Support the new tools / tool_choice fields instead of deprecated functions / function_call

[prismosoft]

[aburkard]

For the new models, OpenAI seems to have changed the format somewhat. For instance, nested description fields used to not be included in the prompt, but now they are.

[andreibondarev]

@hmarr Thank you for this great library! Do you plan on supporting the new tools:[] parameter?

[hmarr]

Glad you like it! I'm keen to add support, I just don't have time personally right now. Either I'll get to it in due course, or I'd happily accept a PR.

[luizzappa]

I didn't test with different cases, but using tools in my use case, there wasn't much difference passing the functions of each tool. I have 12 tools, and the difference between the estimated tokens and the consumed tokens was only 2 (which has happened before only with functions)

// the data structure of tools is like this
"tools": [
    {
      "type": "function",
      "function": { 
         // ...
       }
     },
     {
      "type": "function",
      "function": { 
         // ...
       }
     },
     // ...
 ]
 
const promptTokens = promptTokensEstimate({ messages, functions: tools.map(t => t.function) });

[jpohhhh]

FWIW I had ported this to Dart in the fall, and realized something was off a day or two ago. Here's an updated Dart copy that's passing a bunch of challenging tests: https://www.diffchecker.com/IdI9D4lG/ It should port pretty cleanly, I just don't know TS that well and am kinda tired from working on it, not very fun :)

If you rely on token counting for billing and can afford up to 10 tokens in error, you're probably fine with the current implementation, with one exception:

If users can provide functions, or you have prebuilt functions with have descriptions on subparameters, I'd jump on this ASAP.

As @aburkard flagged above, previously, only top-level parameters in the response object had their description included. Now, sub-parameters do as well. So ex. my tests for that went from consuming ~100 tokens to ~250 tokens.

Beyond that, there's a lot of annoying little small changes that amount to a handful of tokens, some things off the top of my head:

• functions are on one line if none of the parameters have any title/description.
• if parameters have a "title" key, that is included
• if title and description are provided, they're both added with a commented out newline in between...
• lots of little "oh now there needs to be a , here in this special case"