ScottLogic
Update the token counting functions when library supports tool_calls
Feature Request
upgraded OpenAI 3.x to 4.19 where function_calls is depreciated and replaced with tool_calls (which takes functions as input). The openai-chat-tokens library we use to count prompt tokens does not have the tool update supported yet so we have used a workaround to convert the tools to functions to pass to the token counter. See https://github.com/ScottLogic/prompt-injection/pull/590
Description
When openai-chat-tokens is updated, change the function estimateMessageTokens function in openai.ts to pass tool_call in as itself instead of a converting to function as workaround.
If applicable, add screenshots to help explain your request.
Additional context
There may be changes to the number of tokens used by a tool vs. a function - may need to investigate when change is released.
Acceptance criteria
GIVEN openai-chat-tokens has been updated to use tool_calls WHEN a function is called THEN the correct tokens are counted & unit tests pass
Adding the blocked label until the library supports tool_calls.
