[SUPPORT] How do you count function_call / tool call token count?

[dhensen]

How do you calculate token count for a python dict?

I'm coming from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb I adopted the code in this notebook and made a function ensure_messages_fit_limit that, given my messages, and limit, will prune my messages (with the exception of messages at the indices in keep_system_messages).

Right now I have token count for my functions hardcoded. I know this count because I reached the token limit and OpenAI raised an exception stating this number...

Any idea on counting this using tiktoken?

[willstott101]

Is How_to_count_tokens_with_tiktoken.ipynb meant to be the canonical source for this information from openai? Since there is no token information (not even input tokens) returned by streaming completion requests, and due to the gaps in in this document relating to function calling, response counting, and newer models - I don't see a feasible way to monitor usage of openai's services. Your own infrastructure does not support per-api-key monitoring so every single customer is left fending for themselves making up random numbers? There's a long thread here with seemingly no concrete conclusion https://community.openai.com/t/how-to-calculate-the-tokens-when-using-function-call/266573/55

OpenAI need the final event in the SSE stream to be usage or they need to maintain a usable reference for counting tokens.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[santiagotri]

Hello! Any updates on this? OpenAI needs to add usage information in stream completion requests. Or at least documentation to count token usage for tool calls. The only information given is that you can use tiktoken, but tiktoken does not have support for tool calls.