google-gemini
Report total input tokens with cached and non-cached breakdown
Not showing the uncached token count prominently makes the total token usage seem much higher than it is as cached tokens are very cheap.
Gemini 3.0 makes a large # of small requests that much better utilize cached tokens which makes the total token count very misleading. We should instead show the number of non-cached tokens as the input tokens as that is the meaningful stat for most users.
We should likely indicate the # of cached tokens at least for API Key users but need to make sure we don't do it in a way that causes people to be confused about cost of overall efficiency.
@jacob314 Please help me clear some doubts about the UX , do you want to add another columns after the Output Tokens
Cached Token 2307 (24%)
My preference for the UX is it is as clear as possible that the what the number of uncached tokens is as that is the most critical number for performance and cost. We're still having discussions with UX on how to solve that.