junie-explorer icon indicating copy to clipboard operation
junie-explorer copied to clipboard

Cost figures may be incorrect due to cumulative cacheInputTokens and GPT-5 inconsistencies

Open dmeehan1968 opened this issue 4 months ago • 0 comments

Conventional understanding was that cacheInputTokens on each LlmResponseEvent was cumulative through the session.

With Sonnet, one could use the cacheCreateInputTokens figure to cost individual events. However GPT-5 seems not to populate this field, and instead has what appears to be a cumulative cacheInputTokens, but the last event has a large inputTokens value which may be the cumulative total for regular and cache input tokens.

When calculating cost, I have included the cacheInputTokens as it tends to have a different (lower) cost to cacheCreateInputTokens, but this might have been a mistake due to the cumulative nature of it.

In GPT-5 metrics, the cacheCreateInputPrice isn't provided, which correlates with it not being calculated. However this means its also then not costed based on the calculation of the difference in cumulative price.

See #37 for how I'm fudging this for now.

dmeehan1968 avatar Aug 12 '25 13:08 dmeehan1968