[Feature Request] Add Token Usage Logging

[choseh]

Hey,

first of all, thank you for your work! By far the best solution out there right now :)

Would it be possible to add a few logging options, such as logging the token cost? Since openai itself does not allow accounting for individual api keys, it would be nice to have an overview of how much tokens the individual bot instance has used/requested. This might be a very niche request, but I'd appreciate it.

Thanks!

[seratch]

Hi @choseh, I am glad to hear that! Just printing the "usage" part of OpenAI API responses would be relatively easy. Perhaps, we can add a new logger (say, "usage" logger) to write the logs.

[seratch]

Before switching to the streaming mode, the "usage" property existed in the OpenAI response. However, it seems that OpenAI API does not return "usage" data for streaming chunks. So, we may not be able to implement this as of today.