LLPhant
LLPhant copied to clipboard
Added token usage tracker for the latest response
I am not sure this fits your "style" for the project but i added a TokenUsage class to make it easy to retrieve the tokens used for the latest response.
CreateStreamedResponse did not have any usage parameter in the OpenAI library, for now.
Can now access the client and get the tokens for the latest response. I will look into some ways of adding it to the streamed responses.
Hope this is helpful!
Hey @samuelgjekic , Yes perfect. Can you fix the code style and then I'll merge :+1:
Hey im sorry for the spamming in commits i have been battling the linter. I got to make it so i can do the checks in the editor before commiting next time.
Anyways, i think the check was successful.
Thanks @samuelgjekic !
@samuelgjekic if you want to add some doc so that people know it exists and how to use it, it would be amazing