LLPhant icon indicating copy to clipboard operation
LLPhant copied to clipboard

Added token usage tracker for the latest response

Open samuelgjekic opened this issue 9 months ago • 2 comments

I am not sure this fits your "style" for the project but i added a TokenUsage class to make it easy to retrieve the tokens used for the latest response.

CreateStreamedResponse did not have any usage parameter in the OpenAI library, for now.

Can now access the client and get the tokens for the latest response. I will look into some ways of adding it to the streamed responses.

Hope this is helpful!

samuelgjekic avatar Apr 29 '24 21:04 samuelgjekic

Hey @samuelgjekic , Yes perfect. Can you fix the code style and then I'll merge :+1:

MaximeThoonsen avatar May 06 '24 07:05 MaximeThoonsen

Hey im sorry for the spamming in commits i have been battling the linter. I got to make it so i can do the checks in the editor before commiting next time.

Anyways, i think the check was successful.

samuelgjekic avatar May 06 '24 20:05 samuelgjekic

Thanks @samuelgjekic !

MaximeThoonsen avatar May 07 '24 09:05 MaximeThoonsen

@samuelgjekic if you want to add some doc so that people know it exists and how to use it, it would be amazing

MaximeThoonsen avatar May 07 '24 10:05 MaximeThoonsen