Felix Lausch comments

Repositories
Issues
Comments

Results 2 comments of


                                            Felix Lausch

[FEATURE REQ] Add access to CompletionsUsage in StreamingChatCompletions

I have been wondering the same for months. Tracking usage is trivially easy for the non-streaming version but seems impossible for streaming.

[FEATURE REQ] Add access to CompletionsUsage in StreamingChatCompletions

For now it seems like the only feasible option is to count the token usage yourself. In my (limited) experiments the combination of the following two methods has been 100%...