semantic-kernel
semantic-kernel copied to clipboard
Token usage stats from responses should be returned to the client.
OpenAI / Azure OpenAI both return token stats with responses.
These are useful for tracking usage etc... on a case by case basis for things like "inter-departmental billing" etc...
Currently SK only supports returning the completion (string), meaning token usage is not accessible to the end user & unavailable for future evaluation / tracking.
SK, should return an object containing token stats & completion string.
Example use case(s):
- A large enterprise (parent company pays the bills) with subsidiaries (AI implementation end users). The parent will want to track token usage by each subsidiary to ensure cross charging for usage is correct.
- Stored as a statistic in an end users account. This allows the end user to see & be aware of token usage over time VS individual queries.
@SOE-YoungS , great idea, we will take a look at getting this added.