awslabs
Request for invocation token and response token for cost tracking and audit
Use case
It would be great if the default agent also supports adding or emitting a custom callback which gives the number of tokens used for the query. This would help in effective cost tracking on the users query.
Also, if this can auto capture in the storage backend, it would be super helpful.
Currently I can probably write a custom agent but have to figure out on how to get it integrated with custom storage.
Solution/User Experience
Solution probably to have the metadata capture at the end of response and store it's token count details to the corresponding chat history item record/row.
Alternative solutions
No response
New feature in progress that will allows to add more metadata https://github.com/awslabs/multi-agent-orchestrator/pull/193
this can be tracked with #278 See example here: https://awslabs.github.io/agent-squad/cookbook/monitoring/observability/#langfuse-integration-demo
