Show tokens consumption and estimated cost for chat and multi-chat playground

[brnaba-aws]

Currently, there is no indication about the cost related to the call of an LLM. It would be interesting to understand how much the message affects the cost of invoking a LLM but it would be even more interesting when comparing models one to another. In this picture, the cost could be displayed at the bottom, but it could also be part of the metadata section.

[ystoneman]

Great idea, @brnaba-aws!

1. Do you think it would be useful to toggle this "cost insights" feature off and on in the settings?
2. Do you think it would be useful to show both the individual message cost (like in your awesome mockup) and the overall conversation cost?
3. Do you have any favorite frameworks/libraries that you'd recommend for implementing components of this feature?

Please feel free to contribute a draft pull request if you'd like to work on this -- all contributions welcome! And I'm ready to support you in that process.

[github-actions[bot]]

This issue is stale because it has been open for 60 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 30 days since being marked as stale.