litellm
litellm copied to clipboard
[Bug]: Cost not tracked for bedrock coverse when stream=False
What happened?
Here I make two requests to a bedrock converse endpoint, one with stream=True, and another one with stream=False:
Then in the logs page of the dashboard, the stream=False request shows $0.0 cost, while the cost of the stream=True request is tracked correctly:
Relevant log output
N/A
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.63.11-stable
Twitter / LinkedIn details
No response