litellm icon indicating copy to clipboard operation
litellm copied to clipboard

[Feature]: Expose original HTTP headers returned by the LLM APIs

Open drobiazko opened this issue 1 year ago • 11 comments

The Feature

The LLM APIs send multiple headers in their response that are utilized in our system for analysis. When switching to the LiteLLM Proxy Server, these headers are not available any more. It would be greatly beneficial if the proxy could forward these headers to the client instead of omitting them

Motivation, pitch

HTTP headers like x-ratelimit-* and x-ms-region (Azure only) are extremely important for analytical purposes of the LLM usage.

Twitter / LinkedIn details

No response

drobiazko avatar Mar 11 '24 09:03 drobiazko