litellm
litellm copied to clipboard
BerriAI
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropi...
## Auth Errors Before PR - using OpenAI JS/Python ```js AuthenticationError: 401 status code (no body) at APIError.generate (/Users/ishaanjaffer/Github/litellm/litellm/proxy/tests/node_modules/openai/error.js:47:20) at OpenAI.makeStatusError (/Users/ishaanjaffer/Github/litellm/litellm/proxy/tests/node_modules/openai/core.js:255:33) at OpenAI.makeRequest (/Users/ishaanjaffer/Github/litellm/litellm/proxy/tests/node_modules/openai/core.js:294:30) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async...
## Budget Management - [ ] Add testing for `proxy_server.track_cost_callback` - writing to SpendLogs Table - [ ] Allow Users to Set Budgets per user per month https://github.com/BerriAI/litellm/issues/1544 - [...
### The Feature Have completion_cost support image generation models We already have all the pricing in model_cost_map ### Motivation, pitch enable consistent spend tracking ### Twitter / LinkedIn details _No...
Addresses - https://github.com/BerriAI/litellm/issues/1547 Changes: - New `response_cost` kwarg to the Logging object on success (sync + async). - Supports completion + embedding responses - Moves proxy spend tracking to use...
https://github.com/BerriAI/litellm/blob/e354b71df028312d8fcb2220467f92c37fbfcac5/litellm/utils.py#L9 https://setuptools.pypa.io/en/latest/pkg_resources.html > Use of pkg_resources is deprecated in favor of [importlib.resources](https://docs.python.org/3/library/importlib.resources.html#module-importlib.resources), [importlib.metadata](https://docs.python.org/3/library/importlib.metadata.html#module-importlib.metadata) and their backports ([importlib_resources](https://pypi.org/project/importlib_resources), [importlib_metadata](https://pypi.org/project/importlib_metadata)). Some useful APIs are also provided by [packaging](https://pypi.org/project/packaging) (e.g. requirements and version...
### The Feature When I cross my Budget on an API Key I see the following exception from the proxy. There is no `body` set ```js got this exception from...
### What happened? I tried this and it was not working, Azure GPT Vision returns streaming responses weirdly ### Relevant log output _No response_ ### Twitter / LinkedIn details _No...
### What happened? The issue is - I have ollama running on a wsl, so Dockers, browsers and any other application sees ollama running, only litellm doesn't seem to find...
### The Feature When doing an API call to get embeddings for 1 list of 100 strings, I expect the cache to remember *individually* each string instead of only the...
### The Feature - **Request prioritization** = Allow completion requests to be tagged and then prioritize the request to a specific endpoint based on the tag. For example, pass a...
