litellm
litellm copied to clipboard
[Bug]: Creating a user with model_max_budget does not return the specified model_max_budget in the reponse
What happened?
In the below "Relevant log output", you can see model_max_budget":{} in the response, even though I specified "model_max_budget": {"anthropic.claude-3-5-sonnet-20240620-v1:0": 1.0, "anthropic.claude-3-haiku-20240307-v1:0": 1.0} in the request
The problem is here, where we are not returning all of the parameters when creating our NewUserResponse.
https://github.com/BerriAI/litellm/blob/5a81bcd88c16b3ec93e648c3aeb44f24a888106e/litellm/proxy/management_endpoints/internal_user_endpoints.py#L200
It seems several other fields like model_rpm_limit and model_tpm_limit are missing as well.
Relevant log output
curl -X POST https://<YourProxyEndpoint>/user/new \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <YourApiKey>" \
-d '{
"user_email": "[email protected]",
"user_role": "internal_user",
"model_max_budget": {"anthropic.claude-3-5-sonnet-20240620-v1:0": 1.0, "anthropic.claude-3-haiku-20240307-v1:0": 1.0}
}'
{"key_alias":null,"duration":null,"models":[],"spend":0.0,"max_budget":1000.0,"user_id":"b7e5ee23-2708-4aa4-a27d-16bb05cc4057","team_id":null,"max_parallel_requests":null,"metadata":{},"tpm_limit":null,"rpm_limit":null,"budget_duration":"1mo","allowed_cache_controls":[],"soft_budget":null,"config":{},"permissions":{},"model_max_budget":{},"send_invite_email":null,"model_rpm_limit":null,"model_tpm_limit":null,"guardrails":null,"blocked":null,"aliases":{},"key":"<YourNewKey>","key_name":null,"expires":null,"token_id":null,"user_email":"[email protected]","user_role":"internal_user","teams":null,"user_alias":null}
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.53.1
Twitter / LinkedIn details
No response