litellm icon indicating copy to clipboard operation
litellm copied to clipboard

[Bug]: Creating a user with model_max_budget does not return the specified model_max_budget in the reponse

Open mirodrr2 opened this issue 1 year ago • 0 comments

What happened?

In the below "Relevant log output", you can see model_max_budget":{} in the response, even though I specified "model_max_budget": {"anthropic.claude-3-5-sonnet-20240620-v1:0": 1.0, "anthropic.claude-3-haiku-20240307-v1:0": 1.0} in the request

The problem is here, where we are not returning all of the parameters when creating our NewUserResponse.
https://github.com/BerriAI/litellm/blob/5a81bcd88c16b3ec93e648c3aeb44f24a888106e/litellm/proxy/management_endpoints/internal_user_endpoints.py#L200

It seems several other fields like model_rpm_limit and model_tpm_limit are missing as well.

Relevant log output

curl -X POST https://<YourProxyEndpoint>/user/new \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <YourApiKey>" \
-d '{
     "user_email": "[email protected]",
     "user_role": "internal_user",
     "model_max_budget":  {"anthropic.claude-3-5-sonnet-20240620-v1:0": 1.0, "anthropic.claude-3-haiku-20240307-v1:0": 1.0}
 }'

{"key_alias":null,"duration":null,"models":[],"spend":0.0,"max_budget":1000.0,"user_id":"b7e5ee23-2708-4aa4-a27d-16bb05cc4057","team_id":null,"max_parallel_requests":null,"metadata":{},"tpm_limit":null,"rpm_limit":null,"budget_duration":"1mo","allowed_cache_controls":[],"soft_budget":null,"config":{},"permissions":{},"model_max_budget":{},"send_invite_email":null,"model_rpm_limit":null,"model_tpm_limit":null,"guardrails":null,"blocked":null,"aliases":{},"key":"<YourNewKey>","key_name":null,"expires":null,"token_id":null,"user_email":"[email protected]","user_role":"internal_user","teams":null,"user_alias":null}

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.53.1

Twitter / LinkedIn details

No response

mirodrr2 avatar Dec 05 '24 19:12 mirodrr2