[Bug]: AttributeError: 'KeyError' object has no attribute 'request' for mistral

[YangQiuEric]

What happened?

Here is the litellm code: litellm --model mistral/mistral-medium --add_function_to_prompt

And I just run the agentchat_function_call.ipynb, here is the error:

[2024-01-22 21:57:58 -0800] [57078] [INFO] Application startup complete. [2024-01-22 21:58:09 -0800] [57075] [INFO] Handling signal: winch Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 5730, in exception_type request=original_exception.request, AttributeError: 'KeyError' object has no attribute 'request'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion response = await litellm.acompletion(**data) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async result = await original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 264, in acompletion init_response = await loop.run_in_executor(None, func_with_context) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper result = original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion optional_params = get_optional_params( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params "tools", non_default_params.pop("functions") KeyError: 'functions' 127.0.0.1:58634 - "POST /chat/completions HTTP/1.1" 500 Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 5730, in exception_type request=original_exception.request, AttributeError: 'KeyError' object has no attribute 'request'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion response = await litellm.acompletion(**data) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async result = await original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 264, in acompletion init_response = await loop.run_in_executor(None, func_with_context) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper result = original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion optional_params = get_optional_params( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params "tools", non_default_params.pop("functions") KeyError: 'functions' 127.0.0.1:58634 - "POST /chat/completions HTTP/1.1" 500 Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 5730, in exception_type request=original_exception.request, AttributeError: 'KeyError' object has no attribute 'request'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion response = await litellm.acompletion(**data) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async result = await original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 264, in acompletion init_response = await loop.run_in_executor(None, func_with_context) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper raise e File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper result = original_function(*args, **kwargs) File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion raise exception_type( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6743, in exception_type raise original_exception File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion optional_params = get_optional_params( File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params "tools", non_default_params.pop("functions") KeyError: 'functions'

And I notice that when I removed the @chatbot.register_for_llm, it will not have error.

Relevant log output

No response

Twitter / LinkedIn details

No response