[Bug]: Error with Function Calling in AutoGen Using LiteLLM with togetherai

[YangQiuEric]

What happened?

Here is the terminal code: litellm --model together_ai/togethercomputer/CodeLlama-34b-Instruct --add_function_to_prompt ,I encountered a KeyError related to 'functions' and an AttributeError stating that a 'KeyError' object has no attribute 'status_code'. I just try to run the official agentchat_function_call.ipynb and it has this error.

Relevant log output

[2024-01-22 22:05:20 -0800] [57458] [INFO] Application startup complete.
[2024-01-22 22:05:23 -0800] [57455] [INFO] Handling signal: winch
[2024-01-22 22:05:23 -0800] [57455] [INFO] Handling signal: winch
Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion
    optional_params = get_optional_params(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params
    "tools", non_default_params.pop("functions")
KeyError: 'functions'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 275, in acompletion
    response = await loop.run_in_executor(None, func_with_context)  # type: ignore
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper
    result = original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'KeyError' object has no attribute 'status_code'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion
    response = await litellm.acompletion(**data)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async
    result = await original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'AttributeError' object has no attribute 'status_code'
127.0.0.1:58915 - "POST /chat/completions HTTP/1.1" 500
Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion
    optional_params = get_optional_params(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params
    "tools", non_default_params.pop("functions")
KeyError: 'functions'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 275, in acompletion
    response = await loop.run_in_executor(None, func_with_context)  # type: ignore
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper
    result = original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'KeyError' object has no attribute 'status_code'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion
    response = await litellm.acompletion(**data)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async
    result = await original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'AttributeError' object has no attribute 'status_code'
127.0.0.1:58915 - "POST /chat/completions HTTP/1.1" 500
Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 618, in completion
    optional_params = get_optional_params(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 3273, in get_optional_params
    "tools", non_default_params.pop("functions")
KeyError: 'functions'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 275, in acompletion
    response = await loop.run_in_executor(None, func_with_context)  # type: ignore
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2179, in wrapper
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2086, in wrapper
    result = original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 1808, in completion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'KeyError' object has no attribute 'status_code'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1596, in chat_completion
    response = await litellm.acompletion(**data)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2417, in wrapper_async
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 2309, in wrapper_async
    result = await original_function(*args, **kwargs)
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/main.py", line 287, in acompletion
    raise exception_type(
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6741, in exception_type
    raise e
  File "/Users/eric/opt/anaconda3/envs/gltanaka/lib/python3.10/site-packages/litellm/utils.py", line 6499, in exception_type
    status_code=original_exception.status_code,
AttributeError: 'AttributeError' object has no attribute 'status_code'

Twitter / LinkedIn details

No response