litellm
litellm copied to clipboard
Published
20 hours ago •
BerriAI
BerriAI
openai.BadRequestError: Error code: 400 when using mistral large model
Description
I have a multi agent workflow with 1. An OpenAI model assistant and 2. A mistral model critic (workflow implemented using autogen). Flow goes as follows:
- user input is passed to the assistant
- the response is passed to the critic
- if critic has improvement suggestions, pass response to assistant again
When this works
- when the assisant is any openai model and the critic is mistral medium model
- when the assistant is mistral large model and the critic is mistral medium model
When this does not work
- when the assisant is any openai model and the critic is mistral large model
- when the assistant is mistral large model and the critic is mistral large model
I feel like im missing something obivous here but I can’t figure out what. If needed, I will provide a repo with steps for a reproductible example.
My litellm config:
model_list:
- model_name: ollama/codellama
litellm_params:
model: "ollama/codellama"
- model_name: openai/mistral-medium
litellm_params:
model: openai/mistral-medium-latest
api_key: os.environ/MISTRAL_API_KEY
api_base: https://api.mistral.ai/v1
- model_name: mistral/mistral-large
litellm_params:
model: mistral/mistral-large-latest
api_key: os.environ/MISTRAL_API_KEY
api_base: https://api.mistral.ai/v1
litellm_settings:
drop_params: True
set_verbose: True
Litellm version: 1.30.3
Relevant log output
LiteLLM completion() model= mistral-large-latest; provider = mistral
LiteLLM: Params passed to completion() {'functions': None, 'function_call': None, 'temperature': 0.1, 'top_p': None, 'stream': False, 'max_tokens': None, 'presence_penalty': None, 'frequency_penalty': None, 'logit_bias': None, 'user': None, 'response_format': None, 'seed': None, 'tools': None, 'tool_choice': None, 'max_retries': 0, 'custom_llm_provider': 'mistral', 'model': 'mistral-large-latest', 'n': None, 'stop': None}
LiteLLM: Non-Default params passed to completion() {'temperature': 0.1, 'stream': False, 'max_retries': 0}
self.optional_params: {'temperature': 0.1, 'stream': False, 'max_retries': 0, 'extra_body': {}}
PRE-API-CALL ADDITIONAL ARGS: {'headers': {'Authorization': 'Bearer ***********************'}, 'api_base': ParseResult(scheme='https', userinfo='', host='api.mistral.ai', port=None, path='/v1/', query=None, fragment=None), 'acompletion': True, 'complete_input_dict': {'model': 'mistral-large-latest', 'messages': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'temperature': 0.1, 'stream': False, 'extra_body': {}}}
POST Request Sent from LiteLLM:
curl -X POST \
https://api.mistral.ai/v1/ \
-H 'Authorization: Bearer ********************' \
-d '{'model': 'mistral-large-latest', 'messages': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'temperature': 0.1, 'stream': False, 'extra_body': {}}'
RAW RESPONSE:
{"id":"7fca250f757e4b26a7945f36a84831d8","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.","role":"assistant","function_call":null,"tool_calls":null},"logprobs":null}],"created":1709973619,"model":"mistral-large-latest","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":167,"prompt_tokens":471,"total_tokens":638}}
Logging Details Post-API Call: logger_fn - None | callable(logger_fn) - False
Logging Details Post-API Call: LiteLLM Params: {'model': 'mistral-large-latest', 'messages': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'optional_params': {'temperature': 0.1, 'stream': False, 'max_retries': 0, 'extra_body': {}}, 'litellm_params': {'acompletion': True, 'api_key': '***********************', 'force_timeout': 600, 'logger_fn': None, 'verbose': False, 'custom_llm_provider': 'mistral', 'api_base': 'https://api.mistral.ai/v1', 'litellm_call_id': '554191a4-06a8-4458-a93c-6d5d50d42c57', 'model_alias_map': {}, 'completion_call_id': None, 'metadata': {'user_api_key': 'sk-xxxxx', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '2241'}, 'model_group': 'mistral/mistral-large', 'deployment': 'mistral/mistral-large-latest', 'caching_groups': None}, 'model_info': {'id': '84548ab5-db05-4b37-a37a-f405f19a7e20'}, 'proxy_server_request': {'url': 'http://localhost:30000/chat/completions', 'method': 'POST', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '2241'}, 'body': {'messages': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1}}, 'preset_cache_key': None, 'stream_response': {}}, 'start_time': datetime.datetime(2024, 3, 9, 10, 40, 19, 144425), 'stream': False, 'user': None, 'call_type': 'acompletion', 'temperature': 0.1, 'max_retries': 0, 'extra_body': {}, 'input': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'api_key': '***********************', 'additional_args': {'complete_input_dict': {'model': 'mistral-large-latest', 'messages': [{'content': "You are a helpful AI assistant. Solve tasks using your coding and language skills. In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself. 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill. When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user. If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user. If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try. When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.", 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'user'}], 'temperature': 0.1, 'stream': False, 'extra_body': {}}}, 'log_event_type': 'post_api_call', 'original_response': '{"id":"7fca250f757e4b26a7945f36a84831d8","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here\'s the code:\\n\\n```python\\n# Import the math module\\nimport math\\n\\n# Print the value of pi\\nprint(math.pi)\\n```\\n\\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it\'s important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.","role":"assistant","function_call":null,"tool_calls":null},"logprobs":null}],"created":1709973619,"model":"mistral-large-latest","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":167,"prompt_tokens":471,"total_tokens":638}}'}
Async Wrapper: Completed Call, calling async_success_handler: <bound method Logging.async_success_handler of <litellm.utils.Logging object at 0x1102de800>>
Logging Details LiteLLM-Success Call
LiteLLM.Router: Async Response: ModelResponse(id='7fca250f757e4b26a7945f36a84831d8', choices=[Choices(finish_reason='stop', index=0, message=Message(content="The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", role='assistant'))], created=1709973619, model='mistral-large-latest', object='chat.completion', system_fingerprint=None, usage=Usage(completion_tokens=167, prompt_tokens=471, total_tokens=638))
success callbacks: [<bound method Router.deployment_callback of <litellm.router.Router object at 0x106bb1f00>>, <litellm.proxy.hooks.parallel_request_limiter.MaxParallelRequestsHandler object at 0x106a949d0>, <litellm.proxy.hooks.max_budget_limiter.MaxBudgetLimiter object at 0x106a94a00>]
final response: ModelResponse(id='7fca250f757e4b26a7945f36a84831d8', choices=[Choices(finish_reason='stop', index=0, message=Message(content="The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", role='assistant'))], created=1709973619, model='mistral-large-latest', object='chat.completion', system_fingerprint=None, usage=Usage(completion_tokens=167, prompt_tokens=471, total_tokens=638))
success callbacks: Running Custom Callback Function
INFO: 127.0.0.1:65497 - "POST /chat/completions HTTP/1.1" 200 OK
get cache: cache key: mistral/mistral-large-latest:tpm:10-40
get cache: cache result: None
set cache: key: mistral/mistral-large-latest:tpm:10-40; value: 638
get cache: cache key: mistral/mistral-large-latest:rpm:10-40
get cache: cache result: None
set cache: key: mistral/mistral-large-latest:rpm:10-40; value: 1
Custom Logger - final response object: ModelResponse(id='7fca250f757e4b26a7945f36a84831d8', choices=[Choices(finish_reason='stop', index=0, message=Message(content="The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", role='assistant'))], created=1709973619, model='mistral-large-latest', object='chat.completion', system_fingerprint=None, usage=Usage(completion_tokens=167, prompt_tokens=471, total_tokens=638))
Async success callbacks: [<litellm.proxy.hooks.parallel_request_limiter.MaxParallelRequestsHandler object at 0x106a949d0>, <litellm.proxy.hooks.max_budget_limiter.MaxBudgetLimiter object at 0x106a94a00>]
Async success callbacks: CustomLogger
INSIDE ASYNC SUCCESS LOGGING
'complete_streaming_response' is in kwargs: False
'MaxParallelRequestsHandler' object has no attribute 'user_api_key_cache'
Async success callbacks: CustomLogger
receiving data: {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1, 'proxy_server_request': {'url': 'http://localhost:30000/chat/completions', 'method': 'POST', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'body': {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1}}}
Inside Max Parallel Request Pre-Call Hook
Inside Max Budget Limiter Pre-Call Hook
LiteLLM Proxy: final data being sent to completion call: {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1, 'proxy_server_request': {'url': 'http://localhost:30000/chat/completions', 'method': 'POST', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'body': {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1}}, 'metadata': {'user_api_key': 'sk-xxxxx', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}}, 'request_timeout': 600}
LiteLLM.Router: Inside async function with retries: args - (); kwargs - {'stream': False, 'temperature': 0.1, 'proxy_server_request': {'url': 'http://localhost:30000/chat/completions', 'method': 'POST', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'body': {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1}}, 'metadata': {'user_api_key': 'sk-xxxxx', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'model_group': 'mistral/mistral-large'}, 'request_timeout': 600, 'model': 'mistral/mistral-large', 'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'original_function': <bound method Router._acompletion of <litellm.router.Router object at 0x106bb1f00>>, 'num_retries': 3}
LiteLLM.Router: async function w/ retries: original_function - <bound method Router._acompletion of <litellm.router.Router object at 0x106bb1f00>>
LiteLLM.Router: Inside _acompletion()- model: mistral/mistral-large; kwargs: {'stream': False, 'temperature': 0.1, 'proxy_server_request': {'url': 'http://localhost:30000/chat/completions', 'method': 'POST', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'body': {'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'model': 'mistral/mistral-large', 'stream': False, 'temperature': 0.1}}, 'metadata': {'user_api_key': 'sk-xxxxx', 'headers': {'host': 'localhost:30000', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/Python 1.3.3', 'x-stainless-lang': 'python', 'x-stainless-package-version': '1.3.3', 'x-stainless-os': 'MacOS', 'x-stainless-arch': 'arm64', 'x-stainless-runtime': 'CPython', 'x-stainless-runtime-version': '3.10.12', 'authorization': 'Bearer sk-xxxxx', 'content-length': '927'}, 'model_group': 'mistral/mistral-large'}, 'request_timeout': 600}
LiteLLM.Router: initial list of deployments: [{'model_name': 'mistral/mistral-large', 'litellm_params': {'model': 'mistral/mistral-large-latest', 'api_key': '***********************', 'api_base': 'https://api.mistral.ai/v1'}, 'model_info': {'id': '84548ab5-db05-4b37-a37a-f405f19a7e20'}, 'async_client': <openai.AsyncOpenAI object at 0x106c44160>, 'client': <openai.OpenAI object at 0x106c5a470>, 'stream_async_client': <openai.AsyncOpenAI object at 0x106c84700>, 'stream_client': <openai.OpenAI object at 0x106c969b0>}]
get cache: cache key: 10-40:cooldown_models
get cache: cache result: None
LiteLLM.Router: retrieve cooldown models: []
LiteLLM.Router: cooldown deployments: []
LiteLLM.Router: healthy deployments: length 1 [{'model_name': 'mistral/mistral-large', 'litellm_params': {'model': 'mistral/mistral-large-latest', 'api_key': '***********************', 'api_base': 'https://api.mistral.ai/v1'}, 'model_info': {'id': '84548ab5-db05-4b37-a37a-f405f19a7e20'}, 'async_client': <openai.AsyncOpenAI object at 0x106c44160>, 'client': <openai.OpenAI object at 0x106c5a470>, 'stream_async_client': <openai.AsyncOpenAI object at 0x106c84700>, 'stream_client': <openai.OpenAI object at 0x106c969b0>}]
Initialized litellm callbacks, Async Success Callbacks: [<litellm.proxy.hooks.parallel_request_limiter.MaxParallelRequestsHandler object at 0x106a949d0>, <litellm.proxy.hooks.max_budget_limiter.MaxBudgetLimiter object at 0x106a94a00>]
litellm.cache: None
kwargs[caching]: False; litellm.cache: None
kwargs[caching]: False; litellm.cache: None
LiteLLM completion() model= mistral-large-latest; provider = mistral
LiteLLM: Params passed to completion() {'functions': None, 'function_call': None, 'temperature': 0.1, 'top_p': None, 'stream': False, 'max_tokens': None, 'presence_penalty': None, 'frequency_penalty': None, 'logit_bias': None, 'user': None, 'response_format': None, 'seed': None, 'tools': None, 'tool_choice': None, 'max_retries': 0, 'custom_llm_provider': 'mistral', 'model': 'mistral-large-latest', 'n': None, 'stop': None}
LiteLLM: Non-Default params passed to completion() {'temperature': 0.1, 'stream': False, 'max_retries': 0}
self.optional_params: {'temperature': 0.1, 'stream': False, 'max_retries': 0, 'extra_body': {}}
PRE-API-CALL ADDITIONAL ARGS: {'headers': {'Authorization': 'Bearer ***********************'}, 'api_base': ParseResult(scheme='https', userinfo='', host='api.mistral.ai', port=None, path='/v1/', query=None, fragment=None), 'acompletion': True, 'complete_input_dict': {'model': 'mistral-large-latest', 'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'temperature': 0.1, 'stream': False, 'extra_body': {}}}
POST Request Sent from LiteLLM:
curl -X POST \
https://api.mistral.ai/v1/ \
-H 'Authorization: Bearer ********************' \
-d '{'model': 'mistral-large-latest', 'messages': [{'content': 'an advanced critic ai', 'role': 'system'}, {'content': 'what is the value of pi ?', 'role': 'assistant'}, {'content': "The value of pi is a mathematical constant that is approximately equal to 3.14159. In Python, you can use the `math` module to get a more precise value of pi. Here's the code:\n\n```python\n# Import the math module\nimport math\n\n# Print the value of pi\nprint(math.pi)\n```\n\nWhen you execute this code, it will print the value of pi to the console. The value is precise to many decimal places, but it's important to note that pi is an irrational number, which means its exact value cannot be expressed as a finite decimal or fraction. The value that Python (and any other programming language or calculator) gives you is an approximation, accurate to a certain number of decimal places.", 'role': 'user'}], 'temperature': 0.1, 'stream': False, 'extra_body': {}}'
Logging Details: logger_fn - None | callable(logger_fn) - False
Logging Details LiteLLM-Failure Call
LiteLLM.Router: Attempting to add 84548ab5-db05-4b37-a37a-f405f19a7e20 to cooldown list. updated_fails: 1; self.allowed_fails: 0
get cache: cache key: 10-40:cooldown_models
get cache: cache result: None
LiteLLM.Router: adding 84548ab5-db05-4b37-a37a-f405f19a7e20 to cooldown models
set cache: key: 10-40:cooldown_models; value: ['84548ab5-db05-4b37-a37a-f405f19a7e20']
LiteLLM.Router:
EXCEPTION FOR DEPLOYMENTS
LiteLLM.Router: {'mistral/mistral-large-latest': ["<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}", "<class 'litellm.exceptions.BadRequestError'>Status: 400Message: NoneOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}Full exceptionOpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}"]}
LiteLLM.Router: Model mistral/mistral-large-latest had 7 exception
Custom Logger - final response object: None
LiteLLM.Router: An exception occurs: OpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
TracebackTraceback (most recent call last):
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/main.py", line 194, in acompletion
response = await init_response
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/llms/openai.py", line 288, in acompletion
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/llms/openai.py", line 278, in acompletion
response = await openai_aclient.chat.completions.create(**data)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1199, in create
return await self._post(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1474, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1275, in request
return await self._request(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1318, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 474, in async_function_with_fallbacks
response = await self.async_function_with_retries(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 555, in async_function_with_retries
raise original_exception
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 535, in async_function_with_retries
response = await original_function(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 262, in _acompletion
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 256, in _acompletion
response = await litellm.acompletion(**{**data, "messages": messages, "caching": self.cache_responses, "client": model_client, **kwargs})
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 1858, in wrapper_async
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 1789, in wrapper_async
result = await original_function(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/main.py", line 204, in acompletion
raise exception_type(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 5374, in exception_type
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 4438, in exception_type
raise BadRequestError(
litellm.exceptions.BadRequestError: OpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
LiteLLM.Router: Trying to fallback b/w models
An error occurred: OpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
Debug this by setting `--debug`, e.g. `litellm --model gpt-3.5-turbo --debug`
Results from router
Router stats
Total Calls made
mistral/mistral-large-latest: 18
openai/mistral-medium-latest: 4
Success Calls made
mistral/mistral-large-latest: 11
openai/mistral-medium-latest: 4
Fail Calls made
mistral/mistral-large-latest: 7
Traceback (most recent call last):
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/main.py", line 194, in acompletion
response = await init_response
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/llms/openai.py", line 288, in acompletion
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/llms/openai.py", line 278, in acompletion
response = await openai_aclient.chat.completions.create(**data)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1199, in create
return await self._post(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1474, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1275, in request
return await self._request(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/openai/_base_client.py", line 1318, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/proxy/proxy_server.py", line 1013, in chat_completion
response = await llm_router.acompletion(**data)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 234, in acompletion
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 230, in acompletion
response = await self.async_function_with_fallbacks(**kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 524, in async_function_with_fallbacks
raise original_exception
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 474, in async_function_with_fallbacks
response = await self.async_function_with_retries(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 555, in async_function_with_retries
raise original_exception
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 535, in async_function_with_retries
response = await original_function(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 262, in _acompletion
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/router.py", line 256, in _acompletion
response = await litellm.acompletion(**{**data, "messages": messages, "caching": self.cache_responses, "client": model_client, **kwargs})
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 1858, in wrapper_async
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 1789, in wrapper_async
result = await original_function(*args, **kwargs)
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/main.py", line 204, in acompletion
raise exception_type(
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 5374, in exception_type
raise e
File "/Users/dragoscampean/.pyenv/versions/3.10.12/envs/autogen/lib/python3.10/site-packages/litellm/utils.py", line 4438, in exception_type
raise BadRequestError(
litellm.exceptions.BadRequestError: OpenAIException - Error code: 400 - {'object': 'error', 'message': 'Unexpected role RoleEnum.assistant', 'type': 'invalid_request_error', 'param': None, 'code': None}
INFO: 127.0.0.1:65503 - "POST /chat/completions HTTP/1.1" 400 Bad Request
Twitter / LinkedIn details
No response
