camel icon indicating copy to clipboard operation
camel copied to clipboard

[BUG] ChatAgent Fails to Process Function Calls in Messages Containing Single Quotes

Open echo-yiyiyi opened this issue 1 year ago • 0 comments

Required prerequisites

What version of camel are you using?

0.1.3

System information

3.10.9 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)] win32

Problem description

In line 537 of chat_agent.py, within the step_function_call function of the class ChatAgent, converting the message returned by the LLM into JSON format:

func_name = choice.message.function_call.name
func = self.func_dict[func_name]

args_str: str = choice.message.function_call.arguments
args = json.loads(args_str.replace("'", "\""))

I think this might be due to the fact that sometimes the string returned by the LLM is not standardized; for example, in such cases, we need to replace single quotes with double quotes in key-value pairs to load JSON. {'jobTitle_in_query': 'basketball coach', 'city_in_query': 'London'}.

For this situation, a simple args_str.replace("'", "\"") can effectively achieve the desired outcome.

However, when I introduce more functions, often the value of the string itself contains single quotes, such as

{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase 'A stitch in time saves nine' to Chinese"
  }
}

In this case, using args_str.replace("'", "\"") would transform the string to look something like this:

{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase "A stitch in time saves nine" to Chinese"
  }
}

"Due to the multiple double quotes in the last key, if we still run args = json.loads(args_str.replace("'", "\"")), this would lead to an error: json.decoder.JSONDecodeError: Expecting ',' delimiter: line 8 column 42 (char 211).

The occurrence of single quotes in the LLM's return values is not very rare. I often encounter this error causing interruptions in the dialogue between the AI user and assistant when running functions related to language and translation. Perhaps we need to add code to handle different scenarios involving single quotes differently.

Reproducible example code

The Python snippets:

import json

args_str = '''
{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase 'A stitch in time saves nine' to Chinese"
  }
}
'''

print(args_str)
print('After replace:')
print(args_str.replace("'", "\""))
args = json.loads(args_str)
print('After replace:')
args = json.loads(args_str.replace("'", "\""))

Traceback

No response

Expected behavior

No response

Additional context

No response

echo-yiyiyi avatar May 06 '24 04:05 echo-yiyiyi