geekan
LLM may generate comments inside [CONTENT][/CONTENT] , which causes parsing the JSON to fail.
Bug description
parse json from content inside [CONTENT][/CONTENT] failed at retry 1, exp: Expecting ',' delimiter: line 6 column 27 (char 135)
Bug solved method
Perhaps we could consider adding a constraint to the prompt, indicating not to generate comments inside [CONTENT][/CONTENT], or alternatively, we could trim the comments from the LLM's output.
Environment information
- LLM type and model name: OPENAI gpt-4-1106-preview
- System version: macos 12.5.1
- Python version: python 3.9
- packages version: metagpt commit 82a5eec72707dee44174eae8f8ff1490a6819ecd
- installation method: pip install from source
Screenshots or logs
[CONTENT]
{
"Required Python packages": [
"numpy==1.21.2",
"Kivy==2.0.0",
"pygame==2.0.1",
"sqlite3==2.6.0" # sqlite3 is included in Python's standard library, but versioning is for consistency
],
"Required Other language third-party packages": [
"No third-party dependencies required"
],
"Logic Analysis": [
[
"game.py",
"Contains Game class with core game logic, uses numpy for array manipulation, and interacts with UI and Storage classes"
],
[
"main.py",
"Contains main function, initializes the game by calling start_new_game() from Game class"
],
[
"ui.py",
"Contains UI class for user interface, uses Kivy for rendering, and interacts with Game class"
],
[
"storage.py",
"Contains Storage class for saving and loading high scores using SQLite"
]
],
"Task list": [
"storage.py",
"game.py",
"ui.py",
"main.py"
],
"Full API spec": "",
"Shared Knowledge": "'game.py' contains the Game class which is central to the game logic and is used by both 'ui.py' for rendering the game state and 'storage.py' for saving the high score.",
"Anything UNCLEAR": "The monetization strategy for the game is not specified. Will the game include ads, in-app purchases, or be a paid app? This will affect the design of the user interface and potentially the choice of libraries or frameworks."
}
[/CONTENT]
2024-01-10 14:58:53.419 | INFO | metagpt.utils.cost_manager:update_cost:48 - Total running cost: $0.199 | Max budget: $10.000 | Current cost: $0.021, prompt_tokens: 1021, completion_tokens: 352
2024-01-10 14:58:53.423 | WARNING | metagpt.utils.repair_llm_raw_output:run_and_passon:235 - parse json from content inside [CONTENT][/CONTENT] failed at retry 1, exp: Expecting ',' delimiter: line 6 column 27 (char 135)
2024-01-10 14:58:53.424 | INFO | metagpt.utils.repair_llm_raw_output:repair_invalid_json:204 - repair_invalid_json, raw error: Expecting ',' delimiter: line 6 column 27 (char 135)
2024-01-10 14:58:53.424 | ERROR | metagpt.utils.common:log_it:438 - Finished call to 'metagpt.actions.action_node.ActionNode._aask_v1' after 222.144(s), this was the 6th time calling it. exp: RetryError[<Future at 0x7fb828c8d340 state=finished raised JSONDecodeError>]
2024-01-10 14:58:53.424 | WARNING | metagpt.utils.common:wrapper:510 - There is a exception in role's execution, in order to resume, we delete the newest role communication message in the role's memory.
2024-01-10 14:58:53.430 | ERROR | metagpt.utils.common:wrapper:492 - Exception occurs, start to serialize the project, exp:
It seem because of # sqlite3 is included in Python's standard library, but versioning is for consistency. It can not parse the comments.
You can set
REPAIR_LLM_OUTPUT: truein key.yaml to fix the problem.
It seems that there are no test cases to confirm that the current repair utils can fix the comments in json. Should I add some test cases to confirm this and then push a merge request? @voidking
