MetaGPT icon indicating copy to clipboard operation
MetaGPT copied to clipboard

RetryError[<Future at 0x7f806d336a60 state=finished raised ValidationError>]

Open luberion opened this issue 1 year ago • 3 comments

(mgpt) dope@dope:~/Documents/metagpt$ python startup.py "Write a cli snake game" 2023-07-12 21:00:39.873 | INFO | metagpt.config:init:43 - Config loading done. 2023-07-12 21:00:39.873 | INFO | metagpt.config:init:49 - Set OPENAI_API_BASE in case of network issues 2023-07-12 21:00:40.585 | INFO | metagpt.software_company:invest:39 - Investment: $3.0. 2023-07-12 21:00:40.586 | INFO | metagpt.roles.role:_act:155 - Alice(Product Manager): ready to WritePRD 2023-07-12 21:00:40.586 | WARNING | metagpt.actions.search_and_summarize:run:114 - Configure SERPAPI_API_KEY to unlock full feature

Original Requirements:

The boss wants a command-line snake game.

Product Goals:

  • Create a fun and engaging snake game that can be played in the command-line interface.
  • Provide a challenging gameplay experience for users.
  • Implement intuitive controls and smooth animations for a seamless gaming experience.

User Stories:

  • As a user, I want to be able to control the snake using arrow keys.
  • As a user, I want the snake to grow longer each time it eats food.
  • As a user, I want the game to end if the snake collides with the wall or itself.
  • As a user, I want to see my score displayed on the screen.
  • As a user, I want the game to have different levels of difficulty.

Competitive Analysis:

  • Python Snake Game: A basic snake game implemented in Python with simple graphics.
  • Snake Xenzia: A popular snake game with improved graphics and additional features.
  • Snake VS Block: A modern twist on the classic snake game with obstacles and power-ups.
  • Slither.io: An online multiplayer snake game with a competitive leaderboard.
  • Snake '97: A nostalgic recreation of the classic snake game from the Nokia 3310.
  • Snake Rivals: A multiplayer snake game with real-time battles and customizable snakes.
  • Snake Zone: A 3D snake game with immersive graphics and challenging levels.

Competitive Quadrant Chart:

quadrantChart
    title Reach and engagement of snake games
    x-axis Low Reach --> High Reach
    y-axis Low Engagement --> High Engagement
    quadrant-1 Basic Games
    quadrant-2 Popular Games
    quadrant-3 Innovative Games
    quadrant-4 Advanced Games
    "Python Snake Game": [0.2, 0.3]
    "Snake Xenzia": [0.4, 0.4]
    "Snake VS Block": [0.6, 0.6]
    "Slither.io": [0.8, 0.7]
    "Snake '97": [0.3, 0.2]
    "Snake Rivals": [0.7, 0.8]
    "Snake Zone": [0.9, 0.9]
    "Our Target Product": [0.5, 0.6]

Requirement Analysis:

The product should be a command-line snake game with intuitive controls, smooth animations, and challenging gameplay. It should have different levels of difficulty and display the user's score on the screen. The game should end if the snake collides with the wall or itself.

Requirement Pool:

[
    ("Implement arrow key controls for the snake", "P0"),
    ("Implement snake growth when it eats food", "P0"),
    ("Detect collision with the wall or snake's body", "P0"),
    ("Display the user's score on the screen", "P1"),
    ("Implement different levels of difficulty", "P1")
]

Anything UNCLEAR:

There are no unclear points. Warning: gpt-3.5-turbo may change over time. Returning num tokens assuming gpt-3.5-turbo-0301. 2023-07-12 21:00:55.796 | INFO | metagpt.provider.openai_api:update_cost:89 - Total running cost: $0.003 | Max budget: $3.000 | Current cost: $0.003, prompt_tokens=800, completion_tokens=625

Original Requirements:

The boss wants you to create a command-line snake game.

Product Goals:

  • Create a fun and engaging snake game.
  • Provide a smooth and responsive user experience.
  • Include customizable game settings for different difficulty levels.

User Stories:

  • As a user, I want to be able to control the snake using arrow keys.
  • As a user, I want the snake to grow longer when it eats food.
  • As a user, I want the game to end if the snake hits the wall or its own body.
  • As a user, I want to be able to pause and resume the game.
  • As a user, I want to see my current score and the highest score achieved.

Competitive Analysis:

  • Python Snake Game: A simple snake game implemented in Python with basic features.
  • Snake Xenzia: A classic snake game with improved graphics and sound effects.
  • Snake.io: A multiplayer snake game where players compete against each other.
  • Slither.io: A popular online multiplayer snake game with a large player base.
  • Snake '97: A nostalgic snake game that replicates the experience of playing on an old Nokia phone.
  • Snake Blast: A modern twist on the snake game with power-ups and special abilities.
  • Snake Rewind: A snake game with a rewind feature that allows players to undo their moves.

Competitive Quadrant Chart:

quadrantChart
    title Reach and engagement of snake games
    x-axis Low Reach --> High Reach
    y-axis Low Engagement --> High Engagement
    quadrant-1 We should expand
    quadrant-2 Need to promote
    quadrant-3 Re-evaluate
    quadrant-4 May be improved
    "Python Snake Game": [0.3, 0.6]
    "Snake Xenzia": [0.45, 0.23]
    "Snake.io": [0.57, 0.69]
    "Slither.io": [0.78, 0.34]
    "Snake '97": [0.40, 0.34]
    "Snake Blast": [0.35, 0.78]
    "Snake Rewind": [0.6, 0.7]
    "Our Target Product": [0.5, 0.6]

Requirement Analysis:

The product should be a command-line snake game with basic features such as controlling the snake, growing longer when eating food, and ending the game if the snake hits the wall or its own body. It should also include additional features like pausing and resuming the game, displaying the current score and highest score achieved, and customizable game settings for different difficulty levels.

Requirement Pool:

[
    ("End game when snake hits the wall or its own body", "P0"),
    ("Allow pausing and resuming the game", "P1"),
    ("Display current score and highest score achieved", "P1"),
    ("Customizable game settings for different difficulty levels", "P2")
]

Anything UNCLEAR:

There are no unclear points. Warning: gpt-3.5-turbo may change over time. Returning num tokens assuming gpt-3.5-turbo-0301. 2023-07-12 21:01:11.573 | INFO | metagpt.provider.openai_api:update_cost:89 - Total running cost: $0.006 | Max budget: $3.000 | Current cost: $0.003, prompt_tokens=800, completion_tokens=636 Traceback (most recent call last): File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in call result = await fn(*args, **kwargs) File "/home/dope/Documents/metagpt/metagpt/actions/action.py", line 60, in _aask_v1 instruct_content = output_class(**parsed_data) File "pydantic/main.py", line 331, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 3 validation errors for prd Product Goals value is not a valid list (type=type_error.list) User Stories value is not a valid list (type=type_error.list) Competitive Analysis value is not a valid list (type=type_error.list)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/dope/Documents/metagpt/startup.py", line 29, in fire.Fire(main) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/fire-0.5.0-py3.9.egg/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/fire-0.5.0-py3.9.egg/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/fire-0.5.0-py3.9.egg/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/home/dope/Documents/metagpt/startup.py", line 25, in main asyncio.run(startup(idea, investment, n_round)) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/home/dope/Documents/metagpt/startup.py", line 15, in startup await company.run(n_round=n_round) File "/home/dope/Documents/metagpt/metagpt/software_company.py", line 60, in run await self.environment.run() File "/home/dope/Documents/metagpt/metagpt/environment.py", line 64, in run await asyncio.gather(*futures) File "/home/dope/Documents/metagpt/metagpt/roles/role.py", line 229, in run rsp = await self._react() File "/home/dope/Documents/metagpt/metagpt/roles/role.py", line 200, in _react return await self._act() File "/home/dope/Documents/metagpt/metagpt/roles/role.py", line 156, in _act response = await self._rc.todo.run(self._rc.important_memory) File "/home/dope/Documents/metagpt/metagpt/actions/write_prd.py", line 138, in run prd = await self._aask_v1(prompt, "prd", OUTPUT_MAPPING) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped return await fn(*args, **kwargs) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/tenacity/_asyncio.py", line 47, in call do = self.iter(retry_state=retry_state) File "/home/dope/anaconda3/envs/mgpt/lib/python3.9/site-packages/tenacity/init.py", line 326, in iter raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x7f806d336a60 state=finished raised ValidationError>]

luberion avatar Jul 12 '23 13:07 luberion

The issue fundamentally arises from a discrepancy between the output of the LLM and our expectations.

We anticipated this:

Product Goals:

[
    "Create ...",
    "Provide ...",
    "Implement ..."
]

However, the actual output was:

Product Goals:

  1. Create ...
  2. Provide ...
  3. Implement ...

This led to a failure in data validation. The error here occurs because even after retries, we didn't get the expected output.

Solution: We will enhance our data parsing logic to increase the success rate.

qa6300525 avatar Jul 12 '23 23:07 qa6300525

@qa6300525 Some fields can actually be ignored, because the post-role has no dependence on them, even if the parsing error does not affect the effect

geekan avatar Jul 13 '23 01:07 geekan

https://github.com/geekan/MetaGPT/pull/41 almost fix this issue. pull & retry.

geekan avatar Jul 13 '23 06:07 geekan