RD-Agent icon indicating copy to clipboard operation
RD-Agent copied to clipboard

WSL bug RuntimeError: Failed to create chat completion after 10 retries.

Open Tylor-Tian opened this issue 5 months ago • 6 comments

🐛 Bug Description

When I run rdagent fin_quant in wsl there is a bug

To Reproduce

Steps to reproduce the behavior:

2025-07-06 14:12:43.496 | INFO | rdagent.oai.backend.litellm:_create_chat_completion_inner_function:176 - Current Cost: $0.0103800000; Accumulated Cost: $0.1038400000; finish_reason='stop' 2025-07-06 14:12:44.535 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:397 - 1 validation error for dict[str,str] hypothesis Input should be a valid string [type=string_type, input_value=[{'name': '10_Day_Momentu...en', 'window_size': 5}}], input_type=list] For further information visit https://errors.pydantic.dev/2.11/v/string_type 2025-07-06 14:12:44.536 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:398 - Retrying 10th time... Workflow Progress: 25%|████ | 1/4 [00:00<00:00, 1775.74step/s, loop_index=0, step_index=1, step_name=coding] Traceback (most recent call last): File "/root/miniconda3/envs/rdagent/bin/rdagent", line 8, in sys.exit(app()) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/app/cli.py", line 55, in app fire.Fire( File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/app/qlib_rd_loop/quant.py", line 133, in main asyncio.run(quant_loop.run(step_n=step_n, loop_n=loop_n, all_duration=all_duration)) File "/root/miniconda3/envs/rdagent/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/root/miniconda3/envs/rdagent/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/utils/workflow/loop.py", line 336, in run await asyncio.gather( File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/utils/workflow/loop.py", line 285, in kickoff_loop await self._run_step(li) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/utils/workflow/loop.py", line 221, in _run_step result = await func(self.loop_prev_out[li]) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/app/qlib_rd_loop/quant.py", line 71, in direct_exp_gen hypo = self._propose() File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/workflow/rd_loop.py", line 50, in _propose hypothesis = self.hypothesis_gen.gen(self.trace) File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/proposal/init.py", line 53, in gen resp = APIBackend().build_messages_and_create_chat_completion( File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/oai/backend/base.py", line 287, in build_messages_and_create_chat_completion resp = self._try_create_chat_completion_or_embedding( # type: ignore[misc] File "/root/miniconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/oai/backend/base.py", line 400, in _try_create_chat_completion_or_embedding raise RuntimeError(error_message) RuntimeError: Failed to create chat completion after 10 retries.

Expected Behavior

Screenshot

Image

Environment

WSL Ubuntu

Additional Notes

Tylor-Tian avatar Jul 06 '25 06:07 Tylor-Tian

what's your model? Deepseek?

Hoder-zyf avatar Jul 06 '25 06:07 Hoder-zyf

I use openai

Tylor-Tian avatar Jul 06 '25 08:07 Tylor-Tian

I change max_retry = 100 and it is still does not work

Image

Tylor-Tian avatar Jul 06 '25 08:07 Tylor-Tian

Maybe you can switch branch to qlib_ds and try again. It seems that hypothesis doesn't return a string and I have enhanced this string section in that branch though because the observation of deepseek failure. BUT I think you can switch to that branch and have a try.

Hoder-zyf avatar Jul 06 '25 08:07 Hoder-zyf

Now you can pull our latest main branch and have a try. If you have any questions, just comment here.

Hoder-zyf avatar Jul 08 '25 13:07 Hoder-zyf

Version 0.7.1-dev8 deepseek 2025-07-09 08:12:06.888 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:510 - Failed to parse JSON after all attempts: line 1 column 1 (char 0) 2025-07-09 08:12:06.890 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:511 - Retrying 5th time... Implementing: 0%| | 0/10 [03:20<?, ?it/s] Workflow Progress: 50%|███████████████████▌ | 2/4 [03:20<03:20, 100.05s/step, loop_index=0, step_index=2, step_name=running] Workflow Progress: 25%|█████████▌ | 1/4 [00:00<00:00, 21399.51step/s, loop_index=1, step_index=1, step_name=coding]Traceback (most recent call last): File "/mnt/disk01/miniconda3/envs/rdagent/bin/rdagent", line 8, in sys.exit(app()) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/app/cli.py", line 55, in app fire.Fire( File "/mnt/disk01/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/mnt/disk01/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/mnt/disk01/miniconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/app/qlib_rd_loop/quant.py", line 133, in main asyncio.run(quant_loop.run(step_n=step_n, loop_n=loop_n, all_duration=all_duration)) File "/mnt/disk01/miniconda3/envs/rdagent/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/mnt/disk01/miniconda3/envs/rdagent/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/utils/workflow/loop.py", line 336, in run await asyncio.gather( File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/utils/workflow/loop.py", line 304, in execute_loop await self._run_step(li, force_subproc=RD_AGENT_SETTINGS.is_force_subproc()) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/utils/workflow/loop.py", line 224, in _run_step result = func(self.loop_prev_out[li]) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/app/qlib_rd_loop/quant.py", line 83, in coding exp = self.factor_coder.develop(prev_out["direct_exp_gen"]["exp_gen"]) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/components/coder/factor_coder/init.py", line 27, in develop exp = super().develop(exp) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/components/coder/CoSTEER/init.py", line 102, in develop for evo_exp in self.evolve_agent.multistep_evolve(evo_exp, self.evaluator): File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/core/evolving_agent.py", line 95, in multistep_evolve eva if isinstance(eva, Feedback) else eva.evaluate(evo, queried_knowledge=queried_knowledge) File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/components/coder/CoSTEER/evaluators.py", line 232, in evaluate multi_implementation_feedback = multiprocessing_wrapper( File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/core/utils.py", line 146, in multiprocessing_wrapper return [f(*args) for f, args in func_calls] File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/core/utils.py", line 146, in return [f(*args) for f, args in func_calls] File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/components/coder/factor_coder/evaluators.py", line 114, in evaluate ) = self.final_decision_evaluator.evaluate( File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/components/coder/factor_coder/eva_utils.py", line 527, in evaluate api.build_messages_and_create_chat_completion( File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/oai/backend/base.py", line 400, in build_messages_and_create_chat_completion resp = self._try_create_chat_completion_or_embedding( # type: ignore[misc] File "/mnt/disk01/workspaces/worksummary/RD-Agent/rdagent/oai/backend/base.py", line 513, in _try_create_chat_completion_or_embedding raise RuntimeError(error_message) RuntimeError: Failed to create chat completion after 5 retries. Workflow Progress: 25%|██████████▎ | 1/4 [00:00<00:01, 1.99step/s, loop_index=1, step_index=1, step_name=coding]

jyf2100 avatar Jul 09 '25 00:07 jyf2100