RD-Agent
RD-Agent copied to clipboard
microsoft
How to fix the UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 64: invalid start byte?
🐛 Bug Description
To Reproduce
Just run rdagent fin_factor.
Then I get
2025-05-28 15:43:37.286 | INFO | rdagent.utils.env:prepare:522 - Finished building the image from dockerfile: /home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/scenarios/qlib/docker
2025-05-28 15:43:37.419 | INFO | rdagent.utils.env:prepare:700 - We are downloading!
Traceback (most recent call last):
File "/home/demo/Desktop/RDAgent/.venv/bin/rdagent", Line 10, in <module>
sys.exit(app())
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/app/cli.py", line 48, in app
fire.Fire(
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/app/qlib_rd_loop/factor.py", line 40, in main
model_loop = FactorRdLoop(FACTOR_PROP_SETTING)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/components/workflow/rd_loop.py", line 20, in __init__
scen: Scenario = import_class(FACTOR_PROP_SETTING.scen)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/scenarios/qlib/experiment/factor_experiment.py", line 28, in __init__
self.source_data = deepcopy(get_data_folder_intro())
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/scenarios/qlib/experiment/utils.py", line 150, in get_data_folder_intro
generate_data_folder_from_qlib()
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/scenarios/qlib/experiment/utils.py", line 18, in generate_data_folder_from_qlib
qtle.prepare()
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/utils/env.py", line 702, in prepare
self.run(entry=cmd)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/utils/env.py", line 116, in run
stdout, _ = self.run_ret_code(entry=entry, local_path=local_path, env=env, **kwargs)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/utils/env.py", line 189, in run_ret_code
stdout, return_code = self.cached_run(run_add_timeout, local_path, env, running_extra_volume)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/utils/env.py", line 226, in cached_run
[
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/utils/env.py", line 227, in <listcomp>
[str(path.relative_to(Path(local_path))), path.read_text()]
File "/usr/lib/python3.10/pathlib.py", line 1135, in read_text
return f.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 64: invalid start byte
It seemed that something wrong with .venv/lib/python3.10/site-packages/rdagent/utils/env.py:
key = md5_hash(
json.dumps(
[
[str(path.relative_to(Path(local_path))), path.read_text()]##
for path in sorted(Path(local_path).rglob("*.py"))
]
)
+ json.dumps({"entry": entry, "running_extra_volume": dict(running_extra_volume)})
+ json.dumps({"extra_volumes": self.conf.extra_volumes})
+ json.dumps(data_key)
)
Expected Behavior
run successfully.
Screenshot
Environment
Note: Users can run rdagent collect_info to get system information and paste it directly here.
2025-05-28 17:57:19.814 | INFO | rdagent.app.utils.info:sys_info:22 - Name of current operating system: Linux
2025-05-28 17:57:19.817 | INFO | rdagent.app.utils.info:sys_info:22 - Processor architecture: x86_64
2025-05-28 17:57:19.821 | INFO | rdagent.app.utils.info:sys_info:22 - System, version, and hardware information: Linux-6.8.0-59-generic-x86_64-with-glibc2.35
2025-05-28 17:57:19.824 | INFO | rdagent.app.utils.info:sys_info:22 - Version number of the system: #61~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 17:03:15 UTC 2
2025-05-28 17:57:19.828 | INFO | rdagent.app.utils.info:python_info:29 - Python version: 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]
2025-05-28 17:57:19.884 | INFO | rdagent.app.utils.info:docker_info:39 - Container ID: cdeffe654d84d7dd18b98007d079936c7cb4ca0f5748cc268fefae8d72b3d27c
2025-05-28 17:57:19.893 | INFO | rdagent.app.utils.info:docker_info:40 - Container Name: kind_galileo
2025-05-28 17:57:19.904 | INFO | rdagent.app.utils.info:docker_info:41 - Container Status: exited
2025-05-28 17:57:19.914 | INFO | rdagent.app.utils.info:docker_info:42 - Image ID used by the container: sha256:0d18770744437835a0da4f993f1e9fb681db9334556a70d7af089792b7f24d81
2025-05-28 17:57:19.924 | INFO | rdagent.app.utils.info:docker_info:43 - Image tag used by the container: []
2025-05-28 17:57:19.934 | INFO | rdagent.app.utils.info:docker_info:44 - Container port mapping: {}
2025-05-28 17:57:19.941 | INFO | rdagent.app.utils.info:docker_info:45 - Container Label: {'com.nvidia.volumes.needed': 'nvidia_driver', 'org.opencontainers.image.ref.name': 'ubuntu', 'org.opencontainers.image.version': '22.04'}
2025-05-28 17:57:19.953 | INFO | rdagent.app.utils.info:docker_info:46 - Startup Commands: /bin/sh -c pip install tables
2025-05-28 17:57:19.961 | INFO | rdagent.app.utils.info:rdagent_info:54 - RD-Agent version: 0.4.0
Traceback (most recent call last):
File "/home/demo/Desktop/RDAgent/.venv/bin/rdagent", line 10, in <module>
sys.exit(app())
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/app/cli.py", line 48, in app
fire.Fire(
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/app/utils/info.py", line 85, in collect_info
rdagent_info()
File "/home/demo/Desktop/RDAgent/.venv/lib/python3.10/site-packages/rdagent/app/utils/info.py", line 74, in rdagent_info
version = importlib.metadata.version(package)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 996, in version
return distribution(distribution_name).version
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 969, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for mlflow
There is strange dependency with mlflow and rdagent....
Hi there, I also met this problem. I just added RUN pip install --force-reinstall ${YOUR_PACKAGE_NAME}into the dockerfile, but the info still showed that the package was not found. Have you fixed it? thanks.
