microsoft
ModuleNotFoundError: No module named 'numpy._core'
🐛 Bug Description
Hello, thanks for your hardworking on this repo! But here's some issue when I try to run !rdagent fin_factor
-
When running the Qlib workflow, the program encountered a module loading error:
ModuleNotFoundError: No module named 'numpy._core'. -
During the data loading process, the program failed to load required files, resulting in the following errors:
LoadObjectError: No such file or directory: '.../report_normal_1day.pkl'FileNotFoundError: [Errno 2] No such file or directory: '.../ret.pkl'
-
The Docker image uses NumPy version 1.23.5. Replacing it with other versions (e.g., 1.26.4) leads to version conflicts with other dependencies.
-
After pulling the latest version of the repository, deleting the old image, and rebuilding the Docker image, the same issues persist.
P.S. the !rdagent fin_model works properly
To Reproduce
Steps to reproduce the behavior:
- git clone latest RD-Agent and rebuild docker image
- set .env
- !rdagent fin_factor
Screenshot
2025-03-27 10:42:16.242 | INFO | rdagent.log.time:timed:16 - coding took 355.21 sec Workflow Progress: 60%|▌| 3/5 [06:22<05:39, 169.58s/step, loop_index=0, step_in 2025-03-27 10:42:52.592 | INFO | rdagent.utils.env:prepare:202 - Building the image from dockerfile: /home/user/.local/lib/python3.10/site-packages/rdagent/scenarios/qlib/docker ⠋ Successfully tagged local_qlib:latest 2025-03-27 10:42:52.654 | INFO | rdagent.utils.env:prepare:220 - Finished building the image from dockerfile: /home/user/.local/lib/python3.10/site-packages/rdagent/scenarios/qlib/docker 2025-03-27 10:42:52.670 | INFO | rdagent.utils.env:prepare:384 - Data already exists. Download skipped.
[1:MainThread](2025-03-27 10:43:00,061) INFO - qlib.qrun - [cli.py:78] - Render the template with the context: {} [1:MainThread](2025-03-27 10:43:00,094) INFO - qlib.Initialization - [config.py:416] - default_conf: client. [1:MainThread](2025-03-27 10:43:00,102) INFO - qlib.Initialization - [init.py:74] - qlib successfully initialized based on client settings. [1:MainThread](2025-03-27 10:43:00,102) INFO - qlib.Initialization - [init.py:76] - data_path={'__DEFAULT_FREQ': PosixPath('/root/.qlib/qlib_data/cn_data')} [1:MainThread](2025-03-27 10:43:00,105) WARNING - qlib.workflow - [expm.py:230] - No valid experiment found. Create a new experiment with name workflow. [1:MainThread](2025-03-27 10:43:00,107) INFO - qlib.workflow - [exp.py:258] - Experiment 1 starts running ... [1:MainThread](2025-03-27 10:43:00,193) INFO - qlib.workflow - [recorder.py:374] - Fail to log the uncommitted code of $CWD(/workspace/qlib_workspace) when run git diff. [1:MainThread](2025-03-27 10:43:00,199) INFO - qlib.workflow - [recorder.py:374] - Fail to log the uncommitted code of $CWD(/workspace/qlib_workspace) when run git status.
[1:MainThread](2025-03-27 10:45:27,398) ERROR - qlib.workflow - [utils.py:41] - An exception has been raised [ModuleNotFoundError: No module named 'numpy._core'].
...
[1:MainThread](2025-03-27 10:45:31,939) INFO - qlib.Initialization - [config.py:416] - default_conf: client.
[1:MainThread](2025-03-27 10:45:31,988) INFO - qlib.Initialization - [init.py:74] - qlib successfully initialized based on client settings.
[1:MainThread](2025-03-27 10:45:31,989) INFO - qlib.Initialization - [init.py:76] - data_path={'__DEFAULT_FREQ': PosixPath('/root/.qlib/qlib_data/cn_data')}
...
[1:MainThread](2025-03-27 10:45:32,018) ERROR - qlib.workflow - [utils.py:41] - An exception has been raised [LoadObjectError: No such file or directory: '/workspace/qlib_workspace/mlruns/1/a68da2031d814499b69854f72caa16e5/artifacts/portfolio_analysis/report_normal_1day.pkl'].
...
Traceback (most recent call last):
File "/home/user/.local/bin/rdagent", line 8, in
Environment
Note: Users can run rdagent collect_info to get system information and paste it directly here.
Name of current operating system: Linux Processor architecture: x86_64 System, version, and hardware information: Linux-6.8.0-1021-azure-x86_64-with-glibc2.35 Version number of the system: #25~22.04.1-Ubuntu SMP Thu Jan 16 21:37:09 UTC 2025 Python version: 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0] Container ID: c145184a07b31715ec95e6b66d92e5deef7edbd607348e73023045ebcb1d4b6a Container Name: focused_ganguly Container Status: created Image ID used by the container: sha256:65e9861731a8547bdd672b3ac5feaedff25757b32a0d086f6d5bdad12cd19957 Image tag used by the container: ['local_qlib:latest'] Container port mapping: {} Container Label: {'com.nvidia.volumes.needed': 'nvidia_driver', 'org.opencontainers.image.ref.name': 'ubuntu', 'org.opencontainers.image.version': '22.04'} Startup Commands: nvidia-smi RD-Agent version: 0.3.0
This issue is due to the fact that when generating pkl files, the RD-Agent environment is used, and when using pkl files, you are using the docker image's internal environment.
The difference in the numpy version between the RD-Agent environment and the docker image's internal environment caused this problem.
This issue has been fixed in both qlib and RD-Agent, you can try renaming or deleting your local docker image called local_qlib:latest and pulling the latest RD-Agent code, try reinstalling it and running the rdagent fin_factor command.
