ValueError: could not convert string to float: 'SH600000' when i use dump_bin.py

[nb7123]

🐛 Bug Description

To Reproduce

Steps to reproduce the behavior:

1. python scripts/data_collector/baostock_5min/collector.py download_data --source_dir ~/.qlib/stock_data/source/hs300_5min_original --start 2022-01-01 --end 2022-01-30 --interval 5min --region HS300

2. python scripts/dump_bin.py dump_all --csv_path ~/.qlib/stock_data/source/hs300_5min_original --qlib_dir ~/.qlib/qlib_data/samples

Error: """ Traceback (most recent call last): File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/process.py", line 198, in _process_chunk return [fn(*args) for args in chunk] File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/process.py", line 198, in return [fn(*args) for args in chunk] File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 264, in _dump_bin self._data_to_bin(df, calendar_list, features_dir) File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 238, in _data_to_bin np.hstack([date_index, _df[field]]).astype("<f").tofile(str(bin_path.resolve())) ValueError: could not convert string to float: 'SH600000' """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 508, in fire.Fire({"dump_all": DumpDataAll, "dump_fix": DumpDataFix, "dump_update": DumpDataUpdate}) File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/site-packages/fire/core.py", line 143, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/site-packages/fire/core.py", line 568, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 271, in call self.dump() File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 322, in dump self._dump_features() File "/Users/didi/Code/Github/qlib/scripts/dump_bin.py", line 313, in _dump_features for _ in executor.map(_dump_func, self.csv_files): File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/process.py", line 484, in _chain_from_iterable_of_lists for element in iterable: File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator yield fs.pop().result() File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/_base.py", line 444, in result return self.__get_result() File "/Users/didi/miniconda3/envs/qlib/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception ValueError: could not convert string to float: 'SH600000'

Screenshot

Environment

Darwin arm64 macOS-14.2-arm64-arm-64bit Darwin Kernel Version 23.2.0: Wed Nov 15 21:54:55 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8122

Python version: 3.8.19 (default, Mar 20 2024, 15:27:52) [Clang 14.0.6 ]

Qlib version: 0.9.5 numpy==1.23.5 pandas==2.0.3 scipy==1.10.1 requests==2.32.3 sacred==0.8.6 python-socketio==5.11.4 redis==5.0.8 python-redis-lock==4.0.0 schedule==1.2.2 cvxpy==1.5.2 hyperopt==0.1.2 fire==0.6.0 statsmodels==0.14.1 xlrd==2.0.1 plotly==5.24.1 matplotlib==3.7.5 tables==3.7.0 pyyaml==6.0.2 mlflow==1.14.1 tqdm==4.66.5 loguru==0.7.2 lightgbm==4.5.0 tornado==6.4.1 joblib==1.4.2 fire==0.6.0 ruamel.yaml==0.17.36

Additional Notes

[ghyzx]

same issue

[zqingr]

same issue

[SunsetWolf]

According to the information you provided, there is a problem with your operation steps, after download_data, you need to do the normalization_data first, and then you can use dump_bin to convert the data to bin format. The documentation is here. Of course, before you do the normalization, you need to prepare a daily frequency data, the time of the daily frequency data should include the time of the normalized data.