qlib
qlib copied to clipboard
microsoft
get some error in example/rl_order_execution
❓ Questions and Help
We sincerely suggest you to carefully read the documentation of our library as well as the official paper. After that, if you still feel puzzled, please describe the question clearly under this issue.
in examples/rl_order_execution/scripts/gen_training_orders.py, I got two error:
df["date"] = df["datetime"].dt.date.astype("datetime64")
this will throw
TypeError: Casting to unit-less dtype 'datetime64' is not supported. Pass e.g. 'datetime64[ns]' instead.
and after I fix this problem, got another one
order_all = pd.DataFrame(df.groupby(level=(2, 0), group_keys=False).mean().dropna())
IndexError: Too many levels: Index has only 2 levels, not 3
and I change this code to
order_all = pd.DataFrame(df.groupby(level=(1, 0), group_keys=False).mean().dropna())
when traing model, get this error
Traceback (most recent call last):
File "/opt/anaconda3/envs/qlib38/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/utils/data_queue.py", line 181, in _producer
for data in dataloader:
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 46, in fetch
data = self.dataset[possibly_batched_index]
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/contrib/train_onpolicy.py", line 80, in __getitem__
stock_id=row["instrument"],
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/series.py", line 1007, in __getitem__
return self._get_value(key)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/series.py", line 1116, in _get_value
loc = self.index.get_loc(label)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3655, in get_loc
raise KeyError(key) from err
KeyError: 'instrument'
So how can we get this example run?
Env
- python 3.8
- qlib 0.9.7
Hi, @XiaotianZha Thank you for your attention to qlib. The issue you mentioned has been fixed in PR 2023. Please pull it locally and try again.
Hi, @XiaotianZha Thank you for your attention to qlib. The issue you mentioned has been fixed in PR 2023. Please pull it locally and try again.
Thank you for your help, I have already finished training, but when run backtest_opds.yml, got annother error
backtest loop: 87%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 52/60 [00:09<00:01, 5.06it/s]/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py:492: RuntimeWarning: Mean of empty slice
return np.nanmean(self.data)
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/_utils.py", line 72, in __call__
return self.func(**kwargs)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 598, in __call__
return [func(*args, **kwargs)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 598, in <listcomp>
return [func(*args, **kwargs)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/contrib/backtest.py", line 301, in single_with_collect_data_loop
records = _convert_indicator_to_dataframe(indicator_dict["1day"][1].order_indicator_his)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/contrib/backtest.py", line 69, in _convert_indicator_to_dataframe
value_dict = value_dict.to_series()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/backtest/high_performance_ds.py", line 630, in to_series
tmp_metric_dict[metric] = self.get_metric_series(metric)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/backtest/high_performance_ds.py", line 625, in get_metric_series
return self.data[metric].to_series()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/utils/index_data.py", line 615, in to_series
return pd.Series(self.data, index=self.index)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/series.py", line 425, in __init__
index = ensure_index(index)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 7128, in ensure_index
return Index(index_like, copy=copy)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 522, in __new__
raise cls._raise_scalar_data_error(data)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 5066, in _raise_scalar_data_error
raise TypeError(
TypeError: Index(...) must be called with a collection of some kind, <qlib.utils.index_data.Index object at 0x337d53370> was passed
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/anaconda3/envs/qlib38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/anaconda3/envs/qlib38/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/contrib/backtest.py", line 381, in <module>
backtest(
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/qlib/rl/contrib/backtest.py", line 329, in backtest
res = Parallel(**mp_config)(
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 2007, in __call__
return output if self.return_generator else list(output)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 1650, in _get_outputs
yield from self._retrieve()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 1754, in _retrieve
self._raise_error_fast()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 1789, in _raise_error_fast
error_job.get_result(self.timeout)
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 745, in get_result
return self._return_or_raise()
File "/opt/anaconda3/envs/qlib38/lib/python3.8/site-packages/joblib/parallel.py", line 763, in _return_or_raise
raise self._result
TypeError: Index(...) must be called with a collection of some kind, <qlib.utils.index_data.Index object at 0x337d53370> was passed
(qlib38) zhangxiaotian@zhangxiaotiandeMacBook-Pro rl_order_execution % /opt/anaconda3/envs/qlib38/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 16 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '