FinRL
FinRL copied to clipboard
AI4Finance-Foundation
Error occurs in PaperTrading_Demo notbook while running on Colab
Describe the bug The bugs occur in Part 2 Train the agent while running train cell. The error message is ValueError: Too many values to unplack (expected 4) occurs in function explore_env. Here is the message:
ValueError Traceback (most recent call last)
3 frames
ValueError: too many values to unpack (expected 4)
The following is the process for how this error comes out. I think it is from step 3 where I did wrong.
There were some bugs occurred while running on Colab. Those were
- no attribute of
pd.Timedelta(str).deltain /usr/local/lib/python3.10/site-packages/finrl/meta/data_processors/processor_alpaca.py -> Fixed by revising it topd.Timedelta(str).value - ValueError: Parameter start received with timezone defined as 'UTC' although a Date must be timezone naive. -> Fixed by adding this line in 376
ts = ts.tz_localize(None)to /usr/local/lib/python3.10/site-packages/exchange_calendars/calendar_helpers.py - TypeError: tuple indices must be integers or slices, not tuple -> Fixed by deleting
[np.newaxis, :]in line 309agent.states = env.reset()[np.newaxis, :]
Not sure how to debug this error. Also, welcome any suggestions for these debugging approaches.
Item 3. is due to a change in the Gym framework environment reset() method adding the "info" dict. to the return array making the return value a tuple. can be fixed by changing agent.states = env.reset()[np.newaxis, :] to agent.states = env.reset()[0][np.newaxis, :]
there is another issue with the env.step() method which has also been updated
there is another issue with the env.step() method which has also been updated
In new version of gymnasium env.step() return 5 paramters comprative to V21
Here is the update I used for the env.step() change ary_state, reward, terminated, _, info = env.step(ary_action) but there is now a new issue - there is no "done" flag returned, has been replaced by "terminated" - getting an error farther down with what looks like a corrupted "ary_state" which may be due to the interpretation of "termiated" vs. "done" being different The old code has: if done: ary_state = env.reset()
I updated to if terminated: ary_state = env.reset()
which I am not sure is correct All of this was working several weeks ago, and the updates to Gym happened in 2022 so I am not sure what exactly got updated where to make it stop working
This is the necessary update as Gym env.reset returns a dict with the array and "info" as well, we need to pull the array out: The old code has: if done: ary_state = env.reset()
I updated to - if terminated: ary_state = env.reset()[0]
This is the necessary update as Gym env.reset returns a dict with the array and "info" as well, we need to pull the array out: The old code has: if done: ary_state = env.reset()
I updated to - if terminated: ary_state = env.reset()[0]
Yeah it works now but while training I start getting error of:
:391: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:245.)
tensor_state = torch.as_tensor(state, dtype=torch.float32, device=device).unsqueeze(0)
ValueError Traceback (most recent call last)
5 frames
ValueError: expected sequence of length 333 at dim 1 (got 0)
same
I think I found the solution: state = env.reset()[0] state, reward, terminated, _, info = env.step(action) state = environment.reset()[0] In many places, you will see a reset code; when you get the error, see if you have reset on top of it, then index it with 0. like this state = environment.reset()[0]
After this LOC you will get error but everywhere you need the same fix.
Hello @shivesh-pandey, Can you please share your git repo for FinRL which you were able to run successfully after correcting for the above code line you recommended?
Also if possible, please share all packages version (pip list) in your environment.
We are still struggling to make this FinRL_PaperTrading_Demo.ipynb works in current state.
Thanks for your help !!
Ok, this worked for me:
Changed some code in meta/paper_trading/common.py not sure if I also changed meta/env_stock_trading/env_stocktrading_np.py but I added it anyways
https://gist.github.com/svarmo/1d66b92073f2a234ed6488ccb0d780db
pip list
finrl 0.3.6
Btw. i got it to work on the FinRL_PaperTrading_Demo_refactored.py the Jupyter notebook is still failing for some other issue
This is the necessary update as Gym env.reset returns a dict with the array and "info" as well, we need to pull the array out: The old code has: if done: ary_state = env.reset() I updated to - if terminated: ary_state = env.reset()[0]
Yeah it works now but while training I start getting error of:
:391: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:245.)
tensor_state = torch.as_tensor(state, dtype=torch.float32, device=device).unsqueeze(0) ValueError Traceback (most recent call last) in <cell line: 1>() ----> 1 train(start_date = '2022-08-25', 2 end_date = '2022-08-31', 3 ticker_list = ticker_list, 4 data_source = 'alpaca', 5 time_interval= '1Min',
5 frames in get_rewards_and_steps(env, actor, if_render) 389 cumulative_returns = 0.0 # sum of rewards in an episode 390 for episode_steps in range(12345): --> 391 tensor_state = torch.as_tensor(state, dtype=torch.float32, device=device).unsqueeze(0) 392 tensor_action = actor(tensor_state) 393 action = tensor_action.detach().cpu().numpy()[0] # not need detach(), because using torch.no_grad() outside
ValueError: expected sequence of length 333 at dim 1 (got 0)
did u get solution for this?
Ok, this worked for me: Changed some code in
meta/paper_trading/common.pynot sure if I also changedmeta/env_stock_trading/env_stocktrading_np.pybut I added it anywayshttps://gist.github.com/svarmo/1d66b92073f2a234ed6488ccb0d780db
pip list finrl 0.3.6
Thank you, I am getting the following output, which I received with the notebook as well:
TLDR; the error is "RuntimeError: could not create a primitive descriptor for a matmul primitive" Could you please direct me, what might be the issue?
full output:
/home/opc/.local/lib/python3.10/site-packages/pyfolio/pos.py:26: UserWarning: Module "zipline.assets" not found; mutltipliers will not be applied to position notionals. warnings.warn( TRAIN_START_DATE: 2023-11-03 TRAIN_END_DATE: 2023-11-10 TEST_START_DATE: 2023-11-13 TEST_END_DATE: 2023-11-14 TRAINFULL_START_DATE: 2023-11-03 TRAINFULL_END_DATE: 2023-11-14 Alpaca successfully connected Data cleaning started align start and end dates produce full timestamp index Start processing tickers ticker list complete Start concat and rename Data clean finished! Started adding Indicators Running Loop Restore Timestamps Finished adding Indicators Data cleaning started align start and end dates produce full timestamp index Start processing tickers ticker list complete Start concat and rename Data clean finished!
| step: Number of samples, or total training steps, or running times of env.step().
| time: Time spent from the start of training to this moment.
| avgR: Average value of cumulative rewards, which is the sum of rewards in an episode.
| stdR: Standard dev of cumulative rewards, which is the sum of rewards in an episode.
| avgS: Average of steps in an episode.
| objC: Objective of Critic network. Or call it loss function of critic network.
| objA: Objective of Actor network. It is the average Q value of the critic network.
| step time | avgR stdR avgS | objC objA
Traceback (most recent call last):
File "/home/opc/FinRL/FinRL_PaperTrading_Demo_refactored.py", line 80, in
@svarmo , hey thanks for letting us know , can you provide some details how did you get it to work ?
