Error occurs in PaperTrading_Demo notbook while running on Colab

[jiahau3]

Describe the bug The bugs occur in Part 2 Train the agent while running train cell. The error message is ValueError: Too many values to unplack (expected 4) occurs in function explore_env. Here is the message:

ValueError Traceback (most recent call last) in <cell line: 1>() ----> 1 train(start_date = '2022-08-25', 2 end_date = '2022-08-31', 3 ticker_list = ticker_list, 4 data_source = 'alpaca', 5 time_interval= '1Min',

3 frames in train(start_date, end_date, ticker_list, data_source, time_interval, technical_indicator_list, drl_lib, env, model_name, if_vix, **kwargs) 57 ) 58 model = agent.get_model(model_name, model_kwargs=erl_params) ---> 59 trained_model = agent.train_model( 60 model=model, cwd=cwd, total_timesteps=break_step 61 )

in train_model(self, model, cwd, total_timesteps) 74 model.cwd = cwd 75 model.break_step = total_timesteps ---> 76 train_agent(model) 77 78 @staticmethod

in train_agent(args) 315 torch.set_grad_enabled(False) 316 while True: # start training --> 317 buffer_items = agent.explore_env(env, args.horizon_len) 318 319 torch.set_grad_enabled(True)

in explore_env(self, env, horizon_len) 200 201 ary_action = convert(action).detach().cpu().numpy() --> 202 ary_state, reward, done, _ = env.step(ary_action) 203 if done: 204 ary_state = env.reset()

ValueError: too many values to unpack (expected 4)

The following is the process for how this error comes out. I think it is from step 3 where I did wrong.

There were some bugs occurred while running on Colab. Those were

1. no attribute of pd.Timedelta(str).delta in /usr/local/lib/python3.10/site-packages/finrl/meta/data_processors/processor_alpaca.py -> Fixed by revising it to pd.Timedelta(str).value
2. ValueError: Parameter start received with timezone defined as 'UTC' although a Date must be timezone naive. -> Fixed by adding this line in 376 ts = ts.tz_localize(None) to /usr/local/lib/python3.10/site-packages/exchange_calendars/calendar_helpers.py
3. TypeError: tuple indices must be integers or slices, not tuple -> Fixed by deleting [np.newaxis, :] in line 309 agent.states = env.reset()[np.newaxis, :]

Not sure how to debug this error. Also, welcome any suggestions for these debugging approaches.