DQN_Agent in shm

[excellentiam-prog]

This is a work of art, it would be wonderful to find examples inspired by you with PPO and other agents.

I uncommented and tried to test the DQN_Agent in https://github.com/druce/rl/blob/master/Trading_with_RL.ipynb https://github.com/druce/rl/blob/master/Trading%20with%20RL.ipynb because even if it is slow, it still has great value for those who are learning, like myself. I only removed the “import resources” and its dependents because I am running it on Windows

I tried " N_EPISODES = 10000 ticks_per_episode = 1256 nstocks = 1

env = Market(shm_market_gen, nstocks=1, episode_length=ticks_per_episode)

agent = DQN_Agent(state_size=nstocks*32, action_size=2, )

start_time = time.time()

for e in range(N_EPISODES): agent.run_episode() agent.score_episode(e, N_EPISODES)

if e and (e+1) % agent.save_interval == 0:
    agent.save()

elapsed_time = time.time() - start_time print("Train time: ", elapsed_time)
agent.rlplot()" Unfortunately, I got:

Traceback (most recent call last): File "C:\Users\guest\AppData\Roaming\Python\Python310\site-packages\IPython\core\interactiveshell.py", line 3526, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "C:\Users\guest\AppData\Local\Temp\ipykernel_11800\4180752183.py", line 19, in agent.run_episode() File "C:\Users\guest\AppData\Local\Temp\ipykernel_11800\165701735.py", line 198, in run_episode self.action = self.act(self.state.reshape([1, self.state_size])) ValueError: cannot reshape array of size 8 into shape (1,32)"

[excellentiam-prog]

I did some changes fixing the "env = Market(shm_market_gen, lag=16, nstocks=1, episode_length=10)", lag 16 fixed the "shape (1,32)" issue and seemed that the model is working. However, it crashed one time in my pc on Wndows, and other time in colab, I think that is because of the high memory use in DQN_Agent with the memory being a dataframe, while in REINFORCE it is a list. The DQN_Agent is slow, but this is ok to me, the true problem is the quantity of memory