astooke

OK interesting...the environment should only be reset when the `done` signal comes out `True`. Does this happen for a procgen env before the trajectory is finished? Possibly related, the Atari...

oh good idea on that bit of backward compatibility! thnx for posting. at some point will probably just move everything forward to 1.4 or 1.5, unless there is some reason...

Yes, sorry about that, I should clean that up. Meantime, this one should be fresh, and with all the bells and whistles: [rlpyt/experiments/scripts/atari/dqn/launch/pabti/launch_atari_r2d1_async_alt_gravitar.py](rlpyt/experiments/scripts/atari/dqn/launch/pabti/launch_atari_r2d1_async_alt_gravitar.py)

Sorry! Was just a bad link, here it is: https://github.com/astooke/rlpyt/blob/master/rlpyt/experiments/scripts/atari/dqn/launch/pabti/launch_atari_r2d1_async_alt_gravitar.py

@crizCraig Thanks for the kind words! Fixed the gpu_basic case in commit: 229f4bf1a1b9eb274dfa19d858dd3c0443939c05 (had bad, leftover code for putting samples into the replay which was used in non-async modes, such...

Not sure about the problem with the async_alt code, though. What errors are you hitting? Or could you print out what the affinities are that are being made? The affinity...

Hmmm, strange. I haven't used nvprof, so I don't immediately know what's wrong. But maybe that `MemoryError` says that nvprof is somehow interfering with the way that rlpyt is allocating...

Anyone else revisited this?

OK interesting! Then the problem is probably allocated the replay buffer. Try `DQN(replay_size=int(1e5))`? The default is `1e6`.

Right, I don't recall that I ever ran Breakout, but I have previously had DQN implementations come back kind of low on that game, I'm not sure why. One possibility...