Steven Kapturowski

Hi @ionelhosu, I think when it's running at 220 iter/sec the training hasn't actually started yet; it's just filling the replay buffer until it reaches some minimum size. That explains...

The TRPO learner currently isn't being checkpointed like the other algorithms, but I'll try to get a fix for that in tonight. Also note, at present you would need to...

I do. I've been swamped with work recently so I forgot to take care of this but I should have time to fix it this weekend.

@dhfromkorea hmm, that's interesting. I've run this code on mac and ubuntu without any issue. The fast_cts code is significantly faster; I'll have to double check on the exact speedup,...

Can you let me know the python and library versions that you're using? Also, when you tried the the original cts model can you describe precisely which changes you made...

@dhfromkorea I could imagine parallelizing the per-pixel CTS model updates, but as there are really no matrix operations going on I'm not sure that running it on a GPU would...

@dhfromkorea I ran the agent several times and it usually would get close to that score. If it's not getting above 100 then there's something quite seriously wrong. May I...

Can you try adding the following options: --q_target_update_steps=30000 --max_global_steps=160000000 --epsilon_annealing_steps=500000 --replay_size=500000 --clip_norm_type=ignore The first 4 I'm suggesting mainly for consistency with my experiments; I suspect the norm clipping may be...

Hi @sangjin-park, I'd be happy to try to debug what's going on in the server but first could you try running on the commit 39e695696488df83bf6d08a1eb7df0ff4ebd109c and tell me if there's...

I'm going to check the diff between commit 452d57 and master to see what went wrong and get a fix out asap