gliese581gg
two questions on parallel implementation in the batch-A3C
Hello,
I looked at your bA3C code and learned a lot. Thank you so much for sharing your codebase implementing this idea. I have two questions mainly on your code. BTW, I do not think your code has any problem, I just have a few speed concerns that want to discuss with you a little bit.
(1) python multi-threading is known to be slow for CPU-bound problem because of GIL. I am not sure whether this embedded python thread locker would slow down the job although GPU is the main executer within the worker, at least I guess gym environment in charge of playing the game seems to constantly use CPU.
(2) for parallel env exploration, since the policy and value network is globally shared, would GPU performance be reduced due to the concurrent visits by different workers?
Best, Tian
Hi!
(1) You are right. I had no idea about GIL when I implement this.
Now I am using multiprocessing and Pytorch, which supports shared cuda memory across processes.
I will upload pytorch version when I have a time:)
(2) I have not thought about this. Maybe I have to test whether shared network slows down learning in multi processing
Cool, I am glad that we have similar concerns :) BTW, I tried Cython, it has the way to release the GIL so it can provide a real multi-threading env, but all codes need to rewrite in the C grammar.