genrl icon indicating copy to clipboard operation
genrl copied to clipboard

Published 20 hours ago • verified publisher icon SforAiDl

CUDA support Agents

Open sampreet-arthi opened this issue 5 years ago • 9 comments
trafficstars

sampreet-arthi avatar Jun 07 '20 22:06 sampreet-arthi

Is this still an issue?

Sharad24 avatar Jul 18 '20 23:07 Sharad24

I'll resolve it while working on Off Policy Agents.

sampreet-arthi avatar Jul 19 '20 05:07 sampreet-arthi

For this, we just need to check for CUDA support on the system and then use the CUDA functions for computation right?

IshanDaga avatar Sep 02 '20 14:09 IshanDaga

Yes. Just ensure things that can be on the GPU, are on the GPU. E.g. Agents, Tensors in the Buffers and random tensors in the Trainers (there should not be a lot of them though).

If you go through a few agents, you'll see that there's a bunch of sporadic cpu calls which would disrupt that. So I think thats the only other thing to be careful about that.

Let me know if you're planning to take it up or if you need more info about this.

Sharad24 avatar Sep 02 '20 14:09 Sharad24

I'll raise a PR with some initial changes and we'll take it from there?

IshanDaga avatar Sep 02 '20 15:09 IshanDaga

Sounds good!

On 02-Sep-2020, at 8:31 PM, IshanDaga [email protected] wrote:

I'll raise a PR with some initial changes and we'll take it from there?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SforAiDl/genrl/issues/162#issuecomment-685794835, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH72FJY4HQUSVDGQQKYYXWDSDZM57ANCNFSM4NXWOCWA.

Sharad24 avatar Sep 02 '20 15:09 Sharad24

Still working on this @IshanDaga ?

hades-rp2010 avatar Nov 06 '20 08:11 hades-rp2010

Nope. Not at the moment

IshanDaga avatar Nov 08 '20 15:11 IshanDaga

Yes. Just ensure things that can be on the GPU, are on the GPU. E.g. Agents, Tensors in the Buffers and random tensors in the Trainers (there should not be a lot of them though).

If you go through a few agents, you'll see that there's a bunch of sporadic cpu calls which would disrupt that. So I think thats the only other thing to be careful about that.

Let me know if you're planning to take it up or if you need more info about this.

Would it be better to -

  1. Add a separate arg to the buffers, giving a choice to where to place them (ie GPU or CPU)
  2. after sampling the list of tensors from the buffer, send these tensors to the GPU

I think 2 would be slower since we'd have to do a CPU-GPU transfer on every update step, but not sure if 1 is tractable as the buffer size grows larger

hades-rp2010 avatar Nov 23 '20 21:11 hades-rp2010