Any plans for CUDA support?

[slerman12]

[fabiopardo]

Tonic does not prevent CUDA to be used actually. Do you have something specific in mind?

[slerman12]

Is there a toggle or quick way to turn on CUDA?

[slerman12]

Honestly, with a few more basic features — a CUDA toggle, resume training, and (harder to implement) Atari integration — this would be the best library for RL research in my opinion.

[fabiopardo]

If you install TensorFlow GPU, CUDA will automatically be used. I usually disable CUDA with CUDA_VISIBLE_DEVICES='' when not using a large number of workers, images or recurrent networks because I found that the CPU is actually faster for simple RL from states. If you need more control or if you use PyTorch, I believe you should be able to achieve the result you want in a few different ways, for example (1) via the header field when using the training script, (2) by creating your own modules such as models or updaters and loading them as explained in the paper, (3) by creating your own experiment script instead of using the training script, instantiating your agent and environments and launching a trainer. Let me know if you still believe something specific should be added to Tonic regarding CUDA support.

Thanks for your kind words. I unfortunately don't have a lot of time to work on Tonic these days but I would be very happy to help others add features.