rl [Feature Request] TorchRL for Python 3.13t

Hello, PyTorch recently added support for Python 3.13t, aka "no-gil" Python (https://github.com/pytorch/pytorch/issues/130249). Are there any plans to make TorchRL usable from this version? It would be nice if this could be supported soon, as the ability to run CPU code for interactions with the environment from multiple no-gil threads could lead to major performance improvements and higher GPU utilization.

Nov 04 '24 18:11 patchmeifyoucan

That's definitely on our radar! Beyond multithreaded envs another big benefit would be reducing the cpu overheads across the lib (sampling in rbs, iterating over TensorDict etc).

Is there anything we should prioritise in your opinion?

Nov 06 '24 11:11 vmoens

Thank you, good to know.

For me, simply general support would be sufficient to get a better GPU utilization, that's what I'm after. I mostly use torchrl's replay buffers but their performance is far from being a bottleneck in my application.

Nov 06 '24 12:11 patchmeifyoucan

Is it somehow possible to install torchrl with a 3.13t python right now from nightly builds? TorchRL remains my last dependency which I was not able to install onto a 3.13t python. Or is there work in progress that needs to be finished before that release is possible? If yes, is there any expected time schedule for a new version?

Dec 09 '24 18:12 patchmeifyoucan

@vmoens

Beyond multithreaded envs another big benefit would be reducing the cpu overheads across the lib (sampling in rbs, iterating over TensorDict etc). Is there anything we should prioritise in your opinion?

I'm a different user, but at least in my case this would be a huge win for me -> iterating over TensorDict

Thanks for your work! I'll subscribe to this Feature Request. TorchRL (wth TensorDicts) for Python 3.13 (no GIL) would be amazing to have.

Jan 04 '25 10:01 ljfp

Any updates here?

Apr 10 '25 12:04 patchmeifyoucan

I didn't manage to get much speed-ups for TD with python 3.13t -- but i'm not losing hope! We just need to port it for expensive stuff like consolidate etc. At least the packages are now shipped for this python version, it's a start!

May 20 '25 10:05 vmoens

Hi @vmoens, any plans to let MultiaSyncCollectors use threading instead for no-gil versions?

Jun 13 '25 15:06 maichmueller