Albert Bou

Is it possible to specify in which GPU I want the environment to run?

11

I have seen that, by default, all instances of the obstacle tower environment I start run in the primary GPU (in my case GPU:0). Is it possible to specify in...

[Feature Request] Information of RNNs expected inputs and outputs difficult to access when part of larger architectures

2

## Motivation When RNN’s are used in isolation, creating a TensorDictPrimer Transform for the environment to populate the TensorDicts with the expected tensors is pretty straightforward: ```python from torchrl.modules import...

[Feature] Crop Transform

1

## Description Currently TorchRL has a CenterCrop transform. I have recently encountered a situation were I did not want to crop the center, but another part of the obs image....

[BUG] No support for python >= 3.9

3

In Python >= 3.9, `import safe` results in `ImportError: cannot import name 'gcd' from 'fractions' ` I believe it is related to the `math` package https://stackoverflow.com/questions/66174862/import-error-cant-import-name-gcd-from-fractions

[Feature] SAC compatibility with composite distributions.

1

## Description This PR adapts the SAC objective to be compatible with composite distributions. DiscreteSAC seems a little bit more tricky, as it requires computing the log probabilities of all...

[BUG] RuntimeError when passing dialogue data to LLMEnv

1

## Describe the bug I see very cool advancements in the direction of LLM RL training in the repo, awesome work! :) After playing a bit with the LLMEnv I...

No Significant Improvement Observed in Model Training Speed

1

I am trying to speedup inference and training of a `mistralai/Mistral-Small-3.1-24B-Instruct-2503` model. Simply replacing `AutoModelForCausalLM` with `AutoLigerKernelForCausalLM` does not lead to any speedup in my sampling speed or memory usage....