Skander Moalla

A strong benefit of the native frameskip is that it can skip rendering the skipped frames. This results in a considerable rollout speedup. If frameskip is moved out of the...

@ash0ts (cc @kptkin) I've just faced this issue recently in a grid sweep with multiple agents in parallel running `wandb` in offline mode. The runs seems to be duplicated every...

Somehow In the PPO example, the advantage module is called on the rollout batch shape https://github.com/pytorch/rl/blob/147de71d090d5705182bfabd24a99f3b2ee4cec9/examples/ppo/ppo.py#L103 and doesn't crash with the conv2d complaining. https://github.com/pytorch/rl/blob/147de71d090d5705182bfabd24a99f3b2ee4cec9/examples/ppo/utils.py#L341 I also managed to reproduce this...

Okay, so the `ConvNet` of TorchRL actually flattens the batch before running a forward and then unflattens it back. [https://github.com/pytorch/rl/blob/147de71d090d5705182bfabd24a99f3b2ee4cec9/torchrl/modules/models/models.py#L479](https://github.com/pytorch/rl/blob/147de71d090d5705182bfabd24a99f3b2ee4cec9/torchrl/modules/models/models.py#L479) Maybe this could be made clearer to the user so...

I'm not very familiar with `vmap`, but does the memory taken by the model weights stay the same when you `vmap` it?

To reproduce the bug on `ParallelEnv` you need some wizardry: - Change the environment to `"MountainCar-v0"`. - Change the truncation key, otherwise you fall into #1865 - Add an empty...

Nice, thanks! Indeed it's probably device casting gone wrong somewhere as MPS crashed with segfault literally. Could you reproduce the one with ParallelEn? That's as impactful as the SerialEnv one.

```bash VERBOSE=1 python -c """import tqdm from tensordict.nn import TensorDictModule from torch import nn from torchrl.envs import ( EnvCreator, ExplorationType, StepCounter, TransformedEnv, ParallelEnv, ) from torchrl.envs.libs.gym import GymEnv from torchrl.modules...

`SerialEnv` example was solved with #1866. I also tried poking a bit and it was fine. I will the `ParallelEnv` one.

The problem is now different with `ParallelEnv` that's why it probably didn't error for you @vmoens. ```bash VERBOSE=1 python -c """import tqdm from tensordict.nn import TensorDictModule from torch import nn...