Edward Beeching

I think the epoch skipping is related to this issue in trl: https://github.com/huggingface/trl/issues/943 I will aim to fix this next week. cc @lewtun

Thanks, perhaps we missed the `warmup_steps` when we copied the config over from our internal repo @lewtun ? Yes, there is a known bug with the constant length dataset: https://github.com/huggingface/trl/issues/943...

Isn't this in [contributing](https://github.com/huggingface/simulate/blob/main/CONTRIBUTING.md) ? Perhaps we could add a guide on the docs as well?

I noticed this API mismatch when reading the [autoreset documentation](https://envpool.readthedocs.io/en/latest/content/python_interface.html#auto-reset) yesterday. Do you have any idea when this will be fixed?

Hi @pseudo-rnd-thoughts , could you elaborate on why you think the envpool implementation is better? As I believe quite strongly that this will introduce off by one errors and overhead...

What I find so unusual about the envpool auto reset API is that the environment is not actually automatically reset, you have to call an additional step() with dummy actions...

I forgot to run style / quality. I am not on my dev machine at the moment. I will run this in an hour.

Thank you so much for your detailed replies. Unfortunately I do not have sudo privilages on our compute cluster. Is it possible to prepare the environment on my machine with...

Hi, this was removed when I moved to Godot 4.0, as I never got around to updating it. You will find it here: https://github.com/edbeeching/godot_rl_agents/tree/godot3.5/envs/example_envs/SpaceShooter

@Hardwarize did you manage to get the example working? If you managed to update it to Godot 4 then feel free to submit a PR!