Antonin RAFFIN comments

Results 880 comments of


                                            Antonin RAFFIN

When using custom environments in a loop that trains multiple models, CUDA VRAM is never deallocated when iteration is done.

Hello, this sounds more like a pytorch issue, no?

When using custom environments in a loop that trains multiple models, CUDA VRAM is never deallocated when iteration is done.

> My understanding was that you can del the tensors, etc. when the work is done, but I'm not sure where to add them to the code within SB3 that...

[Feature Request] Saving rendered trajectories in evaluations

Hello, it seems that you need to use a `VecVideoRecorder` as we do in the RL Zoo: https://github.com/DLR-RM/rl-baselines3-zoo/blob/8cecab429726d7e6aaebd261d26ed8fc23b7d948/rl_zoo3/record_video.py#L141-L147 > In a headless server, the simplest way to examine the behavior...

[Feature Request] Saving rendered trajectories in evaluations

> Bump for the returning of the rendered data from evaluate_policy. I manipulate the rendered data directly before sending it to Weights&Biases, and saving it as a file to re-open...

[Feature Request] Saving rendered trajectories in evaluations

> Sorry, I re-read the FE and I guess it's not exactly what I was looking for after all. FE? > Essentially, evaluate_policy calls env.render() but if this is rgb_array...

[Feature Request] Implementation of Lattice exploration (Chiappa et al., NeurIPS 2023)

Hello, thanks for the proposal. I still need to read the paper, but I would welcome anyway if you could do a PR for the project section of the documentation...

[Feature Request] Implementation of Lattice exploration (Chiappa et al., NeurIPS 2023)

> I still need to read the paper, but I would welcome anyway if you could do a PR for the project section of the documentation =) (combining Lattice exploration...

Documentation/Implementation mismatch in sum_independent_dims function

Hello, you are right, the doc is wrong. However, I would keep the behavior as-is to avoid any breaking change. A PR that updates the doc would be welcomed =)

[Bug]: Iteration not updated in locals while learning

Hello, if you want to have the number of iterations, the best is to have a counter that you increment as every call of `on_rollout_end()`.

[Feature Request] Customize the features_extractor from a static or pre-trained model

Hello, a VecEnv/gym wrapper would do the job in your case, no?