[Question] rollout_buffer_class in RecurrentPPO

[SaltyPoseidon]

❓ Question

Hello, when custom rollout will be added to Recurrent networks as in main sb3? Is there easy way to solve this problem now?

Checklist

• [x] I have checked that there is no similar issue in the repo
• [x] I have read the documentation
• [x] If code there is, it is minimal and working
• [x] If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Hello, could you elaborate?

[SaltyPoseidon]

I'm trying to implement this paper https://arxiv.org/pdf/2207.07467 and add LSTM-support. But they are using their custom rollout buffer.

I have implemented it in the base sb3 PPO as it supports rollout_buffer_class=None, https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html#stable_baselines3.ppo.PPO

Now I'm trying to perfom this in the RecurrentPPO, but it doesn't support such features

Actually, I found such PR https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/pull/214

[araffin]

Actually, I found such PR https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/pull/214

Yes, from the PR: "I decided not to add it to MaskeablePPO and RecurrentPPO since they require special rollout buffers."

Custom rollout buffers for mask PPO or LSTM PPO require additional changes, so I would recommend you to either subclass or fork SB3 contrib for your use-case.

[SaltyPoseidon]

Thank you