[Feature Request] Reducing memory consumption when using HerReplayBuffer

[TMats]

🚀 Feature

• In HerReplayBuffer, initialize self.self._buffer considering dtype of each inputs

Motivation

In the implementation of HerReplayBuffer, self.self._buffer is initialized with zeros of np.float32, which may lead to huge memory allocation. If the buffer for each input is initialized with the respective dtypes, we can save memory and increase the buffer size especially when using image observations (saving images as uint8 instead of float32).

https://github.com/DLR-RM/stable-baselines3/blob/2cc1477fa28e654caa4b7bf9b367febe726513e8/stable_baselines3/her/her_replay_buffer.py#L133-L148

A similar one is implemented in DictReplayBuffer

https://github.com/DLR-RM/stable-baselines3/blob/2cc1477fa28e654caa4b7bf9b367febe726513e8/stable_baselines3/common/buffers.py#L530-L537

Pitch

• This enables increasing the buffer size when using HERReplayBuffer

### Checklist

• [x] I have checked that there is no similar issue in the repo (required)

Thanks developers!

[araffin]

Duplicate of https://github.com/DLR-RM/stable-baselines3/issues/720, it is addressed in https://github.com/DLR-RM/stable-baselines3/pull/704 but I think I would welcome a separate PR as it is not clear if #720 will be merged and when ;)

[TMats]

oh, sorry I overlooked this issue. thanks!