Recurrent Experience Replay

[smorad]

Is your feature request related to a problem? Please describe.

It seems that the implemented replay buffers only operate over transitions, with no ability to operate over entire sequences. This prevents the use of recurrent policies for tackling POMDPs.

Describe the solution you'd like

A SequenceReplayBuffer that returns contiguous episodes instead of shuffled transitions.

Describe alternatives you've considered

Additional context