[Feature]: bridging module for backends and frontends

[typoverflow]

Prerequisites

• [X] I have tried updating UtilsRL to the newest version.
• [X] I have checked both open and closed issues but found nothing related to my request.

UtilsRL verison when proposing this request

0.4.8

What I am expecting

For now although we have RNN and attention blocks implemented, we cannot use them directly because it's somewhat incompatible with the RL actors and critics.

• RNN will yield intermediate cell states and cell hiddens each time we forward-pass the tensors. The block will return the intermediate products as result, but RL actors and critics are unable to collect and pass them to buffers.
• Attention blocks may need a 'reduce' method the absorb the timestep dimension. For example, in Decision Transformer, we pass in a sequence of tuples but only need the last action prediction as the output of our actor.

In short, we need a bridging module between the backends and frontends to efficiently 1) collect the intermediate results, and 2) reduce the dimension and arrange the results into their expected shape.

Possible solutions

No response

Any additional messages which might help

No response

Urgency

Very urgent, this feature is relevant and should be implemented immediately.