Meta-RL or feeding output *and* loss back into the input

[jpeg729]

Do you intend to work on meta-RL any time soon?

In order to better train an agent that interacts with a world, the basic idea involves building an RNN capable of keeping track of the particularities of its environment from one time step to the next. https://arxiv.org/pdf/1611.05763v3.pdf

This involves concatenating the last action and its reward to the inputs for the next time step.

I am interested in a simpler case which would only need me to add the last output and its associated loss to the input at each time step. And I can see that being particularly inefficient in Keras, not to mention that with timesteps set to 1 there can be no significant BPTT.

[crosleythomas]

I think this would be done by setting readout='pack'

see https://github.com/datalogai/recurrentshop/blob/master/recurrentshop/engine.py#L272

but there seems to be a bug in RecurrentShop right now because Keras backend does not have a pack function. @farizrahman4u was that meant to be a call to the Tensorflow backend?