Fix bootstrapped DQN

[Kaixhin]

The test on Beam Rider is failing badly, and does not look promising.

[iassael]

Hi @Kaixhin, did you try using my layer?

[Kaixhin]

@iassael couple of questions about your layer. Can it use more complicated heads (like the dueling head)? How does it work on picking a new head for a new episode vs. using the mode in ensemble mode (during evaluation)? Is it possible to train with the "full" version of the bootstrap - when each head requires a separate experience replay memory?

[iassael]

hey @Kaixhin currently nope. For the former we could pass the module as a parameter, and for the latter it should be super easy to extend it with an extra parameter of the episode id.

[Kaixhin]

@iassael I'm focusing on some of the other components at the moment so I'm not sure I'll get to this any time soon, but feel free to give it a shot if you can.

[iassael]

@Kaixhin I'll keep you posted and thanks for the awesome work cheers~