Cedric Derstroff

Question: TorchApproximator.predict - Why no torch.no_grad() and why call forward directly?

1

**Is your feature request related to a problem? Please describe.** I was wondering why the [predict method in class TorchApproximator](https://github.com/MushroomRL/mushroom-rl/blob/af7dec46c28a83d133c2b985746f54c8b25aa95e/mushroom_rl/approximators/parametric/torch_approximator.py#L80) calculates the gradients and calls ``self.network.forward(*torch_args, **kwargs)``. **Describe the solution...

Your replay memory only stores samples one by one. Since you're also training RNNs, you should use sequntial training samples. [This paper by Hausknecht and Stone](https://arxiv.org/pdf/1507.06527.pdf) explains what I mean.

Solving #3 by initializing U and V instead of directly using what's in the memory.

At first, thanks for the great implementation. I found that this initialization is somewhat critical: https://github.com/dhruvramani/Transformers-RL/blob/fc3d8af2f9b2ffc055855c755a1c892e4ae359d2/layers.py#L256 It may occur that self.u and self.v will be initialized containing nan. Eventually, this...

wandb sync: --no-include-synced does not do anything

3

### Current Behavior When using ``wandb sync`` with the ``--no-include-synced`` and ``--mark-synced`` flags, every single run gets synced again which takes quite a lot of time if you have done...

 When clicking the link to export to Google Calendar, an error message pops up. Changing https to http in the source: https://calendar.google.com/calendar/r?cid=http://aideadlin.es/ai-deadlines.ics instead of https://calendar.google.com/calendar/r?cid=https://aideadlin.es/ai-deadlines.ics fixes this issue.

When I put a history of length 0 for the MuJoCo environment wrapper to just get the POMDP verion, I get ```diff - ValueError: History length must be greater or...