Antonin RAFFIN

Hello, I'm afraid the problem comes from the algorithm itself. ACKTR uses KFAC as an optimizer and apparently the current implementation is memory hungry, so it works fine with vector...

I managed to train ACKTR for atari games in the [rl zoo](https://github.com/araffin/rl-baselines-zoo). Hyperparameters can be found [here](https://github.com/araffin/rl-baselines-zoo/blob/master/hyperparams/acktr.yml) I had the same problem of learning getting slower and slower, but 32...

>@araffin comments? I favor not changing stuff, given maintenance mode of SB2. I would agree with this. We mostly accept only bug fixes and non-breaking changes now. And if you...

Hello, >However, the trained neural network is incapable of eliminating the steady-state error, as shown in the figure below. Correct me if I'm wrong, but a PD controller cannot remove...

>3 different environments each with its own dataset..... If you have datasets, you can probably compute the statistics and normalize in advance, so you should not need `VecNormalize`, no?

>when I save a training to a model file, are the weights being intelligently blended together, regardless of if I'm training the same model in 5 different processes, or will...

Hello, > Is there a reason why it is fixed, or is it dependent on any other parameter? this is legacy code from OpenAI baselines... so only @andrewliao11 knows probably......

>Most algorithms have an episode count, and it would be nice to have a common axis (other than total_timesteps) to plot stuff with. If you use a `Monitor` wrapper (included...

> Could you expand more on this? I am using EvalCallback during training to check performance and save the best model This is unrelated to the issue, please read the...

Hello, the paper your are looking for is maybe [Continuous Deep Q-Learning with Model-based Acceleration ](https://arxiv.org/abs/1603.00748) with the Normalized Advantage Function (NAF) see [notes](https://github.com/keras-rl/keras-rl/blob/master/docs/sources/agents/naf.md) in Keras RL. You also have...