[Feature request] Hyperparameter optimization from pretrained agent

[Jonathan2021]

Enabling the possibility to run --optimize with the --trained-agent flag would be great ! In my case, I pre-trained an agent on a simplified task and want to continue training it on the real task (which involves a modified reward, more obstacles etc.). It would be great to be able to run a hyperparameter search for this second phase of the training. (Even though some hyperparameters, such as the network architecture, can't be tuned here). For now, when I run both flags together, it just continues training (weirdly outputting less info than without the --optimize flag by the way). Thanks for the awesome training framework !

[araffin]

Hello, I'm unsure about such feature. On one side, it seems to be a reasonable (even though unconventional) request. On the other side there are some behaviors that may be ill-defined. For instance, if your pre-trained agent has a replay buffer size of 1e6, you should not change that during hyperparameter optimization. The same goes with other hyperparameters.

As a compromise you can fork this repo and create the feature in it (and post a link there ;)).

[Jonathan2021]

Not sure I will have enough time nor the courage to do so but maybe. Thanks for the reply anyways ;)