What would be the easiest way to initialise the value or policy networks differently?

[nbro]

Is the only possible way just to re-define all layers with a different initialization, something like in this example: https://stable-baselines.readthedocs.io/en/master/guide/custom_policy.html? Is there a concrete example around for my purpose? I just want to initialize the value or policy networks differently. I don't want to do any other stuff (at least, for now).

[Miffyli]

Scroll down a bit to this example, which should work as a good template. You can change the initialization of layers when they are created (e.g. the tf.layers.dense calls). Alternatively you could modify parameters directly after creation with load_parameters function.

As a sidenote, stable-baselines3 is easier to tweak like this as it is based on PyTorch.