Necessity of a Custom optimizer for the Critic (A2C).

[davinellulinvega]

Hello Germain / Everyone,

I am currently trying to implement the A2C algorithm as part of a simulation for my PhD. Given that, I have very limited time to do so, your source code is a great help, since the algorithm and operations are clearly outlined and not hidden away as is the case for OpenAI baseline implementation. Still after having a look at the code in critic.py, I was wondering why did you define a custom optimizer for the critic has well (it is clearly justified for the actor), when simply compiling the critic network and passing MSE as the loss seem to have the same effect? Is there something I am missing here? Anyway, that was just a though nothing game changing. Thanks a lot for sharing those implementations.