The effect of NormalizedEnv

[pengzhi1998]

Hi, thank you for this great implementation!! However, I'm not very sure about the effect of normalized_env.py. Actually, if I remove it, the results seem to be worse than not removing it. What does it do? Look forward to your reply!

[zhihanyang2022]

There are several levels for my answer:

• Strangely, I had to change the method names from _action and _reverse_action to action and reverse_action for the code to work - maybe this has to do with the gym version (mine has version 0.18.0).
• If you try adding print statements in the action and reverse_action methods, you would see that action is being called repeatedly but reverse_action is never called.
• The reason why Gym has the ActionWrapper class and the action method is that: "Used to modify the actions passed to the environment." See reference at https://alexandervandekleut.github.io/gym-wrappers/. I'm actually not sure about the reverse_action method so I would welcome any other thoughts on this.
• So basically the actor network would output something in the range of Tanh, but you can imagine an environment in which the range of possible actions is, for example, [2, 7].

[pengzhi1998]

Thank you for your response!