Denys Makoviichuk

It looks like half done and I didn't find KL loss parts. Only policy loss.

Exploration loss question

4

I see you have two options for the exploration loss: symmetric_kl_with_uniform_prior and entropy. And you used symmetric KL as the loss in the vizdoom config. Is there any paper or...

zero_rnn_on_done by default is True.

Added masks for envs

2

It was requested in this ticket: https://github.com/Denys88/rl_games/issues/194 Two things should be done: 1) env_info returned from the vec_env should have 'env_masks' : True 2) method get_env_masks(self) should be implemented

[WIP] Added SHAC Support

1

- Added SHAC implementation - Aligned SHAC with PPO and SAC implementations. - Exposed min_lr for linear and adaptive schedules and max_lr for adaptive

Simply doesn't work. Not going to merge right now. To try vmpo one can run: python runner.py --train --file rl_games/configs/vmpo_lunar_discrete_torch.yaml

Some issues if try to use it in a real life :)

1

Hi, I tried to use your product but got a lot of small issues and found some lack of functionality. 1) Please don't expect that git address is something line...

Added gymnasium and gym 0.26 support

3

We used https://gymnasium.farama.org/content/migration-guide/ and shimmy library. https://github.com/Denys88/rl_games/issues/261

https://arxiv.org/pdf/2212.06343.pdf

auto lambdaWithReturn = [](){return 5; }; std::function f = lambdaWithReturn;