Deep-Reinforcement-Learning-Algorithms-with-PyTorch A question about critic-loss in discrete sac？

A question about critic-loss in discrete sac？

Open outshine-J opened this issue 2 years ago • 5 comments

I applied the code of discrete sac to a custom discrete action environment. During the training process, I found that the loss of critic did not decrease but increased, and the critic-loss value after the increase was very large, even reaching 200+, what is the problem? Caused, how can I fix it? thanks.

Oct 08 '22 07:10 outshine-J

您好，我是范仁义，您的邮件我已经收到，我会尽快处理，谢谢。

Oct 08 '22 07:10 fry404006308

Added, the same happens even if I crop the reward.

Oct 08 '22 07:10 outshine-J

@outshine-J Hello, I have encountered the same problem, have you solved it?

Nov 04 '22 08:11 Mengyu-Messic

您好，我是范仁义，您的邮件我已经收到，我会尽快处理，谢谢。

Nov 04 '22 08:11 fry404006308

@Mengyu-Messic You can find the answer by following the link. https://github.com/toshikwa/sac-discrete.pytorch/issues/12#issue-708665275. Other than that you can change this by setting a fixed temperature.

Nov 04 '22 09:11 outshine-J

Deep-Reinforcement-Learning-Algorithms-with-PyTorch Deep-Reinforcement-Learning-Algorithms-with-PyTorch copied to clipboard

A question about critic-loss in discrete sac？

Deep-Reinforcement-Learning-Algorithms-with-PyTorch
Deep-Reinforcement-Learning-Algorithms-with-PyTorch copied to clipboard