Big bug in PPO2

[Vinson-sheep]

In dist = Normal(mu, sigma) , sigma should be a positive value, but actor_net output can be negative, so action_log_prob = dist.log_prob(action) can be nan.

Try:

import torch
a = torch.FloatTensor([1]).cuda()
b = torch.FloatTensor([-1]).cuda()
dist = Normal(a,b)
action = dist.sample()
action_log_prob = dist.log_prob(action)

print(action.cpu().numpy())
print(action_log_prob.item())

[jzl20]

so how can I fix the bug ？

[flyinglife001]

return sigma*sigma

[WhiteNightSleepless]

You can add an activation function before the output of actor network. Using relu or softplus function may change sigma into a positive value. Hope it helps.