function _sigmoid() question

[annzheng]

Hi, I found in function _sigmoid(), you clamp the value between (1e-4, 1-1e-4) after sigmoid, why need to do this? Here is the code: def _sigmoid(x): x = torch.clamp(x.sigmoid_(), min=1e-4, max=1-1e-4) return x

[heilaw]

We did it to make the training more stable.

[developer0hye]

Hi @heilaw!

How does it make the training more stable?