Doubt in Probability Attention Module

[vidit98]

Thanks for the work. May I know if there is an explanation for the condition on the value of gamma in this line

    if self.gamma < -0.01:
        out = x
        attention = None
    else:
        proj_value = x.view(B, -1, W * H)  # D reshape (B,C,H*W)

        proj_query = x.view(B, -1, W * H).permute(0, 2, 1)  # B reshape & transpose (B,H*W,C)
        proj_key = x.view(B, -1, W * H)  # C reshape (B,C,H*W)
        energy = torch.bmm(proj_query, proj_key)  # batch matrix multiplication, (B, H*W, H*W)
        attention = self.softmax(energy)

        out = torch.bmm(proj_value, attention.permute(0, 2, 1))  # (B,C,H*W)
        out = out.view(B, C, H, W)  # new attentioned features

        out = self.gamma * out + x

    return out, attention