Regression loss smooth L1

[rvandeghen]

Hello, I'm using your code and i managed to understand almost everything expect this line: regression_loss = torch.where( torch.le(regression_diff, 1.0 / 9.0), 0.5 * 9.0 * torch.pow(regression_diff, 2), regression_diff - 0.5 / 9.0 ) I don't understant this implementation of the smooth L1 loss (why compare \Delta reg with 1/9) ? and thus why mutliply by 9 in one case and divide 0.5/9 in the other ? I'm referencing this implementation foe the loss : https://pytorch.org/docs/stable/nn.html#smoothl1loss

[yhenon]

It's kind of confusing because of the paper text not fully explaining it. The smooth loss transitions from l1 to l2 behaviour at a point, See here for an explanation: https://github.com/fizyr/keras-retinanet/issues/652 .

Basically when regression_diff is below 1/9, we have L2 behaviour. When regression_diff is above 1/9, we have L1 behaviour (to ensure loss isn't dominated by a few outliers). We want a smooth transition (both the value and the gradient should be equal at the transition point) which the formulation ensures.