label smoothing should NOT only work under multi-class settings

[makecent]

Labeling smoothing is a widely used regularation method in action recognition tasks. However, currenly label_smooth_eps only works when the multi-class is toggled on (which default False): https://github.com/open-mmlab/mmaction2/blob/6d6685632f28344e98cf34a14d1226cd6c008391/mmaction/models/heads/base.py#L106-L108

I suggest to decompose the multi-class and label smoothing, e.g.:

if cls_score.size() != labels.size():
    top_k = top_k_accuracy(cls_score, labels, ...)
if label_smooth_eps !=0:
    if cls_score.size() != labels.size():
        labels = F.one_hot(labels)
    labels = ((1 - self.label_smooth_eps) * labels +
              self.label_smooth_eps / self.num_classes)
loss_cls = self.loss_cls(cls_score, labels, **kwargs)

Despite this, there should be at least a warning when the multi-class=False and the label_smooth_eps > 0 because the smoothing will be ignored in this case.

By the way, is that appropriate to apply the label smoothing on multi-label? The sum of soft label is no longer equal to the positive lables after the smoothing. For example. multi-label [1 1 0] becomes [0.933, 0.933, 0.033] when label_smooth_eps=0.1 and the resulting tensor's sum is 1.9 < 2.0.

[makecent]

From torch 1.10.0 on, the CrossEntropyLoss offcially supports inputing with soft labels (probabilities) and the lable smoothing.

[Dai-Wenxun]

@makecent, thanks for your advice, we will adopt your elegant modification. After the discussion, we now do not recommend to use label_smoothing in multi-class case. As for the advanced CrossEntropyLoss, we will not update our old implementation of CrossEntropyLoss for compatibility.