about the forward loss function problem

[sillytheif]

Hi, I have some problem about your Forward loss function: def forward_loss(output, target, trans_mat, index = None): # l_{forward}(y, h(x)) = l_{ce}(y, h(x) @ T) outputs = F.softmax(output, dim=1) if index is None: outputs = outputs @ trans_mat else: outputs1 = outputs.view(outputs.shape[0] * outputs.shape[1],-1).repeat(1,outputs.shape[1]) T1 = trans_mat[index].view(outputs.shape[0] * trans_mat.shape[1], trans_mat.shape[2]) outputs = torch.sum((outputs1 * T1).view(outputs.shape[0],trans_mat.shape[1],trans_mat.shape[2]),1)

outputs = torch.log(outputs)
#loss = CE(outputs, target)
loss = F.cross_entropy(outputs,target)

return loss

the input of F.cross_entropy, should be the logits before log softmax. Thus you should use the F.nll_loss(outputs, target) here

[zwzhu-d]

Thank you for the good catching. Yes, the output is taken log twice. I comment out this Line # outputs = torch.log(outputs) now.