Issue of ablation study performance

[Cascol-Chen]

I am verifying the effect of pairwise cross entropy, and implement it with the following code:

class SoftCrossEntropyLoss(nn.Module):
   def __init__(self):
      super().__init__()

   def forward(self, y_hat, y):
      p = F.log_softmax(y_hat, 0)
      loss = -(y*p).sum() / (y).sum()
      return loss

softce = SoftCrossEntropyLoss().cuda()
   
def pairwise_entropy(outputs):
    logits = outputs - outputs.logsumexp(dim=-1, keepdim=True)
    loss, B = 0, outputs.shape[0]
    for i in range(B):
        for j in range(B):
            if i == j: continue
            loss += softce(outputs[i], outputs[j])
    loss /= B * (B-1)
    return loss, logits

However, such implementation cannot achieve performance gain in level 5 gaussian noise compared with original model. Could you kindly provide the code of pairwise cross entropy in the ablation study?