Why gradient does not scale linearly with CTCloss in pytorch binding?

[lifelongeek]

I am pytorch user, and I am curious this issue is generally occurs in every platform of CTC. Originally, this issue is posted in https://github.com/SeanNaren/deepspeech.pytorch/issues/250.

I am trying to use CTCLoss with another loss, and want coefficients of CTCLoss smaller. But when I tried different

This is the extreme cases for small CTCLoss. (CTCLoss coefficient = 0)

criterion = CTCLoss() loss = 0*criterion(out, targets, sizes, target_sizes) model.zero_grad() loss.backward() optimizer.step()

for param in model.parameters(): if param.grad is not None: grad_norm += param.grad.norm().data[0] In above example, loss is always 0. However, fairly large gradient value still exists even CTCLoss coefficient is 0.

Can anyone explain why gradient does not scale linearly with CTCLoss?

[weedwind]

@gmkim90

Hi, I found this problem too. Did you solve it?

Thank you very much.

[lifelongeek]

No. still does not understand the reason.