loss don't go down when finetune and about requires_grad option in backward()

[graykode]

I tried finetuning with SS-Youtube dataset on examples/finetune_youtube_last. but There are some minor errors in there :(... First. The loss are not decrease well about each epochs when finetune using SS-Youtube. I tried training as setting original val loss 0.01, but Each epochs loss almost same

1 Epoch val acc 62.54 val loss 0.67718
2 Epoch val acc 62.54 val loss 0.67718
3 Epoch val acc 62.54 val loss 0.67718
...

Second. In finetune.py, There is error element 0 of tensors does not require grad and does not have a grad_fn if i not include loss = Variable(loss, requires_grad=True) in code.

[vladimirG123]

Same thing. But in TF version everything works fine

[tomgun132]

I just started to use Torchmoji and had the same problem. I tried adding loss = Variable(loss, requires_grad=True) like OP but the loss doesn't improve. Then I looked into the root of the problem why the element 0 of tensors does not require grad and does not have a grad_fn error appears. I noticed in model_def line 249-250 the code tries to get the tensor data from what was supposed to be Variable (if return_tensor: outputs = outputs.data). However since Variable has long been deprecated, I figured that this might be the one that removes the grad_fn from loss so I removed those lines of code then the loss starts improving when I finetuned the model.

[Ling312]

I just started to use Torchmoji and had the same problem. I tried adding loss = Variable(loss, requires_grad=True) like OP but the loss doesn't improve. Then I looked into the root of the problem why the element 0 of tensors does not require grad and does not have a grad_fn error appears. I noticed in model_def line 249-250 the code tries to get the tensor data from what was supposed to be Variable (if return_tensor: outputs = outputs.data). However since Variable has long been deprecated, I figured that this might be the one that removes the grad_fn from loss so I removed those lines of code then the loss starts improving when I finetuned the model.

works for me, I think it's caused by the different version of pytorch (Variables is deprecated from version 0.4)