Some problems when training

[by2101]

Thank you for your transformer code! When I ran the code, I encountered such issue:

Traceback (most recent call last): File "train.py", line 539, in train() File "train.py", line 451, in train loss.backward() File "/data1/baiye/miniconda3/envs/torch04py3/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/data1/baiye/miniconda3/envs/torch04py3/lib/python3.6/site-packages/torch/autograd/init.py", line 89, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

My enviroment is pytorch 0.4. And I checked the code and did not find any inplace operation.

[kimiyoung]

Could you please provide more information about how you ran the code? If you use pytorch 0.4.1 and run it as is, it should be ok.

[goodbai-nlp]

I met the same problem and solved it by upgrading pytorch from 0.4.0 to 0.4.1.