CasRelPyTorch icon indicating copy to clipboard operation
CasRelPyTorch copied to clipboard

Runtime error in backward

Open ridiculouz opened this issue 3 years ago • 1 comments

你好,在尝试使用这份代码时,我遇到了如下的bug: Traceback (most recent call last):
File "run.py", line 64, in trainer.train() File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 622, in train raise e File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 615, in train self._train() File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 683, in _train self._grad_backward(loss) File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 781, in _grad_backward loss.backward() File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor.py", line 255, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) File "/home/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/init.py", line 147, in backward Variable._execution_engine.run_backward( RuntimeError: Function MmBackward returned an invalid gradient at index 0 - got [712, 768] but expected shape compatible with [712, 21128] 我比较奇怪的是为什么forward没有报错而在backward报错了;以及想问下您,结果是在目前这个master版本下面跑出来的吗?有没有什么debug的头绪呢? 感谢!

environment: pytorch==1.9.0+cuda111 transformers==4.8.2 fastNLP==0.6.0

ridiculouz avatar Jul 23 '21 03:07 ridiculouz

我的版本能够运行,尝试打印一下tensor的维度。

Onion12138 avatar Jul 23 '21 03:07 Onion12138