Volumetric-Aggregation-Transformer icon indicating copy to clipboard operation
Volumetric-Aggregation-Transformer copied to clipboard

RuntimeError: one of the variables needed for gradient computation

Open SirojbekSafarov opened this issue 2 years ago • 2 comments

Hello.

Thank you for this amazing work.

I want to retrain your model, but when I traid I got this error. Do you have any experience with this error?

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2, 128, 32, 32, 8, 8]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

SirojbekSafarov avatar Apr 22 '22 07:04 SirojbekSafarov

I am facing the same issue. Any solution ?

Ehteshamciitwah avatar Jul 26 '22 06:07 Ehteshamciitwah

I found one solution. we need to change the inplace operation to non -inplace operations. There was some operation in ours.py file. i re-wrote the codes of lines. image

Ehteshamciitwah avatar Jul 27 '22 04:07 Ehteshamciitwah