volo icon indicating copy to clipboard operation
volo copied to clipboard

SystemError: returned NULL without setting an error

Open JIAOJIAYUASD opened this issue 3 years ago • 0 comments

When I run the code of volo, I get a debug error as follows: /pytorch/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:312: operator(): block: [0,0,0], thread: [6,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:312: operator(): block: [0,0,0], thread: [40,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:312: operator(): block: [0,0,0], thread: [1,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed. Traceback (most recent call last): File "main.py", line 951, in main() File "main.py", line 665, in main optimizers=optimizers) File "main.py", line 800, in train_one_epoch create_graph=second_order) File "/opt/conda/lib/python3.6/site-packages/timm/utils/cuda.py", line 22, in call scaled_loss.backward(create_graph=create_graph) File "/opt/conda/lib/python3.6/site-packages/torch/tensor.py", line 221, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/opt/conda/lib/python3.6/site-packages/torch/autograd/init.py", line 132, in backward allow_unreachable=True) # allow_unreachable flag SystemError: <built-in method run_backward of torch._C._EngineBase object at 0x7f9d92671420> returned NULL without setting an error

Can you give me some advice to fix it?

JIAOJIAYUASD avatar Aug 19 '21 03:08 JIAOJIAYUASD