RuntimeError: CUDA error: unrecognized error code

[charmeleonz]

Hi,

I ran into the following error when running bash train.sh:

Traceback (most recent call last): File "tools/train_val.py", line 113, in main() File "tools/train_val.py", line 55, in main model = model.to(device) File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 612, in to return self._apply(convert) File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) [Previous line repeated 3 more times] File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 381, in _apply param_applied = fn(param) File "/home/user/.conda/envs/monodetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 610, in convert return t.to(device, dtype if t.is_floating_point() else None, non_blocking) RuntimeError: CUDA error: unrecognized error code

Any idea why this is happening? My environment is: python=3.8.16, torch=1.7.1+cu110, torchvisioon=0.8.2+cu110, cudatoolkit=11.5.1.

[MarriedSenior]

did you fix it? give me idea too bro