pytorch-video-recognition icon indicating copy to clipboard operation
pytorch-video-recognition copied to clipboard

C3D training from scratch met RuntimeError: CUDNN_STATUS_EXECUTION_FAILED

Open Magsun opened this issue 5 years ago • 3 comments
trafficstars

Hello @jfzhang95 , thanks for your code firstly.

I'm trying to train C3D from scratch on my own ucf101 style dataset.

I changed ucf101 config from 101 to 2 & num_workers=1 in train.py and dataset path in mypath.py, except mentioned above I didn't change any other settings.

When I run 'python train.py', I got this runtime error and don't know what happened.

Traceback (most recent call last): File "C:/Users/google/Desktop/pytorch-video-recognition-master/train.py", line 203, in train_model() File "C:/Users/google/Desktop/pytorch-video-recognition-master/train.py", line 131, in train_model outputs = model(inputs) File "D:\Anaconda3\envs\video\lib\site-packages\torch\nn\modules\module.py", line 491, in call result = self.forward(*input, **kwargs) File "C:\Users\google\Desktop\pytorch-video-recognition-master\network\C3D_model.py", line 46, in forward x = self.relu(self.conv1(x)) File "D:\Anaconda3\envs\video\lib\site-packages\torch\nn\modules\module.py", line 491, in call result = self.forward(*input, **kwargs) File "D:\Anaconda3\envs\video\lib\site-packages\torch\nn\modules\conv.py", line 421, in forward self.padding, self.dilation, self.groups) RuntimeError: CUDNN_STATUS_EXECUTION_FAILED


The env is win10 cuda9 torch0.4.0, I'm not sure if I should run this under linux.

Thanks if anyone can help.

Magsun avatar Apr 15 '20 07:04 Magsun

cudann 7.6.1

Magsun avatar Apr 15 '20 08:04 Magsun

Hi magsun, Hope you have solved the error, but if not this might help you. This error usually occurs when there is compatibility issue with the driver or with the cudnn and the pytorch. Please refer here : https://discuss.pytorch.org/t/cudnn-error-cudnn-status-mapping-error/69079 for me by adding the line torch.backends.cudnn.enabled = False worked.

Thanks.

vaibhav0195 avatar May 21 '20 10:05 vaibhav0195

Hi magsun, Hope you have solved the error, but if not this might help you. This error usually occurs when there is compatibility issue with the driver or with the cudnn and the pytorch. Please refer here : https://discuss.pytorch.org/t/cudnn-error-cudnn-status-mapping-error/69079 for me by adding the line torch.backends.cudnn.enabled = False worked.

Thanks.

Oh, thanks for your help, I tried another version of pytorch and it works.

I just use torch=0.4.1 instead of 0.4.0. I'll try it your way.

Thanks again!

Magsun avatar May 27 '20 06:05 Magsun