Tim Gillis comments

Results 6 comments of


                                            Tim Gillis

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

I am getting a similar error: >dataset loading [0/3570] dataset loading [1000/3570] dataset loading [2000/3570] dataset loading [3000/3570] run train at epoch 1 THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THC/generated/../THCReduceAll.cuh line=339 error=59 : devic...

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

When I run the same commands above without CUDA I get this using hmdb51_2.json and hmdb51_3.json: >dataset loading [0/3570] dataset loading [1000/3570] dataset loading [2000/3570] dataset loading [3000/3570] run train...

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

I figured out the issue, get_labels() in hmdb51_json.py was adding an extra empty string label from the jsons generated from the previous folds. Adding the following at line 41 in...

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

@FelixZhang7 I was able to train it on a subset of the dataset, 600 training videos across 5 classes

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

@FelixZhang7 I'm not sure, I fine-tuned one of the pretrained models. This is what I ran: `python src/3D-ResNets-PyTorch/main.py --root_path src/3D-ResNets-PyTorch --video_path data/hmdb51/jpg-tmp/ --dataset hmdb51 --model resnet --model_depth 18 --resnet_shortcut A...

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generated/../THCReduceAll.cuh:339 terminate called after throwing an instance of 'std::runtime_error' what(): cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184

@FelixZhang7 you're going to have to modify the pre-processing scripts to work with your dataset.