RuntimeError: CUDA error: no kernel image is available for execution on the device

[XiaoMing7867]

I have checked that the mapping between my cuda version and torch version is correct, and that the mmcv and mmdet versions are set according to the environment of the author of the source code. Why is there still a RuntimeError: CUDA error: no kernel image is available for execution on the device. My computer graphics card is 3060, and the computing power is sufficientRuntimeError: CUDA error: no kernel image is available for execution on the device my environment： sys.platform: win32 Python: 3.7.16 (default, Jan 17 2023, 16:06:28) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1 NVCC: Not Available GCC: n/a PyTorch: 1.8.1+cu111 PyTorch compiling details: PyTorch built with:

• C++ Version: 199711
• MSVC 192829913
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 2019
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.0.5
• Magma 2.5.4
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=C:/w/b/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

TorchVision: 0.9.1+cu111 OpenCV: 4.7.0 MMCV: 1.4.0 MMCV Compiler: MSVC 192930137 MMCV CUDA Compiler: 11.1 MMDetection: 2.17.0+

[mousecpn]

It's weird. Maybe you should try on some other examples like mnist, and see whether the problem maintains.

[mousecpn]

I have checked that the mapping between my cuda version and torch version is correct, and that the mmcv and mmdet versions are set according to the environment of the author of the source code. Why is there still a RuntimeError: CUDA error: no kernel image is available for execution on the device. My computer graphics card is 3060, and the computing power is sufficientRuntimeError: CUDA error: no kernel image is available for execution on the device my environment： sys.platform: win32 Python: 3.7.16 (default, Jan 17 2023, 16:06:28) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1 NVCC: Not Available GCC: n/a PyTorch: 1.8.1+cu111 PyTorch compiling details: PyTorch built with:

• C++ Version: 199711
• MSVC 192829913
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 2019
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.0.5
• Magma 2.5.4
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=C:/w/b/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

TorchVision: 0.9.1+cu111 OpenCV: 4.7.0 MMCV: 1.4.0 MMCV Compiler: MSVC 192930137 MMCV CUDA Compiler: 11.1 MMDetection: 2.17.0+

I notice your NVCC is not available. I think you should install some dependencies.

[XiaoMing7867]

I have checked that the mapping between my cuda version and torch version is correct, and that the mmcv and mmdet versions are set according to the environment of the author of the source code. Why is there still a RuntimeError: CUDA error: no kernel image is available for execution on the device. My computer graphics card is 3060, and the computing power is sufficientRuntimeError: CUDA error: no kernel image is available for execution on the device my environment： sys.platform: win32 Python: 3.7.16 (default, Jan 17 2023, 16:06:28) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1 NVCC: Not Available GCC: n/a PyTorch: 1.8.1+cu111 PyTorch compiling details: PyTorch built with:

• C++ Version: 199711
• MSVC 192829913
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 2019
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.0.5
• Magma 2.5.4
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=C:/w/b/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

TorchVision: 0.9.1+cu111 OpenCV: 4.7.0 MMCV: 1.4.0 MMCV Compiler: MSVC 192930137 MMCV CUDA Compiler: 11.1 MMDetection: 2.17.0+

I notice your NVCC is not available. I think you should install some dependencies.

If I type nvcc -V in cmd, it can be normally displayed. The displayed content is as follows: (mmdet) PS D:\Software\deep_learning\underwater_object_detection\Boosting-R-CNN-masternew> nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Mon_Oct_12_20:54:10_Pacific_Daylight_Time_2020 Cuda compilation tools, release 11.1, V11.1.105 Build cuda_11.1.relgpu_drvr455TC455_06.29190527_0

I wonder if it's because I didn't install visual studio, or what dependencies do I need to install for "nvcc is not avaliable"? Went to the Internet to find relevant information but did not get a solution. It's so much trouble for you！！！

[mousecpn]

Can it work on other pytorch examples like mnist with GPU?

[XiaoMing7867]

Can it work on other pytorch examples like mnist with GPU? This is a new environment I created specifically to run this code. I used this environment to run the mmdet demo program and also show cuda error, but I used another environment, cuda version is 11.3, torch1.12.1, mmcv1.7.1, mmdet2.28, When running demo program can run smoothly, I have been a little confused, thank you so patient answer!!

[mousecpn]

So it works?

[XiaoMing7867]

So it works?

It also runs with the same error: cuda error

[mousecpn]

So it works?

It also runs with the same error: cuda error

Try your environment on mnist program.

[XiaoMing7867]

So it works?

It also runs with the same error: cuda error

Try your environment on mnist program.

I tested it on the official pytorch implementation of mnist code, and it ran properly without error Train Epoch: 14 [58880/60000 (98%)] Loss: 0.009802 Train Epoch: 14 [59520/60000 (99%)] Loss: 0.002173

Test set: Average loss: 0.0260, Accuracy: 9920/10000 (99%)

[mousecpn]

So it works?

It also runs with the same error: cuda error

Try your environment on mnist program.

I tested it on the official pytorch implementation of mnist code, and it ran properly without error Train Epoch: 14 [58880/60000 (98%)] Loss: 0.009802 Train Epoch: 14 [59520/60000 (99%)] Loss: 0.002173

Test set: Average loss: 0.0260, Accuracy: 9920/10000 (99%)

on GPU？

[XiaoMing7867]

yes！

[mousecpn]

It is really weird. Maybe you can set some breakpoint and debug my code line by line to see which line causes the error. It would be clear to find the solutions.

[XiaoMing7867]

It is really weird. Maybe you can set some breakpoint and debug my code line by line to see which line causes the error. It would be clear to find the solutions.

Ok, I will try again, thank you very much for your patient answer!! I'll share the solution if I can solve it, but I don't really believe in my own ability. hahahahahaha

[XiaoMing7867]

It is really weird. Maybe you can set some breakpoint and debug my code line by line to see which line causes the error. It would be clear to find the solutions.

A problem occurred to me. My computer runs windows and I find that other people seem to install the environment more smoothly in linux. I wonder if it is a problem that the compatibility with windows system is still insufficient

[mousecpn]

Yup. Windows is not possible, which is warned in the mmdet github page.