RTX 3080 Error sm_86 is not compatible

[byungguan-yoon]

Hello guys! I'm trying to serve my face detection model using torchserve docker.

I Installed Dockerfile which is working well on RTX 2080, But it is not working on RTX 3080. I changed cuda version to 11.3 and I can see pytorch version is torch==1.10.1+cu113.

I found tmp/model/{encrypted}/torch/version.py has cuda 10.2. It seems while model archiver compile my pytorch to mar file, something went wrong. I want to know what tmp folder really store....

How can I successfully serve on RTX 3080??? Please save me guys Thanks!!!

[msaroufim]

Hi @byungguan-yoon could you please try cuda 11.0 or 11.1 instead?

See this thread for more info on Pytorch on RTX 3080 https://discuss.pytorch.org/t/rtx-3070-3080-support/104895

I'd just double check Pytorch installs correctly by itself before also adding torchserve to the mix

[byungguan-yoon]

Thanks for your reply @msaroufim ! I tried cuda 11.0 / 11.1 both, still same warning ocurred. I've check pytorch version by python interactive shell

import torch
print("Pytorch version：")
print(torch.__version__)
print("CUDA Version: ")
print(torch.version.cuda)
print("cuDNN version is :")
print(torch.backends.cudnn.version())
print("Arch version is :")
print(torch._C._cuda_getArchFlags())

And I checked cuda version by cat /usr/local/cuda

I did this same in cuda 11.0 but same warning/error message popped up.

[chauhang]

@byungguan-yoon This type of problem generally happens due to Nvidia driver versions mismatch. Please share output of this command from your RTX 3080 python -m torch.utils.collect_env

[Gabibing]

I'm having trouble with same problem. I built the image by cu113 production build. but I got same error

NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

result python -m torch.utils.collect_env below

python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.10.2+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Home Insider Preview
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.25126-SP0
Is CUDA available: True
CUDA runtime version: 11.4.152
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 512.52
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.3
[pip3] numpydoc==1.1.0
[pip3] torch==1.10.2+cu113
[pip3] torchaudio==0.10.2+cu113
[pip3] torchvision==0.11.3+cu113
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] mypy_extensions           0.4.3            py39haa95532_0
[conda] numpy                     1.20.3           py39ha4e8547_0
[conda] numpy-base                1.20.3           py39hc2deb75_0
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1
[conda] torch                     1.10.2+cu113             pypi_0    pypi
[conda] torchaudio                0.10.2+cu113             pypi_0    pypi
[conda] torchvision               0.11.3+cu113             pypi_0    pypi

[ozancaglayan]

is this actually an issue related to torchserve or rather to torch packages?

[Gabibing]

I think it's related to torchserve because it doesn't happen with torch only

[AI-P-K]

Anyone managed to run Torchserve on RTX3070 ?