.\scripts\gui.bat reports error `Setting up PyTorch plugin "bias_act_plugin"... Failed!`

[hitzhangjie]

I used rtx 4090, and installed cudatoolkit11.1 and 11.5.

cudatoolkit 11.1, build bias_act_plugin failed.

(stylegan3) D:\AI\DragGAN>.\scripts\gui.bat
Loading "D:\AI\DragGAN\checkpoints\stylegan2_lions_512_pytorch.pkl"... Done.
()
{'z_dim': 512, 'c_dim': 0, 'w_dim': 512, 'img_resolution': 512, 'img_channels': 3, 'mapping_kwargs': {'num_layers': 8, 'embed_features': None, 'layer_features': None, 'activation': 'lrelu', 'lr_multiplier': 0.01, 'w_avg_beta': 0.995}, 'synthesis_kwargs': {'channel_base': 32768, 'channel_max': 512, 'num_fp16_res': 0, 'conv_clamp': None, 'architecture': 'skip', 'resample_filter': [1, 3, 3, 1], 'use_noise': True, 'activation': 'lrelu'}}
Setting up PyTorch plugin "bias_act_plugin"... Failed!

Traceback (most recent call last):
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 2100, in _run_ninja_build
    subprocess.run(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\AI\DragGAN\viz\renderer.py", line 112, in render
    self.init_network(res, **args)
  File "D:\AI\DragGAN\viz\renderer.py", line 249, in init_network
    w = G.mapping(z, label, truncation_psi=trunc_psi, truncation_cutoff=trunc_cutoff)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\AI\DragGAN\training\networks_stylegan2.py", line 247, in forward
    x = layer(x)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\AI\DragGAN\training\networks_stylegan2.py", line 125, in forward
    x = bias_act.bias_act(x, b, act=self.activation)
  File "D:\AI\DragGAN\torch_utils\ops\bias_act.py", line 84, in bias_act
    if impl == 'cuda' and x.device.type == 'cuda' and _init():
  File "D:\AI\DragGAN\torch_utils\ops\bias_act.py", line 41, in _init
    _plugin = custom_ops.get_plugin(
  File "D:\AI\DragGAN\torch_utils\custom_ops.py", line 136, in get_plugin
    torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1308, in load
    return _jit_compile(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1710, in _jit_compile
    _write_ninja_file_and_build_library(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1823, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 2116, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'bias_act_plugin': [1/3] C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\TH -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\THC "-IC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA\v11.1\include" -IC:\Users\Administrator\miniconda3\envs\stylegan3\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use_fast_math --allow-unsupported-compiler -c C:\Users\Administrator\AppData\Local\torch_extensions\torch_extensions\Cache\py39_cu121\bias_act_plugin\d197188c6eb43b2685156af41c1c4adc-nvidia-geforce-rtx-4090\bias_act.cu -o bias_act.cuda.o
FAILED: bias_act.cuda.o
C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\TH -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\THC "-IC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA\v11.1\include" -IC:\Users\Administrator\miniconda3\envs\stylegan3\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use_fast_math --allow-unsupported-compiler -c C:\Users\Administrator\AppData\Local\torch_extensions\torch_extensions\Cache\py39_cu121\bias_act_plugin\d197188c6eb43b2685156af41c1c4adc-nvidia-geforce-rtx-4090\bias_act.cu -o bias_act.cuda.o
CreateProcess failed: The system cannot find the file specified.
ninja: fatal: ReadFile: The handle is invalid.

[hitzhangjie]

then I change cudatoolkit11.1 to 11.5 (I installed this version too), it still failed:

.\scripts\gui.bat
Loading "D:\AI\DragGAN\checkpoints\stylegan2_lions_512_pytorch.pkl"... Done.
()
{'z_dim': 512, 'c_dim': 0, 'w_dim': 512, 'img_resolution': 512, 'img_channels': 3, 'mapping_kwargs': {'num_layers': 8, 'embed_features': None, 'layer_features': None, 'activation': 'lrelu', 'lr_multiplier': 0.01, 'w_avg_beta': 0.995}, 'synthesis_kwargs': {'channel_base': 32768, 'channel_max': 512, 'num_fp16_res': 0, 'conv_clamp': None, 'architecture': 'skip', 'resample_filter': [1, 3, 3, 1], 'use_noise': True, 'activation': 'lrelu'}}
Setting up PyTorch plugin "bias_act_plugin"... Failed!

Traceback (most recent call last):
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 2100, in _run_ninja_build
    subprocess.run(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\AI\DragGAN\viz\renderer.py", line 112, in render
    self.init_network(res, **args)
  File "D:\AI\DragGAN\viz\renderer.py", line 249, in init_network
    w = G.mapping(z, label, truncation_psi=trunc_psi, truncation_cutoff=trunc_cutoff)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\AI\DragGAN\training\networks_stylegan2.py", line 247, in forward
    x = layer(x)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\AI\DragGAN\training\networks_stylegan2.py", line 125, in forward
    x = bias_act.bias_act(x, b, act=self.activation)
  File "D:\AI\DragGAN\torch_utils\ops\bias_act.py", line 84, in bias_act
    if impl == 'cuda' and x.device.type == 'cuda' and _init():
  File "D:\AI\DragGAN\torch_utils\ops\bias_act.py", line 41, in _init
    _plugin = custom_ops.get_plugin(
  File "D:\AI\DragGAN\torch_utils\custom_ops.py", line 136, in get_plugin
    torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1308, in load
    return _jit_compile(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1710, in _jit_compile
    _write_ninja_file_and_build_library(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1823, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "C:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 2116, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'bias_act_plugin': [1/3] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\TH -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\include" -IC:\Users\Administrator\miniconda3\envs\stylegan3\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use_fast_math --allow-unsupported-compiler -c C:\Users\Administrator\AppData\Local\torch_extensions\torch_extensions\Cache\py39_cu121\bias_act_plugin\d197188c6eb43b2685156af41c1c4adc-nvidia-geforce-rtx-4090\bias_act.cu -o bias_act.cuda.o
FAILED: bias_act.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\TH -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\include" -IC:\Users\Administrator\miniconda3\envs\stylegan3\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use_fast_math --allow-unsupported-compiler -c C:\Users\Administrator\AppData\Local\torch_extensions\torch_extensions\Cache\py39_cu121\bias_act_plugin\d197188c6eb43b2685156af41c1c4adc-nvidia-geforce-rtx-4090\bias_act.cu -o bias_act.cuda.o
nvcc fatal   : Unsupported gpu architecture 'compute_89'
[2/3] cl /showIncludes -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\TH -IC:\Users\Administrator\miniconda3\envs\stylegan3\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\include" -IC:\Users\Administrator\miniconda3\envs\stylegan3\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /wd4624 /wd4067 /wd4068 /EHsc /std:c++17 -c C:\Users\Administrator\AppData\Local\torch_extensions\torch_extensions\Cache\py39_cu121\bias_act_plugin\d197188c6eb43b2685156af41c1c4adc-nvidia-geforce-rtx-4090\bias_act.cpp /Fobias_act.o
Microsoft (R) C/C++ Optimizing Compiler Version 19.37.32825 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

ninja: build stopped: subcommand failed.

[hitzhangjie]

Please help to solve this problem.

[hitzhangjie]

It's really awkward I spent nearly 2 ~ 3 days on this, and searched many similar problems and finally still cannot run it without errors. it's just ... a little hard to setup

[Vadim2S]

The Internet says what "nvcc fatal : Unsupported gpu architecture 'compute_89'" error means old nvcc version what do not known about new GPU like RTX 4090 yet. See: https://github.com/ggerganov/llama.cpp/issues/1420 Try install new CUDA toolkit and/or newest CUDA vesion. Can not check it, sorry. Do not have enough money for RTX 4090 :-)

[hitzhangjie]

Thanks very much, I will try again :)