could not compile

[derVedro]

I run into famous "name: 'str2optimizer8bit_blockwise' is not defined" issue with 0.35.4, 0.36.0.pos2 and 0.37.2 bitsandbytes installed via pip in venv on my ubuntu 22.10 system. nvidia-cuda-toolkit ubuntu package is installed from repositories and some another libcudart.so are well placed in the same venv. Somehow it seems impossible for bitsandbytes and CUDA to become friends.

For example, if I use kohya-ss scripts, that depends on bitsandbytes, bitsandbytes crying with user warning "...installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.", so I tried to compile it manually. But it comes not so easy.

I checkout 49a0425 and first I tried to follow compile_from_source.md guide, set CUDA_VERSION=115 and then make cuda11x. I get this:

me@beast:/tmp/bis/bitsandbytes/$ make cuda11x
ENVIRONMENT
============================
CUDA_VERSION: 115
============================
NVCC path: /usr/bin/nvcc
GPP path: /usr/bin/g++ VERSION: g++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CUDA_HOME: /usr
CONDA_PREFIX: 
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/me/.local/bin://home/me/.cargo/bin:/tmp/bis/bitsandbytes/cuda/bin
LD_LIBRARY_PATH: /tmp/bis/bitsandbytes/cuda/lib64
============================
/usr/bin/nvcc -gencode arch=compute_75,code=sm_75 -Xcompiler '-fPIC' --use_fast_math -Xptxas=-v -dc /tmp/bis/bitsandbytes/csrc/ops.cu /tmp/bis/bitsandbytes/csrc/kernels.cu -I /usr/include -I /tmp/bis/bitsandbytes/csrc -I /include -I /tmp/bis/bitsandbytes/include -L /usr/lib64 -lcudart -lcublas -lcublasLt -lcurand -lcusparse -L /lib --output-directory /tmp/bis/bitsandbytes/build
ptxas info    : 11 bytes gmem
ptxas info    : Compiling entry function '_ZN3cub11EmptyKernelIvEEvv' for 'sm_75'
ptxas info    : Function properties for _ZN3cub11EmptyKernelIvEEvv
    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info    : Used 4 registers, 352 bytes cmem[0]
/usr/include/c++/10/type_traits:71:52: error: redefinition of ‘constexpr const _Tp std::integral_constant<_Tp, __v>::value’
   71 |   template<typename _Tp, _Tp __v>
      |                                                    ^                           
/usr/include/c++/10/type_traits:59:29: note: ‘constexpr const _Tp value’ previously declared here
   59 |       static constexpr _Tp                  value = __v;
      |                             ^~~~~
make: *** [Makefile:63: all] Fehler 1

Hmmm, then I tried to get CUDA locally. Install it like in cuda_install.sh to cuda directory and with --no-drm --no-man-page --override --toolkitpath=$PWD/cuda/ --toolkit --silent flags, exported LD_LIBRARY_PATH and PATH, then try to build and get that:

me@beast:/tmp/bis/bitsandbytes/$ make cuda11x
ENVIRONMENT
============================
CUDA_VERSION: 115
============================
NVCC path: /tmp/bis/bitsandbytes/cuda/bin/nvcc
GPP path: /usr/bin/g++ VERSION: g++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CUDA_HOME: /tmp/bis/bitsandbytes/cuda
CONDA_PREFIX: 
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/me/.local/bin://home/me/.cargo/bin:/tmp/bis/bitsandbytes/cuda/bin
LD_LIBRARY_PATH: /tmp/bis/bitsandbytes/cuda/lib64
============================
/tmp/bis/bitsandbytes/cuda/bin/nvcc -gencode arch=compute_75,code=sm_75 -Xcompiler '-fPIC' --use_fast_math -Xptxas=-v -dc /tmp/bis/bitsandbytes/csrc/ops.cu /tmp/bis/bitsandbytes/csrc/kernels.cu -I /tmp/bis/bitsandbytes/cuda/include -I /tmp/bis/bitsandbytes/csrc -I /include -I /tmp/bis/bitsandbytes/include -L /tmp/bis/bitsandbytes/cuda/lib64 -lcudart -lcublas -lcublasLt -lcurand -lcusparse -L /lib --output-directory /tmp/bis/bitsandbytes/build
In file included from /tmp/bis/bitsandbytes/cuda/include/cuda_runtime.h:83,
                 from <command-line>:
/tmp/bis/bitsandbytes/cuda/include/crt/host_config.h:139:2: error: #error -- unsupported GNU version! gcc versions later than 11 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
  139 | #error -- unsupported GNU version! gcc versions later than 11 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
      |  ^~~~~
make: *** [Makefile:63: all] Fehler 1

What I am missing?

Additional info:

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

[bghira]

roughly same issue here:

ENVIRONMENT
============================
CUDA_VERSION: 118
============================
NVCC path: /bin/nvcc
GPP path: /usr/bin/g++ VERSION: g++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CUDA_HOME: 
CONDA_PREFIX: 
PATH: /home/bghira/src/discord-tron-client/.venv/bin:/home/bghira/miniconda3/condabin:/snap/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/libexec:/usr/local/libexec:/usr/local/samba/sbin:/usr/local/samba/bin:/usr/games:/opt/bin:/home/bghira/.local/bin
LD_LIBRARY_PATH: 
============================
/bin/nvcc -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -Xcompiler '-fPIC' --use_fast_math -Xptxas=-v -dc /home/bghira/src/bitsandbytes/csrc/ops.cu /home/bghira/src/bitsandbytes/csrc/kernels.cu -I /include -I /home/bghira/src/bitsandbytes/csrc -I /include -I /home/bghira/src/bitsandbytes/include -L /lib64 -lcudart -lcublas -lcublasLt -lcusparse -L /lib --output-directory /home/bghira/src/bitsandbytes/build
ptxas info    : 14 bytes gmem
ptxas info    : Compiling entry function '_ZN3cub11EmptyKernelIvEEvv' for 'sm_75'
ptxas info    : Function properties for _ZN3cub11EmptyKernelIvEEvv
    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info    : Used 4 registers, 352 bytes cmem[0]
ptxas info    : 14 bytes gmem
ptxas info    : Compiling entry function '_ZN3cub11EmptyKernelIvEEvv' for 'sm_80'
ptxas info    : Function properties for _ZN3cub11EmptyKernelIvEEvv
    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info    : Used 4 registers, 352 bytes cmem[0]
ptxas info    : 14 bytes gmem
ptxas info    : Compiling entry function '_ZN3cub11EmptyKernelIvEEvv' for 'sm_86'
ptxas info    : Function properties for _ZN3cub11EmptyKernelIvEEvv
    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info    : Used 4 registers, 352 bytes cmem[0]
/usr/include/c++/10/type_traits:71:52: error: redefinition of ‘constexpr const _Tp std::integral_constant<_Tp, __v>::value’
   71 |   template<typename _Tp, _Tp __v>
      |                                                    ^                           
/usr/include/c++/10/type_traits:59:29: note: ‘constexpr const _Tp value’ previously declared here
   59 |       static constexpr _Tp                  value = __v;
      |                             ^~~~~
make: *** [Makefile:83: cuda11x] Error 1

[dbelenko]

Something along the lines of https://stackoverflow.com/questions/6622454/cuda-incompatible-with-my-gcc-version helps. You can have more than one version of GCC on the system.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[bghira]

nope, mine didn't have the same GCC version error. it does seem like some problem to bitsandbytes, but only insofar as it won't detect the issue and give a proper error message. i don't expect this project to fix the host library issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.