TypeError: expected string or bytes-like object

[Energiz3r]

I'm on Windows 10, using conda, and following instructions from here https://github.com/oobabooga/text-generation-webui/wiki/LLaMA-model

Running python setup_cuda.py install from the cuda branch give this error TypeError: expected string or bytes-like object

I've got VS Build Tools 2019 installed with Desktop development with C++ workload, and on the right:

MSVC - VS 2019 C++ x64/x86 build tools (latest)
Windows 10 SDK
C++ CMake tools for Windows
Testing tools core features - Build Tools
C++ AddressSanitizer

To troubleshoot, I think I installed an alternative set of C++ compilers, but I'm not sure if this was useful or not: conda install compilers -c conda-forge

Full output:

No CUDA runtime is found, using CUDA_HOME='C:\Users\Tangles\miniconda3\envs\ooba\Library'
running install
C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing quant_cuda.egg-info\PKG-INFO
writing dependency_links to quant_cuda.egg-info\dependency_links.txt
writing top-level names to quant_cuda.egg-info\top_level.txt
reading manifest file 'quant_cuda.egg-info\SOURCES.txt'
writing manifest file 'quant_cuda.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_ext
Traceback (most recent call last):
  File "E:\text-generation-webui\repositories\GPTQ-for-LLaMa\setup_cuda.py", line 4, in <module>
    setup(
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\dist.py", line 1208, in run_command
    super().run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\install.py", line 74, in run
    self.do_egg_install()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\dist.py", line 1208, in run_command
    super().run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\dist.py", line 1208, in run_command
    super().run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\install_lib.py", line 11, in run
    self.build()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\command\install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\dist.py", line 1208, in run_command
    super().run_command(command)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\command\build_ext.py", line 84, in run
    _build_ext.run(self)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 346, in run
    self.build_extensions()
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\torch\utils\cpp_extension.py", line 499, in build_extensions
    _check_cuda_version(compiler_name, compiler_version)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\torch\utils\cpp_extension.py", line 383, in _check_cuda_version
    torch_cuda_version = packaging.version.parse(torch.version.cuda)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\pkg_resources\_vendor\packaging\version.py", line 49, in parse
    return Version(version)
  File "C:\Users\Tangles\miniconda3\envs\ooba\lib\site-packages\pkg_resources\_vendor\packaging\version.py", line 264, in __init__
    match = self._regex.search(version)
TypeError: expected string or bytes-like object

[Death-777]

I'm getting the same error, not exactly sure what to do either. Hopefully someone can figure it out.

[MaxDeVos]

I've had the same error on Windows 11 and a GTX 3080 TI, with a nearly identical log. The first line is telling us the problem: No CUDA runtime is found, using CUDA_HOME='C:\Users\<user>\miniconda3\envs\ooba\Library'

I was able to fix this by reinstalling CUDA and PyTorch with CUDA 11.7 (as opposed to CUDA 11.3.0/11.3.1) via the following commands:

conda remove -y pytorch; conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
conda remove -y cuda; conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit

Directly following this, I ran into another exception with a very similar stack trace: fatal error C1083: Cannot open include file: 'corecrt.h': No such file or directory quant_cuda_kernel.cu To save you the time of looking it up, this problem can be solved by installing the Windows 11 SDK in the Visual Studio Installer.

This worked flawlessly, and running the entire stack with CUDA 11.7 presented zero problems.