CUDA found but cublas is not loaded

[Hihoshi]

在跑python cli_demo.py llama的时候出问题

(stable_vicuna) PS G:\stable_vicuna\JittorLLMs> python cli_demo.py llama
[i 0603 18:44:57.225000 56 compiler.py:955] Jittor(1.3.7.16) src: d:\anaconda\lib\site-packages\jittor
[i 0603 18:44:57.242000 56 compiler.py:956] cl at G:\stable_vicuna\jittor_home\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe(19.29.30133)
[i 0603 18:44:57.242000 56 compiler.py:957] cache_path: G:\stable_vicuna\jittor_home\.cache\jittor\jt1.3.7\cl\py3.8.3\Windows-10-10.xe8\IntelRCoreTMi5xd3\default
[i 0603 18:44:57.245000 56 install_cuda.py:93] cuda_driver_version: [12, 1, 0]
[i 0603 18:44:57.275000 56 __init__.py:411] Found G:\stable_vicuna\jittor_home\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe(11.2.67) at G:\stable_vicuna\jittor_home\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe.
[i 0603 18:44:57.365000 56 __init__.py:411] Found gdb(8.1) at C:\mingw64\bin\gdb.EXE.
[i 0603 18:44:57.396000 56 __init__.py:411] Found addr2line(2.30) at C:\mingw64\bin\addr2line.EXE.
[i 0603 18:44:57.435000 56 compiler.py:1010] cuda key:cu11.2.67
[i 0603 18:44:57.437000 56 __init__.py:227] Total mem: 15.94GB, using 5 procs for compiling.
[i 0603 18:44:58.090000 56 jit_compiler.cc:28] Load cc_path: G:\stable_vicuna\jittor_home\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe
[i 0603 18:44:58.091000 56 init.cc:62] Found cuda archs: [86,]
[w 0603 18:44:58.275000 56 compile_extern.py:203] CUDA related path found in LD_LIBRARY_PATH or PATH(['', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\lib64', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\mkl\\dnnl_win_2.2.0_cpu_vcomp\\bin', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\mkl\\dnnl_win_2.2.0_cpu_vcomp\\lib', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\jt1.3.7\\cl\\py3.8.3\\Windows-10-10.xe8\\IntelRCoreTMi5xd3\\default', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\jt1.3.7\\cl\\py3.8.3\\Windows-10-10.xe8\\IntelRCoreTMi5xd3\\default\\cu11.2.67', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\bin', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\lib\\x64', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\msvc\\win10_kits\\lib\\ucrt\\x64', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\msvc\\win10_kits\\lib\\um\\x64', '', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\msvc\\VC\\lib', '', 'd', '\\anaconda\\libs', 'G', '\\stable_vicuna\\jittor_home\\.cache\\jittor\\msvc\\VC\\_\\_\\_\\_\\_\\bin', 'D', '\\Anaconda\\envs\\stable_vicuna', 'D', '\\Anaconda\\envs\\stable_vicuna\\Library\\mingw-w64\\bin', 'D', '\\Anaconda\\envs\\stable_vicuna\\Library\\usr\\bin', 'D', '\\Anaconda\\envs\\stable_vicuna\\Library\\bin', 'D', '\\Anaconda\\envs\\stable_vicuna\\Scripts', 'D', '\\Anaconda\\envs\\stable_vicuna\\bin', 'D', '\\Anaconda\\condabin', 'D', '\\Apps\\openjdk17\\bin', 'C', '\\Program Files\\java\\jdk-11\\bin', 'C', '\\ProgramData\\Oracle\\Java\\javapath', 'C', '\\Program Files (x86)\\Common Files\\Intel\\Shared Libraries\\redist\\intel64\\compiler', 'D', '\\Anaconda', 'D', '\\Anaconda\\Scripts', 'C', '\\WINDOWS\\system32', 'C', '\\WINDOWS', 'C', '\\WINDOWS\\System32\\Wbem', 'C', '\\WINDOWS\\System32\\WindowsPowerShell\\v1.0', 'C', '\\WINDOWS\\System32\\OpenSSH', 'C', '\\mingw64\\bin', 'C', '\\Program Files (x86)\\zlib123dllx64\\dll_x64', 'C', '\\Users\\Eternaldweller\\.cargo\\bin', 'C', '\\Program Files\\NVIDIA Corporation\\NVIDIA NvDLISR', 'C', '\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common', 'C', '\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\cudnn\\bin', 'C', '\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\cudnn\\lib', 'C', '\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\cudnn\\include', 'C', '\\Program Files\\Git\\cmd', 'C', '\\Program Files\\dotnet', 'D', '\\Apps\\neo4j-community-5.5.0\\bin', 'C', '\\Program Files\\NVIDIA Corporation\\Nsight Compute 2022.3.0', 'C', '\\Users\\Eternaldweller\\AppData\\Local\\Microsoft\\WindowsApps', 'D', '\\Apps\\openjdk17\\bin', 'D', '\\Apps\\openjdk17\\jre\\bin', 'C', '\\Program Files\\JetBrains\\IntelliJ IDEA 2020.2.3\\bin', 'C', '\\Program Files\\Azure Data Studio\\bin', 'D', '\\Apps\\Microsoft VS Code\\bin', 'C', '\\Users\\Eternaldweller\\.dotnet\\tools']), This path may cause jittor found the wrong libs, please unset LD_LIBRARY_PATH and remove cuda lib path in Path.
Or you can let jittor install cuda for you: `python3.x -m jittor_utils.install_cuda`
Traceback (most recent call last):
  File "D:\Anaconda\lib\site-packages\jittor\compile_extern.py", line 235, in setup_cuda_extern
    setup_cuda_lib(lib_name, extra_flags=link_cuda_extern)
  File "D:\Anaconda\lib\site-packages\jittor\compile_extern.py", line 312, in setup_cuda_lib
    culib = compile_custom_ops(culib_src_files, return_module=True,
  File "D:\Anaconda\lib\site-packages\jittor_utils\lock.py", line 59, in inner
    ret = func(*args, **kw)
  File "D:\Anaconda\lib\site-packages\jittor\compiler.py", line 765, in compile_custom_ops
    exec(f"import {gen_name}")
  File "<string>", line 1, in <module>
ImportError: DLL load failed while importing gen_ops_cublas_acc_matmul_cublas_batched_matmul_cu___hashe0fa5d: 找不到指定的程序。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "cli_demo.py", line 8, in <module>
    model = models.get_model(args)
  File "G:\stable_vicuna\JittorLLMs\models\__init__.py", line 38, in get_model
    globals()[f"get_{model_name}"]()
  File "G:\stable_vicuna\JittorLLMs\models\util.py", line 33, in get_llama
    new_paths.append(download_fromhub(f"jittorhub://{f}", tdir="llama"))
  File "G:\stable_vicuna\JittorLLMs\models\util.py", line 5, in download_fromhub
    import jittor as jt
  File "D:\Anaconda\lib\site-packages\jittor\__init__.py", line 25, in <module>
    from . import compile_extern
  File "D:\Anaconda\lib\site-packages\jittor\compile_extern.py", line 596, in <module>
    setup_cuda_extern()
  File "D:\Anaconda\lib\site-packages\jittor\compile_extern.py", line 247, in setup_cuda_extern
    LOG.f(msg)
  File "D:\Anaconda\lib\site-packages\jittor_utils\__init__.py", line 104, in f
    def f(self, *msg): self._log('f', 0, *msg)
  File "D:\Anaconda\lib\site-packages\jittor_utils\__init__.py", line 89, in _log
    cc.log(fileline, level, verbose, msg)
RuntimeError: [f 0603 18:44:58.382000 56 compile_extern.py:247] CUDA found but cublas is not loaded:

or you can let jittor install cuda and cudnn for you:
>>> python3.8 -m jittor_utils.install_cuda

我试过了移除环境变量，也试过更新版本，也试过python3.8 -m jittor_utils.install_cuda自动安装，都没用