运行jittor.test.test_example报错

[ZongWei-HUST]

看起来是导入cuda出现了问题。CUDA found but cublas is not loaded. ImportError: /usr/local/cuda-11.1/lib64/libcublas.so.11: symbol free_gemm_select version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference. 采用python3.7 -m jittor_utils.install_cuda安装一遍仍然报同样的错误.

[i 0418 11:07:55.032550 48 compiler.py:951] Jittor(1.3.2.7) src: /home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor
[i 0418 11:07:55.037185 48 compiler.py:952] g++ at g++(7.5.0)
[i 0418 11:07:55.037262 48 compiler.py:953] cache_path: /home/user/.cache/jittor/jt1.3.2/g++7.5.0/py3.7.0/Linux-5.4.0-10xb5/IntelRCoreTMi9x9f/default
[i 0418 11:07:55.049304 48 install_cuda.py:53] cuda_driver_version: [11, 4]
[i 0418 11:07:55.054726 48 __init__.py:411] Found /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc(11.2.152) at /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc.
[i 0418 11:07:55.124027 48 __init__.py:411] Found gdb(8.1.1) at /usr/bin/gdb.
[i 0418 11:07:55.129579 48 __init__.py:411] Found addr2line(2.30) at /usr/bin/addr2line.
[i 0418 11:07:55.262538 48 compiler.py:1006] cuda key:cu11.2.152_sm_86
[i 0418 11:07:55.471185 48 __init__.py:227] Total mem: 125.58GB, using 16 procs for compiling.
[i 0418 11:07:55.563649 48 jit_compiler.cc:28] Load cc_path: g++
[i 0418 11:07:55.563675 48 jit_compiler.cc:31] Load nvcc_path: /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc
[i 0418 11:07:55.661321 48 init.cc:62] Found cuda archs: [86,]
[i 0418 11:07:55.765463 48 compile_extern.py:516] mpicc not found, distribution disabled.
[w 0418 11:07:55.790474 48 compile_extern.py:200] CUDA related path found in LD_LIBRARY_PATH or PATH(['/usr/local/cuda-11.1/lib64', '/home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64', '/usr/local/cuda-11.1/bin', '/home/user/.vscode-server/bin/dfd34e8260c270da74b5c2d86d61aee4b6d56977/bin/remote-cli', '/home/user/anaconda3/envs/zw_jt/bin', '/home/user/anaconda3/condabin', '/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/games', '/usr/local/games', '/snap/bin', '/usr/local/cuda-11.1/bin']), This path may cause jittor found the wrong libs, please unset LD_LIBRARY_PATH and remove cuda lib path in Path. 
[i 0418 11:07:55.814458 48 compile_extern.py:30] found /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include/cublas.h
[i 0418 11:07:55.823393 48 compile_extern.py:30] found /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64/libcublas.so
[i 0418 11:07:55.823487 48 compile_extern.py:30] found /home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64/libcublasLt.so.11
Traceback (most recent call last):
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/compile_extern.py", line 231, in setup_cuda_extern
    setup_cuda_lib(lib_name, extra_flags=link_cuda_extern)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/compile_extern.py", line 307, in setup_cuda_lib
    extra_flags=f" -I\"{jt_cuda_include}\" -I\"{jt_culib_include}\" {link_flags} {extra_flags} ")
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor_utils/lock.py", line 58, in inner
    ret = func(*args, **kw)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/compiler.py", line 761, in compile_custom_ops
    exec(f"import {gen_name}")
  File "<string>", line 1, in <module>
ImportError: /usr/local/cuda-11.1/lib64/libcublas.so.11: symbol free_gemm_select version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/__init__.py", line 25, in <module>
    from . import compile_extern
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/compile_extern.py", line 585, in <module>
    setup_cuda_extern()
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor/compile_extern.py", line 243, in setup_cuda_extern
    LOG.f(msg)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor_utils/__init__.py", line 104, in f
    def f(self, *msg): self._log('f', 0, *msg)
  File "/home/user/anaconda3/envs/zw_jt/lib/python3.7/site-packages/jittor_utils/__init__.py", line 89, in _log
    cc.log(fileline, level, verbose, msg)
RuntimeError: [f 0418 11:07:55.836160 48 compile_extern.py:243] CUDA found but cublas is not loaded:

or you can let jittor install cuda and cudnn for you:
>>>python3.7 -m jittor_utils.install_cuda

[Gword]

这是您的环境中包含了多个cuda，您可以参考这个warning把conda及cuda相关的PATH和LD_LIBRARY_PATH删除掉。

[ZongWei-HUST]

感谢您的回复！参考您的回答，我重新指定了当前环境下的LD_LIBRARY_PATH，将原来的/usr/local/cuda-11.1/lib64换成了/home/user/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64，可以通过jittor.test.test_example.