Huatuo-Llama-Med-Chinese
Huatuo-Llama-Med-Chinese copied to clipboard
SCIR-HI
[BUG]运行infer.py cuda设置失败报错
请问一下，测试用的环境是什么啊。我是用的python3.10.11，安装完环境，cuda（版本为11.7）后运行infer.py
python infer.py --base_model 'decapoda-research/llama-7b-hf' --lora_weights './lora-llama-med' --use_lora True --instruct_dir './data/infer.json' --prompt_template 'med_template'会报如下错误：
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/Users/cqy/.conda/envs/huatuo_cuda117/lib'), WindowsPath('C')}
warn(msg)
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: C:\Users\cqy.conda\envs\huatuo_cuda117 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/127.0.0.1'), WindowsPath('7890'), WindowsPath('http')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
warn(msg)
C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...
warn(msg)
CUDA SETUP: Loading binary C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
File "E:\cqy-gpt\gpt-fine-tune\Huatuo-Llama-Med-Chinese\infer.py", line 8, in
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
请问这个问题要如何解决？感谢
