用测试数据训练时bitsandbytes报的错，有大佬知道是什么回事吗

[Thzny]

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

/opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/nvidia/lib')} warn(msg) /opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths... warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... /opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')} warn(msg) /opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)! warn(msg) CUDA SETUP: Highest compute capability among GPUs detected: 7.5 CUDA SETUP: Detected CUDA version 116 CUDA SETUP: Loading binary /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so... /opt/conda/lib/python3.10/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored. Overriding torch_dtype=None with torch_dtype=torch.float16 due to requirements of bitsandbytes to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning. Loading checkpoint shards: 0%| | 0/8 [00:01<?, ?it/s] Traceback (most recent call last): File "/home/python/finetune.py", line 162, in main() File "/home/python/finetune.py", line 121, in main model = ChatGLMForConditionalGeneration.from_pretrained( File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2646, in from_pretrained ) = cls._load_pretrained_model( File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2969, in _load_pretrained_model new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model( File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 676, in _load_state_dict_into_meta_model set_module_8bit_tensor_to_device(model, param_name, param_device, value=param) File "/opt/conda/lib/python3.10/site-packages/transformers/utils/bitsandbytes.py", line 70, in set_module_8bit_tensor_to_device new_value = bnb.nn.Int8Params(new_value, requires_grad=False, has_fp16_weights=has_fp16_weights).to(device) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 196, in to return self.cuda(device) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 160, in cuda CB, CBt, SCB, SCBt, coo_tensorB = bnb.functional.double_quant(B) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1616, in double_quant row_stats, col_stats, nnz_row_ptr = get_colrow_absmax( File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1505, in get_colrow_absmax lib.cget_col_row_stats(ptrA, ptrRowStats, ptrColStats, ptrNnzrows, ct.c_float(threshold), rows, cols) File "/opt/conda/lib/python3.10/ctypes/init.py", line 387, in getattr func = self.getitem(name) File "/opt/conda/lib/python3.10/ctypes/init.py", line 392, in getitem func = self._FuncPtr((name_or_ordinal, self)) AttributeError: /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cget_col_row_stats

[jiangliqin]

我也遇到了，参考这个bitsandbytes的issuehttps://github.com/TimDettmers/bitsandbytes/issues/156还没解决

[Starrylun]

我也遇到了这个问题，在官方文档上发现 ：The bitsandbytes library is currently only supported on Linux distributions. Windows is not supported at the moment. 因此 bitsandbytes 的优化方式无法在win系统上使用，至今没有解决。换成 linux 系统后应该就可以了。如果有在 win 系统上处理过的大佬，麻烦解答一下。

[SmileSmith]

https://github.com/TimDettmers/bitsandbytes/issues/156

我是windows11，cuda降级到11.6，参考这个issue解决了（我不会c++，只好用别人编译好的116版本dll） https://github.com/TimDettmers/bitsandbytes/issues/30#issuecomment-1257676341