合并13B模型时报错

[jayecho51]

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

CUDA SETUP: CUDA runtime path found: /home/sd/miniconda3/envs/textgen1/lib/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 7.5 CUDA SETUP: Detected CUDA version 117 CUDA SETUP: Loading binary /home/sd/miniconda3/envs/textgen1/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so... Loading checkpoint shards: 100%|██████████████████████████████████████████| 3/3 [03:19<00:00, 66.39s/it] Traceback (most recent call last): File "/home/sd/cctext2023/chinese-LLaMA/Chinese-LLaMA-Alpaca/scripts/merge_llama_with_chinese_lora.py", line 41, in base_model = LlamaForCausalLM.from_pretrained( File "/home/sd/miniconda3/envs/textgen1/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2787, in from_pretrained dispatch_model(model, device_map=device_map, offload_dir=offload_folder, offload_index=offload_index) TypeError: dispatch_model() got an unexpected keyword argument 'offload_index' 指定了offload_dir，猜测是不是机器内存不够导致的，机器内存32g

[ymcui]

从你的log中看似是把GPU也用上了。 转13B模型（使用offload）的话RAM需要26-27G（也就是模型本身大小）。 我们的colab notebook转换是没出现这个问题的，不知道是不是你的机器可用内存不够导致的。

[jayecho51]

32g机器按理说够了啊- -

[airaria]

如果内存够，可以试试不指定offload_dir； 此外，请升级accelerate版本再试试

[Sped0n0ld]

我这边有offload大概29G，没有38G

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

[github-actions[bot]]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.

[iamhere1]

我也遇到这个问题，通过 pip install accelerate==0.18.0 解决了