ff1Zzd

torch._C._LinAlgError: linalg.cholesky 即使讲quant_n_samples提到了2048依然报错

1

**Describe the bug** What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) `CUDA_VISIBLE_DEVICES=0 swift export --ckpt_dir 'output/qwen2-7b-instruct/v2-20240731-102444/checkpoint-30' --merge_lora true --quant_bits 4 --dataset alpaca-zh alpaca-en --quant_method gptq --quant_n_samples=2048` 使用这个命令量化微调后的模型时报错...

qwen2.5 vllm engine fail to load on multi-gpu cards

1

**Describe the bug** What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) I am using `get_vllm_engine` to load qwen2.5 72b int4, and I have specified using multiple cuda...