mybigbug

https://pytorch.org/get-started/previous-versions/ version

> > Another question, can you guys (i mean authors) share the quantize scripts? we need the script after sft this model. > > > Another question, can you guys...

> 可以啊 > > 在finetune_.lora.sh 改成 MODEL="openbmb/MiniCPM-Llama3-V-2_5-int4" > > --tune_vision false --deepspeed ds_config_zero3.json > > 就可以了 @nickyisadog 我是使用finetune_ds.sh脚本不是lora脚本微调int-4模型得到了报错，ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters...

我已经使用命令 pip install -i https://pypi.org/simple/ bitsandbytes ，之后仍然报错

> 由于bitsandbytes (see [issse](https://github.com/TimDettmers/bitsandbytes/issues/252))现在还不支持MPS，int4模型暂时无法在mac运行。 感谢解答，等支持后再Close该issue

you try **set base_url=http://xxxxx/gateway/common/chat/completions** replace to **set base_url=http://xxxxx/gateway/common/** remove **chat/completions**