Qwen1.5-7B-Chat-GPTQ-Int4模型加载报错

[yhxuyou]

我在用快速使用的例子，加载Qwen1.5-7B-Chat-GPTQ-Int4模型时，会报 Traceback (most recent call last): File "test_qwen_1.5.py", line 4, in model = AutoModelForCausalLM.from_pretrained( File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained return model_class.from_pretrained( File "/opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py", line 3340, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /nfsc/pazb_cv_platform_id136211_vol1001_dev/pabj/xuyouzheng/xyz/code/uiex/qwen_model_1.5_7b_chat_int4.

但是加载Qwen1.5-7B-Chat-GPTQ-Int8模型是好的，可以正常推理，请问这是什么原因呢？