全参数微调的模型如何infer呢？

[HJT9328]

运行命令 RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=2 swift infer
--model_type chatglm2-6b
--model_id_or_path /data/LLM_checkpoint/chatglm2-6b/chatglm2-6b
--infer_backend vllm --tensor_parallel_size 1

报错，其中model_id_or_path是全参数微调的模型没有经过lora

[INFO:swift] Due to ckpt_dir being None, load_args_from_ckpt_dir is set to False. Traceback (most recent call last): File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/cli/infer.py", line 5, in infer_main() File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/run_utils.py", line 25, in x_main args, remaining_argv = parse_args(args_class, argv) File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/utils.py", line 98, in parse_args args, remaining_args = parser.parse_args_into_dataclasses( File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/transformers/hf_argparser.py", line 338, in parse_args_into_dataclasses obj = dtype(**inputs) File "", line 53, in init File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 649, in post_init set_model_type(self) File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 975, in set_model_type raise ValueError( ValueError: Please use --ckpt_dir vx-xxx/checkpoint-xxx to use the checkpoint.

[Jintao-Huang]

RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=2 swift infer
--ckpt_dir /data/LLM_checkpoint/chatglm2-6b/chatglm2-6b
--infer_backend vllm --tensor_parallel_size 1

[HJT9328]

RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=2 swift infer --ckpt_dir /data/LLM_checkpoint/chatglm2-6b/chatglm2-6b --infer_backend vllm --tensor_parallel_size 1

这个是对版本有要求吗？ ms-swift 1.7.3 我目前的版本执行这个必须要添加merge_rola参数

[Jintao-Huang]

哦 是需要添加的

[HJT9328]

但是全参数微调是没有使用到lora的，设置为true会报错

[Jintao-Huang]

要不把完整的报错发一下呗

然后升级一下swift, 对齐一下