swift icon indicating copy to clipboard operation
swift copied to clipboard

全参数微调的模型如何infer呢?

Open HJT9328 opened this issue 2 months ago • 5 comments

运行命令 RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=2 swift infer
--model_type chatglm2-6b
--model_id_or_path /data/LLM_checkpoint/chatglm2-6b/chatglm2-6b
--infer_backend vllm --tensor_parallel_size 1

报错,其中model_id_or_path是全参数微调的模型没有经过lora

[INFO:swift] Due to ckpt_dir being None, load_args_from_ckpt_dir is set to False. Traceback (most recent call last): File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/cli/infer.py", line 5, in infer_main() File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/run_utils.py", line 25, in x_main args, remaining_argv = parse_args(args_class, argv) File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/utils.py", line 98, in parse_args args, remaining_args = parser.parse_args_into_dataclasses( File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/transformers/hf_argparser.py", line 338, in parse_args_into_dataclasses obj = dtype(**inputs) File "", line 53, in init File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 649, in post_init set_model_type(self) File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 975, in set_model_type raise ValueError( ValueError: Please use --ckpt_dir vx-xxx/checkpoint-xxx to use the checkpoint.

HJT9328 avatar Apr 24 '24 02:04 HJT9328