运行命令
RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=2 swift infer
--model_type chatglm2-6b
--model_id_or_path /data/LLM_checkpoint/chatglm2-6b/chatglm2-6b
--infer_backend vllm --tensor_parallel_size 1
报错,其中model_id_or_path是全参数微调的模型没有经过lora
[INFO:swift] Due to ckpt_dir
being None
, load_args_from_ckpt_dir
is set to False
.
Traceback (most recent call last):
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/cli/infer.py", line 5, in
infer_main()
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/run_utils.py", line 25, in x_main
args, remaining_argv = parse_args(args_class, argv)
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/utils/utils.py", line 98, in parse_args
args, remaining_args = parser.parse_args_into_dataclasses(
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/transformers/hf_argparser.py", line 338, in parse_args_into_dataclasses
obj = dtype(**inputs)
File "", line 53, in init
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 649, in post_init
set_model_type(self)
File "/root/anaconda3/envs/baichuan2/lib/python3.8/site-packages/swift/llm/utils/argument.py", line 975, in set_model_type
raise ValueError(
ValueError: Please use --ckpt_dir vx-xxx/checkpoint-xxx
to use the checkpoint.