PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

[Bug]: llm merge_lora_params 合并后不保存 merge权重

Open sanbuphy opened this issue 8 months ago • 5 comments

软件环境

- paddlepaddle: 
- paddlepaddle-gpu:  develop 
- paddlenlp: lastest 162d8d31c84f60b804a0abeee8f4f1e4b32308ef

重复问题

  • [X] I have searched the existing issues

错误描述

使用 llm merge_lora_params.py,合并一个 QLora 训练好的模型,但是没有合并后的模型结果,输出文件夹什么都没出现

稳定复现步骤 & 代码

python merge_lora_params.py
--model_name_or_path FlagAlpha/Llama2-Chinese-7b-Chat
--lora_path /home/aistudio/data/checkpoints/llama_lora_ckpts/checkpoint-286
--merge_lora_model_path /home/aistudio/data/llama_lora_merge
--device "gpu"
--low_gpu_mem True

似乎一直卡在加载的阶段,然后过一阵子后直接结束进程。(怀疑内存不够,但应该不至于吧 ,aistudio 32g v100 开发机)

image

image

但并非是 lora 问题,因为可以动态图加载推理

python predictor.py --model_name_or_path FlagAlpha/Llama2-Chinese-7b-Chat \
                    --data_file /home/aistudio/data/dummy/dev.json --dtype float16 \
                    --lora_path /home/aistudio/data/checkpoints/llama_lora_ckpts/checkpoint-286

sanbuphy avatar Jun 10 '24 08:06 sanbuphy