ChatGLM-Efficient-Tuning
ChatGLM-Efficient-Tuning copied to clipboard
hiyouga
Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调
/home/ma-user/anaconda3/envs/chatglmeV2/bin/python /home/ma-user/work/zhanghongjun/ChatGLM-Efficient-Tuning-main/src/train_ppo.py --do_train --dataset alpaca_gpt4_en --finetuning_type lora --checkpoint_dir path_to_sft_checkpoint --reward_model path_to_rm_checkpoint --output_dir path_to_ppo_checkpoint --per_device_train_batch_size 2 --gradient_accumulation_steps 4 --lr_scheduler_type cosine --logging_steps 10 --save_steps 1000 --learning_rate 1e-5 --num_train_epochs 1.0 --fp16 /home/ma-user/anaconda3/envs/chatglmeV2/lib/python3.7/site-packages/requests/__init__.py:104: RequestsDependencyWarning:...
/src/pet/core/model.py文件第71行tokenizer = AutoTokenizer.from_pretrained载入tokenizer 之后，打印出tokenizer.eos_token_id=2，并不是130005，导致133行的断言assert tokenizer.eos_token_id == 130005触发 
您好，我在用DeepSpeed进行多卡训练时，会保存一些zero_pp_rank_XX_mp_rank_XX_model_states.pt文件，文件夹有20多个G，导致存储空间不足。这应该如何处理？ 而用accelerate进行多卡训练时，会报错： ··· File "/root/miniconda3/envs/llm/lib/python3.8/site-packages/peft/tuners/lora.py", line 565, in forward result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias) RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling "cublasCreate(handle)" ··· 训练脚本是： ··· accelerate launch src/train_sft.py \ --do_train...
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/wmnlab/chat/ChatGLM-Efficient-Tuning/src/train_bash.py:21 in │ │ │ │ 18 │ │ 19 │ │ 20 if __name__ == "__main__": │ │ ❱ 21 │...
