LLM-Tuning icon indicating copy to clipboard operation
LLM-Tuning copied to clipboard

这个微调代码不能直接用来baichuan-13B模型的微调?在13B一直报错

Open DaiJitao opened this issue 1 year ago • 2 comments

File "baichuan_lora_tuning.py", line 142, in main() File "baichuan_lora_tuning.py", line 134, in main trainer.train() File "/data01/conda_pkg/envs/baichuan/lib/python3.8/site-packages/transformers/trainer.py", line 1664, in train return inner_training_loop( File "/data01/conda_pkg/envs/baichuan/lib/python3.8/site-packages/transformers/trainer.py", line 1940, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "/data01/conda_pkg/envs/baichuan/lib/python3.8/site-packages/transformers/trainer.py", line 2735, in training_step loss = self.compute_loss(model, inputs) File "baichuan_lora_tuning.py", line 45, in compute_loss return model( AttributeError: 'tuple' object has no attribute 'loss'

DaiJitao avatar Jul 13 '23 02:07 DaiJitao

每个模型结构可能会有不一样,代码应该需要小的改动。近期我抽空调试一下

beyondguo avatar Jul 13 '23 07:07 beyondguo

每个模型结构可能会有不一样,代码应该需要小的改动。近期我抽空调试一下

现在支持baichuan-13b的模型吗

kunden0612 avatar Sep 05 '23 08:09 kunden0612