LLaMA-Factory

LLaMA-Factory copied to clipboard

[Feature] 支持DPO-Positive (DPOP)，DPO改进版

17

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 1. https://github.com/abacusai/smaug 2. https://arxiv.org/abs/2402.13228 ### Expected behavior _No response_ ### System Info _No response_...

WangRongsheng

llama2-7B 全量微调中存在异常？

2

想请教下， 在gsm8k数据集上采取完全相同的设置，仅修改精度bf16,fp16 和 学习率 、epoch等 训练均已收敛，存在如下三种效果：（多次实验，很普遍的现象。训练和推理用的是同一个模版）   

huangjf11

Documentation update of some python scripts

1

Dear Developers, I'm delighted to inform you that the documentation update for Python scripts has been successfully completed. The updated documentation provides clear explanations of function parameters, return types, and...

louisbrulenaudet

如题，打算引入 trl 库的KTOTrainer吗？

wsp317

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction We observe that across models and datasets using Zero3 with bf16 yields much higher...

mnmueller

单机多卡 保存模型的时候忽然显存增加

1

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction deepspeed --num_gpus 2 src/train_bash.py \ --deepspeed ds_config.json \ --stage sft \ --do_train \ --model_name_or_path...

DinahD967102ick

https://huggingface.co/docs/trl/main/en/dpo_trainer#downsides-to-merging-qlora-before-dpo-approach-2 https://github.com/jondurbin/qlora/blob/main/qmerge.py

hiyouga

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction Multiturn datasets are supported in the original dpo example. Wouldn't it be appropriate for...

kostum123

sft_packing实现的问题

4

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 看目前sft_packing的实现只是单纯将不同的单轮sft数据拼接到一起，然后分别计算target部分的loss def preprocess_packed_supervised_dataset( examples: Dict[str, List[Any]], tokenizer: "PreTrainedTokenizer", template: "Template", data_args: "DataArguments", ) ->...

dyh1996

[Feature request / help] Evaluate on different dataset to training dataset

1

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction I would like to know if there is an option to evaluate the model...

Peter-Devine