Suchun Xie

> 这是我们重新训练，对齐了最新版本（包括peft tokenizers）的一个效果： > > ``` > >what is the capital of America > The capital of the United States of America is Washington, DC. > >When is the Thanks...

Hi @mreso , thanks for the confirmation! also regrading the whole finetuning process, I noticed that when run several times with all the same parameter settings, the loss at each...

Great! Thanks for your answer, it helps a lot.

@katossky @tuanio Hi, can you provide a sample on this solution? I have origin llamamodel, and peft-lora finetuned llamamodel , and still get error not support for text generation, when...

@Facico 想请教下，如果是文本摘要这种输入一般很长的任务，超过截断长度要不要设置加eos呢，我现在看各个版本的llama finetune都是超过就不加eos, 而且训练的时候有很多样本是没有标签的， 训练来的模型有时候会说不出完整的句子。这种情况下厨了增大max_seq-len, 请问还有什么比较好的方法吗？ 还是说这种长文本就设置一律加上eos呢

@KJ-Waller @FHL1998 i get the same loss trend when full finetune on llama 2, have you solved this problem finally?