Ziqing Yang
Ziqing Yang
没太明白你的意思。 Transformers在默认配置下保存lora时是不保存tokenizer相关文件的；如果是因为缺少tokenizer_config.json，直接从发布的lora包里复制一份即可。
-f 后只能加 ./prompts/alpaca.txt 如果使用其他的prompt，效果可能会很差（因为模型不是用那些prompt训练的）
@Tungsong 你的输入还是没有用到模版，应该是这样的 https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/110#issuecomment-1501725033。 输入的instruction要放在模版里
> After you collect vocab from [sentencepiece](https://github.com/google/sentencepiece) Did you add the vocals to the tokenizer using sentencepieces and create a new tokenizer? Yes, We create a new tokenizer by adding...
In stage One, the input is Original LLaMA, we use run_clm.py to pre-train, and we get Model A In stage Two, we first load Model A, and add the LoRA...
> So did you train run_clm.py two times? Yes. We run run_clm.py two times for 7B model
Thank you for your interest in this project. We will soon release the code for processing tokenizers in one day or two, and keep adding more resources. Please be patient.
> @airaria Thank you for sharing the script for merging tokenizers. I have another question regarding the training of Llama. Have you also experienced a loss of 0 while training...
> @airaria in strart of the training the loss was 4.3 then it is be 0 > > {'loss': 0.0, 'learning_rate': 4.8303582439699045e-05, 'epoch': 0.03} {'loss': 0.0, 'learning_rate': 4.829654336268535e-05, 'epoch': 0.03}...
我们本地测试暂时没遇到过这种错误。 是否和库的缺失有关？ 请确认是否安装了accelerate? pip install accelerate