Ziqing Yang
Ziqing Yang
1. Chinese-LLaMA可以使用float16配合peft进行LoRA训练，训练脚本里也默认开启了fp16 2. 暂无明确计划，或许会考虑针对个别场景采用更长的block size
参考 https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/361
Would you mind if I copy and put your Colab notebook into the repo, so that the repo is self-contained?
After I click the submit, html tags are added to the texts (The input is "你好"): Is there a way to remove them? @sunyuhan19981208
@sunyuhan19981208 My colleague and I can both reproduce the problem with the notebook link you gave https://colab.research.google.com/drive/1mG8CwGub-hcoeo79FjfoWtxqOuu3P_y1?usp=sharing. We have tested Safari + MacOS, Chrome + MacOS, Chrome + Win11, Edge...
> > @sunyuhan19981208 My colleague and I can both reproduce the problem with the notebook link you gave https://colab.research.google.com/drive/1mG8CwGub-hcoeo79FjfoWtxqOuu3P_y1?usp=sharing. > > We have tested Safari + MacOS, Chrome + MacOS,...
We temporarily merge the PR into `gradio_notebook` branch, and then merge it into the main branch after reorganizing some code and docs.
1. 可以在新的合适的数据上继续精调模型，提升降低拒答概率 2. CPU精调目前实践上基本不太可行，见 #346
`data/pt_data.txt`只是示例，预训练数据与指令数据无关， 比如可以采用中文wiki语料，悟道语料（WuDaoCorpus）、ROOTS数据集中的中文部分等语料
> 好的， > > 几个问题哈， > > 1）那我是在合并之后的模型基础上微调的，是不是应该和合并后的模型再合并，而不是去和原版llama-hf合并？ 2）合并还是用merge_llama_with_chinese_lora.py么 1) 对，你基于什么模型训练，就和什么模型去合并 2) 可以用merge_llama_with_chinese_lora.py，但需要准备好相关模型权重和配置文件，仿照我们发布的LoRA的内容即可。