Tungsong

There were missing keys in the checkpoint model loaded

6

**I trained Alpaca-LoRA model with params:** base_model: /usr/local/dbbd/model/llama-7b-hf data_path: alpaca_data.json output_dir: ./lora-alpaca batch_size: 128 micro_batch_size: 4 num_epochs: 2 learning_rate: 0.0001 cutoff_len: 512 val_set_size: 2000 lora_r: 8 lora_alpha: 16 lora_dropout: 0.05...

How to improve training efficiency and shorten training time

1

I have two T4 on my machine, and I want to improve training efficiency， because it has enough memory when I use the default params  I tried to update...

llama-7b和Lora权重合并之后，在自己的数据上继续fine-tuning

16

微调完给出的answer一直在重复同一句话，而且也不是答案 微调方式参考的https://github.com/27182812/ChatGLM-LLaMA-chinese-insturct 训练了3个epoch，效果如下，请教有谁知道为什么会这样吗 

N次循环会初始化N次还是只初始化一次，怎么初始化的呢，不太懂这个α怎么来的，有什么作用 

微调完多卡推理时报精度不对的问题 expected scalar type Half but found Float ，单卡推理就没有这个问题

1

多卡推理参考的ChatGLM-6B官方的多卡部署 https://github.com/THUDM/ChatGLM-6B#%E5%A4%9A%E5%8D%A1%E9%83%A8%E7%BD%B2

微调bge-large-zh-v1.5后在自己的数据上评估

1

不懂这行代码有什么作用，引入时报错