training results of alpaca-lora 7b and original llama 7b

[frankzhao112]

1 first we load both llama 7b and aplaca-lora-7b . 2 u can use colab to test both llama -7b and aplaca-lora-7b ， u can not find any improvements at all. Using int8 type and peft lora training llama significantly reduces the cost of gpu memory , but the training results can not satisfy the needs of a online chat system.

3 after I use other datasets to train llama using int8 type and peft lora traning, I still get the same results.

[BinWang28]

I tried to fine-tune with my own dataset. Similarly, it seems not learning much. Should I adjust the learning rate or parameters for LORA? Any advice?