LongQLoRA

LongQLoRA copied to clipboard

Thanks for the work, I reproduced the training, but merging lora modules got an error. if q_len % group_size > 0: raise ValueError("q_len %d should be divisible by group size...

xjwhy

generartion -> generation

eltociear

It also combine qlora of longlora, what's the difference between longQLora and longlora+qlora? Model url as follow: https://huggingface.co/Yukang/LongAlpaca-7B-qlora-weights

skykiseki

按照教程，15k的上下文，8张v100 32G的跑不起来， 10K的可以跑起来

Liuchunyangboy

Hello author, could you please tell me how the pre-training dataset and the instruction fine-tuning dataset are made? Thank you very much!

Vansay28