LongQLoRA

LongQLoRA copied to clipboard

Thanks for the work, I reproduced the training, but merging lora modules got an error. if q_len % group_size > 0: raise ValueError("q_len %d should be divisible by group size...

xjwhy

generartion -> generation

eltociear

It also combine qlora of longlora, what's the difference between longQLora and longlora+qlora? Model url as follow: https://huggingface.co/Yukang/LongAlpaca-7B-qlora-weights

skykiseki