Rituraj Singh comments

Repositories
Issues
Comments

Results 2 comments of


                                            Rituraj Singh

Convert lit-llama weights to huggingface

Hi! I was wondering that Is there any update regarding the conversion of the lit-llama fine-tuned merged weights (LoRA) to hugging face format?

[Question] Have you finished LoRA/QLoRA training?

Hi... I find that LoRA Fine-tuning is slower (almost half - 5.72 s/it) as compared to full training (2.77 s/it). Same LR - 2e-5 and Batch Size -4, Grad-accum- 1...