litgpt icon indicating copy to clipboard operation
litgpt copied to clipboard
verified publisher icon Lightning-AI

Port finetuning from Lit-LLaMA

Open carmocca opened this issue 2 years ago • 2 comments

  • [ ] full #117
  • [x] LoRA #128
  • [x] Adapter #31

carmocca avatar May 05 '23 19:05 carmocca

hi is fine tuning from lit-Llama file available yet? For now it seems 1 time training but if we can continue fine tuning form lit-LLama It would be quite helpful

joseph35533553 avatar Jun 10 '23 14:06 joseph35533553

@joseph35533553 We have adapter merged (https://github.com/Lightning-AI/lit-parrot/blob/main/howto/finetune_adapter.md), and Lora on its way: #128

carmocca avatar Jun 10 '23 17:06 carmocca