WIP/Add: NEFTune to Lit-GPT

Open nkasmanoff opened this issue 2 years ago • 0 comments

Picking up an issue I opened last week, feel free to close if no interest or being done elsewhere.

So far I've added it to the LoRA implementation, but could use it as well on the full fine-tuning too.

Lmk!

https://github.com/Lightning-AI/lit-gpt/issues/642

Oct 18 '23 17:10 nkasmanoff