Phi1.5 LoRA example

Open harryjulian opened this issue 2 years ago • 3 comments

I've managed to get Phi1.5 fine-tuning with LoRA on my machine using a combination of the LoRA and Phi2 examples. Would it be worth me opening a PR to add this to the examples? If so, where would it fit?

Dec 18 '23 15:12 harryjulian

Good question! We probably shouldn't have a separate example for this. But think it would be great to try and generalize the LoRA example to support multiple models. Right now it supports llama (v1?) and mistral, but it would be great to add phi-* support for it as well.

Dec 18 '23 15:12 awni

@harryjulian can you share your code, I think I can help to adopt it

Jan 06 '24 22:01 shchahrykovich

Here's the scripts I cobbled together.

Jan 09 '24 17:01 harryjulian