Guidance on finetuning with PEFT/LoRA

[AviSoori1x]

trafficstars

Could you please share any specific guidelines on doing this with PEFT/ LoRA? Particularly the LoRA config e.g.target_modules=["q_proj","v_proj"] or something else. I notices the finetuning script provided and is much appreciated but some example on how to do this with more resource efficient methods such as LoRA would be awesome. Thanks in advance!

[bdqnghi]

you can use CodeTF to fine-tune with PEFT/LORA: https://github.com/salesforce/CodeTF