PEFT LORA / QLORA

[louisoutin]

Hello, Are you planning to add support for parameter efficient finetuning methods? Also does it support doing inference using those adapters models to optimize VRAM? Thanks

[aarnphm]

Yes, I'm currently working on fine tuning API atm

[aarnphm]

Peft and LoRA is now supported during serving and building time via --adapter-id

I will continue to gradually add support for all default fine-tuning layers for each of the models.

Will create a new tracking issue to specific set method support for each of these models