Dhruv Jain

Support for seq2seq models

2

I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?

Using trl SFTTrainer creates empty adapter.safetensors file while saving when training with LoRA and Deepspeed Zero3

4

### System Info When training with trl SFTTrainer with peft and deepspeed zero3 configuration it results in adapter_model.safetensors file of just 40 bytes i.e empty. However when training with deepspeed...