Support for seq2seq models

Open maximus-21 opened this issue 1 year ago • 2 comments

I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?

Jun 10 '24 09:06 maximus-21

I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?

Hi, I also want to apply xlora to T5, did you try to do that?

Jun 19 '24 07:06 yuxiang-guo

@yuxiang-guo Yeah I did. But I faced a tensor related error while doing the forward pass. I tried debugging through it but couldn't exactly understand what was the problem here.

Jun 19 '24 07:06 maximus-21