Combining multiple LoRA adapters

[winglian]

Feature request

Is it possible to combine multiple LoRA adapters like you might do to combine multiple styles with Stable Diffusion?

Motivation

I think we could get higher quality model outputs from multiple LoRAs by combining them.

Your contribution

I'm not familiar with the codebase, but if this is architecturally possible and someone pointed me in the right direction, I'm happy to take a stab at it and create a PR.

[tgaddair]

Hi @winglian, this is posible today using Adapter Merging like TIES and DARE. You can also linearly combine adapters per request. Please take a look and let me know if you have any feedback!