feature: linearly interpolating one or multiple LoRA with base model

[jon-chuang]

Once https://github.com/ggerganov/llama.cpp/pull/820 is merged, it would be nice to allow linearly interpolating one or multiple LoRA.

LoRA should be able to be loaded interactively, and interpolation weights also adjusted interactively.

[ghost]

@jon-chuang what seems to the blocker currently to make lora adapters swappable?

[yacineMTB]

https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp#L2838-L2876

Proposal: Just Prototyping interactive LoRA and showing that it might be valuable Shouldn't be too hard!

Think of it as sharing a python notebook that uses torch, but instead of torch, ggml

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.