LGM icon indicating copy to clipboard operation
LGM copied to clipboard

Is there any current way to run inference on this model with a LoRA?

Open x-CK-x opened this issue 1 year ago • 1 comments

I didn't know if there was support for something for this yet, but even if there wasn't I was curious as to know what might be needed to add that level of modularity to let users try from all the different LoRAs available with this during inference?

x-CK-x avatar Feb 11 '24 22:02 x-CK-x

@x-CK-x Hi, since our model is essentially a U-Net with attention like stable diffusion, it's possible to add LoRA. But it needs experiments to decide which layers to apply, and what kind of training is needed. Also, since the model itself is not really big, maybe you can first try to directly finetune the whole model.

ashawkey avatar Feb 12 '24 02:02 ashawkey