ExponentialML

>The problem is, regular inference via dreambooth with diffusers isn't possible, because dreambooth needs a ckpt. So, in order to use lora + model - we need to drop the...

From my testing, Tthere's also a lot of variance in how the model is trained that needs to be taken into account. When you use lower LoRA ranks, you generally...

@jorgemcgomes Have you tried looking at the [unofficial implementation ](https://github.com/bryandlee/Tune-A-Video)? The maintainer uses the PseudoConv3d for `conv` and Spatio Temporal Attention which seem to provide more of what you're looking...

> @ExponentialML I didn't know about that one. Thanks, I will check it out. But comparing the code, it seems that both implementations use the same kind of attention? No...

I can take a shot to see if this works with current available implementation floating around. If we're just to training the CrossAttention layers (finetuning the Psuedo Conv3D layers are...

I created a repository for Text2Video finetuning here using the recent Diffusers addition. Let me know how it goes if you give it a shot! https://github.com/ExponentialML/Text-To-Video-Finetuning

@kabachuha Didn't realize you posted it. All good, thanks for doing it!

Hi. I'm unable to resolve this at the moment, but I think I could lead you in the right direction (I haven't looked at it in full). I don't know...

Hey, sorry for the late response @kenkenissocool! This is due to Diffusers implementing their own version of LoRA in recent versions, which causes this error. I will look to resolve...

Hi, and thanks! Just to clarify, I'm not the original author of the code. The two forward passes are for to the text to image training part if the user...