Train unet and text_encoder separately? (feature request)

[b4nn3d]

trafficstars

Can be useful For low vram, and/or bigger batch sizer and/or fine tuning only one network, maybe?

[brian6091]

You can already do this, disabling train_text_encoder will only train the unet, enabling it will train both.

Or perhaps you mean you want to train the text_encoder by itself?

[b4nn3d]

I know that we can train unet alone, I'm asking if theorically we can get some advantages training only the text encoder ;)

[brian6091]

Ah got it. That's a pretty easy change, I could do that in a day or so if you want to test it ;-)

[cloneofsimo]

Sure thats no problem. I also think that we can update them one at a time, iteratively. This will get the best of both worlds: training both text, unet + small memory

[b4nn3d]

Sure thats no problem. I also think that we can update them one at a time, iteratively. This will get the best of both worlds: training both text, unet + small memory

this is a very good idea too :) still, the ability to train the two networks indipendently.. i think it can be a good source for experiments ;)