finetuning model using diffuser

[MahdiMohseni0033]

does it possible to fine-tune this model using diffuser in this stage?

[haofanwang]

Do you mean train the ControlNet and keep the base model frozen? @MahdiMohseni0033

[MahdiMohseni0033]

Do you mean train the ControlNet and keep the base model frozen? @MahdiMohseni0033

it doesn't matter , I just customize this model for a specific task , and I followed the training stage in official ControlNet repository and I was faced with Cuda OOM Error I want to train it using RTX 3060 , and the reason which I wanted to use Diffuser , is using memory efficient technique like xformer to make it feasible to train it in GPU with less memory

[haofanwang]

This reply may help.

[MahdiMohseni0033]

This reply may help.

Thank you for responding kindly, but my primary inquiry is regarding how to train or fine-tune your code (ControlNet by diffuser). Do you have a training code available, or can you provide it to us?

[haofanwang]

Here is the training example. You just need modify a few lines.