diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

examples\controlnet add only_mid_control=true and sd_locked=false

Open geroldmeisinger opened this issue 11 months ago • 9 comments

Is your feature request related to a problem? Please describe. The original controlnet training tutorial mentions the options

  • only_mid_control=True: This can be helpful when your computation power is limited and want to speed up the training, or when you want to facilitate the "global" context learning.
  • sd_locked=False: This will unlock some layers in SD and you will train them as a whole.

Describe the solution you'd like Please make the option available as well.

Describe alternatives you've considered Going back to the original training script without any optimizations implemented.

geroldmeisinger avatar Sep 16 '23 09:09 geroldmeisinger

cc @williamberman @patil-suraj

patrickvonplaten avatar Sep 18 '23 11:09 patrickvonplaten

Yeah these are neat ideas! I think these would be good options for community PRs, happy to open up to the community :)

williamberman avatar Sep 18 '23 20:09 williamberman

Have any updates about this project ?

kelisiya avatar Oct 24 '23 08:10 kelisiya

How to Implement SD_ Locked=False in Diffusers?

jiangyuhangcn avatar Oct 27 '23 09:10 jiangyuhangcn

I would like to try this issue. Seemingly, It can be done by set different gradient recording switch and keep it compatible with accelerate. @williamberman @patrickvonplaten @patil-suraj

ValMystletainn avatar Nov 06 '23 09:11 ValMystletainn

Seems pretty straightforward, just add an option to freeze/unfreeze certain weights.

jon-chuang avatar Nov 24 '23 21:11 jon-chuang

Hi @ValMystletainn, are you still on it? :) Otherwise I'd tackle it.

UmerHA avatar Nov 28 '23 14:11 UmerHA

Okay, I'd tackle this :) Eta 1 week.

UmerHA avatar Dec 12 '23 10:12 UmerHA

This is not an option still in current controlnet train script

Night1099 avatar May 07 '24 07:05 Night1099