StableCascade icon indicating copy to clipboard operation
StableCascade copied to clipboard

Why the parameters of the generator(Stage_C) model need to calculate the gradient when training the control net?

Open xhh12381 opened this issue 11 months ago • 1 comments

In train_c_controlnet.py, why the parameters of the generator model need to calculate the gradient when training the control net, and only the parameters of the control net are set in the optimizer, I think this will extremely increase the consumption of VRAM, and I would like to know if setting generator.eval().requires_grad_(False) will not affect the control net training.

xhh12381 avatar Mar 04 '24 01:03 xhh12381

In my opinion, the training of the controlnet dependents on the parameters, as the controlnet injects the information into the stageC modules to control the contents of the generated images.

Unified-Robots avatar Mar 04 '24 12:03 Unified-Robots