[Feature Request] Possibility of more than one ControlNet input?

[Dekker3D]

This one may be iffy, but based on what I read, it seems like it might be possible to stack more than one ControlNet onto Stable Diffusion, at the same time. If that's possible, it would be really interesting to use that. Being able to define both depth and normals when generating images of a building, or depth + pose for characters, would allow a lot of control.

[Mikubill]

Will implement with/after T2I-Adapter

[CCRcmcpe]

Yes it is possible and relatively easy to implement. There are 2 ways.

1. Add weighted sum of multiple ControlNet intermediate hidden states to UNet down -> up skip connections. This is already implemented by T2I-Adapter, although they hardcoded weights and that only 2 Adapters can be used at the same time.
2. Using Classifier-Free Guidance style, add eps-predition. This will be more expensive computationally but somewhat results images of better quality in my experiments.

[Mikubill]

impl in https://github.com/Mikubill/sd-webui-controlnet/pull/262