[Feature request] Guess Mode (or use another better name)

[lllyasviel]

By putting control map in unconditional guidance side and scale the U-net skip connections, we can enforce the encoder to recognize the contents in control maps.

See also: Examples - see updated readme in https://github.com/lllyasviel/ControlNet (Section: Guess Mode / Non-Prompt Mode) Implementation - https://github.com/lllyasviel/ControlNet/commit/005008b44d9a29725310a19bd173a509fabcdd30

Note that this may be experimental. Let me know what you think. This feature is not very eager and perhaps we can consider test it with some experiments first. Let us put it in a secondary priority.

Thank you for the great work of maintaining the webui.

[lllyasviel]

also, to predict unconditional eps, perhaps we may also try to ignore the unet adding rather than using zero as control map, to save computation. but we may experiment a bit more about it.

[Mikubill]

Quick test shows different cfg-alike method will produce small bias:

by using zero as control net input

by ignore control map on uncond

base

All generation use (cfg3, no-prompt) preset. Currently implemented in extension is the latter. If former option provides better quality I will switch to it.

[lllyasviel]

cool! perhaps ignoring control map on uncond is better considering some controls do not support zeros as uncond, like seg or normal map. ignoring on uncond should be better for these cases. let me also update main repo.