controlnet_aux icon indicating copy to clipboard operation
controlnet_aux copied to clipboard

did you put the invert(from white bg & black line) preprocessor?

Open igorleo10 opened this issue 1 year ago • 4 comments

igorleo10 avatar Jun 29 '23 15:06 igorleo10

Can you please link to a code line or something? I don't know what you mean with this title

patrickvonplaten avatar Jul 03 '23 14:07 patrickvonplaten

I got the same question regarding the lineart and linart_anime preprocessors. I'm curious about why we need to invert the black and white colors in the last step, where the original controlnet did not do this invert process (or if it was done somewhere else, could you please clarify this for me?)

detected_map = 255 - detected_map

syunar avatar Aug 11 '23 04:08 syunar

In the original sources, the invert happens here:

https://github.com/lllyasviel/ControlNet-v1-1-nightly/blob/main/gradio_lineart.py#L48

The controlnet_aux processor output can be directly used by the model. In the original implementation, most of the code is in the annotator but there was often minor post-processing done in the example.

I assume the original question is referring to the "invert (from white bg & black line)" processor that exists in the A111 extension. There's no implementation here but it would be trivial for someone to add.

pdoane avatar Aug 11 '23 04:08 pdoane

我对 Lineart 和 linart_anime 预处理器有同样的问题。我很好奇为什么我们需要在最后一步中反转黑白颜色,而原始的控制网络没有执行此反转过程(或者如果它是在其他地方完成的,您能为我澄清一下吗?)

detected_map = 255 - detected_map

In A111, the "invert" operation is applied to images that do not go through controlnet_aux, and it involves reversing white background to black lines, turning them into black background with white lines. Other preprocessing modes are handled within controlnet_aux. The input for controlnet needs to be in black background with white lines.

leeguandong avatar May 13 '24 12:05 leeguandong