ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard

ControlNet Causes Black Images

Open pphoto808 opened this issue 1 year ago • 10 comments

ControlNet seems to be the culprit (at least for me) for black images. If I bypass or remove controlnet (apply or apply advance), images are generated. I saw the other thread about black images, but none addressed controlnet.

I just now updated Comfy and that's when the problems started. I saw an update for controlnet aux when i did update via manger. Thought that would solve the problem but it did not.

Manger says Comfy is up to date.

pphoto808 avatar May 19 '24 01:05 pphoto808

Doing more testing. Even after removing ControlNet from previous workflows, still get black images. if I delete loaders (checkpoint, lora, etc), prompts, and sampler, install them again I get images. However still black images if I add controlnet.

pphoto808 avatar May 19 '24 04:05 pphoto808

What's your GPU, checkpoint you are using, controlnet, etc..?

comfyanonymous avatar May 19 '24 06:05 comfyanonymous

Sorry, normal I lead off with specs: Apple Studio M2 Ultra, MacOS Sonoma 14.5

okkkkkk....I just spent the last two hours running through various combinations of configs. I would get black images sometimes depending on how many steps I used. Fewer steps, less black images. But what finally dawn on me was the default workflow worked with any amount of steps EVERY TIME. The sampler and scheduler was the key. This time narrowed it down for sure. Karras! I tested euler, euler_a, dpmpp_sde, dpmpp_2m, dpmpp_2m_sde with normal, exponential, and sgm_uniform schedulers. No black images, even with workflows prior to updates. ANDDDD with ControlNet. EVERY TIME I used Karras, black images. I say this every time...not sure if this is just a Mac issue or not. But the last two hours were not wasted.


Update: Just tried default workflow with controlnet depth and revanimated (SD15) checkpoint, seems like WITH controlnet 17 steps is the max before black images.

pphoto808 avatar May 19 '24 09:05 pphoto808

Can you give me a simple workflow that returns a black image for you?

comfyanonymous avatar May 19 '24 22:05 comfyanonymous

Screenshot 2024-05-19 at 15 03 55

Screenshot 2024-05-19 at 14 25 55 blackscreen.json

Default workflow. Just changed the scheduler to karras. Included screen shot so you see what i see.

pphoto808 avatar May 20 '24 00:05 pphoto808

If you run ComfyUI with --dont-upcast-attention does that fix it?

comfyanonymous avatar May 20 '24 02:05 comfyanonymous

Nope :(

pphoto808 avatar May 20 '24 02:05 pphoto808

Sorry I meant --force-upcast-attention

Can you also try updating pytorch to see if it fixes the issue?

comfyanonymous avatar May 20 '24 02:05 comfyanonymous

Sorry I meant --force-upcast-attention

Can you also try updating pytorch to see if it fixes the issue?

Ok, the --force-upcast-attention did the trick. Tested with controlnet at 30 steps and all looks good.

Updating pytorch was the first thing i tried lol.

I just need to modify start script for Pinokio now and test it there.

pphoto808 avatar May 20 '24 05:05 pphoto808

Modified start.json for Pinokio. All good.

Thanks for the assist!

pphoto808 avatar May 20 '24 05:05 pphoto808