krita-ai-diffusion icon indicating copy to clipboard operation
krita-ai-diffusion copied to clipboard

Published 20 hours ago • verified publisher icon Acly

Layer Diffusion integration

Open paprik opened this issue 1 year ago • 3 comments

Basic implementation of ComfyUI Layer Diffusion: https://github.com/huchenlei/ComfyUI-layerdiffuse Uses two new custom nodes and adds a new transparency checkbox for the Generate action.

Showcase: transparency_test

paprik avatar Sep 05 '24 17:09 paprik

Created as draft since I likely missed something. Generate (with upscaling too) works great. Could not manage to make it work with Refine/Inpaint yet, it keeps adding a white background for some reason.

paprik avatar Sep 05 '24 17:09 paprik

For Refine/Inpaint a special VAE encode is needed that understands transparency. The regular VAE only does RGB (ignores alpha channel).

It was added in forge a while ago, but is not implemented in the ComfyUI nodes.

How can the 2-pass generation (with upscale step) work though? It should run into the same problem?

Acly avatar Sep 06 '24 08:09 Acly

You're right, 2-pass upscale doesn't work, I've just generated a 2k in one step. Hopefully it gets added to ComfyUI nodes soon.

I still have to try Attention injection, because the Conv one changes the model a quite a lot.

paprik avatar Sep 06 '24 10:09 paprik