hordelib icon indicating copy to clipboard operation
hordelib copied to clipboard

Change or Add inpainting to support new `InpaintModelConditioning` from ComfyUI

Open tazlin opened this issue 1 year ago • 3 comments

As of https://github.com/comfyanonymous/ComfyUI/commit/10f2609fdd2231faf40346df11372601385b6cbb, ComfyUI now supports a new kind of inpainting which purportedly supports lower denoise strengths for inpainting-specific models (as originally proposed here: https://github.com/comfyanonymous/ComfyUI/pull/2501).

  • Problems with inpainting are a very common end-user support topic - it strikes me as very likely that this alternative implementation may be better received in general.
  • Should we convert the existing inpainting pipeline or add this as an option?
  • Are there any gotchas that might trip users when changing to/adding this?

@Efreak @spinagon @db0

tazlin avatar Feb 01 '24 12:02 tazlin

I tested it a bit, the new node seems to work better at both low and high denoise. It also doesn't wash out the rest of the image (or at least not as much). I'd just convert inpaint to this, barring any unexpected pitfalls. Seed values would obviously change, but I don't think anyone cares with inpaint.

spinagon avatar Feb 01 '24 13:02 spinagon

Three only problem I see with converting is that existing saved settings will no longer work. I recently re-ran a bunch of old generations on newer SDXL models, which is something I do occasionally. I'm not sure how common this is.

I can't personally test this change without it being on a running worker; I've never actually used ComfyUI except via Horde, and I'm also out of runpod credit because I spent 2.5 months worth last month 😢.

Efreak avatar Feb 04 '24 21:02 Efreak

I'm OK with switching. Unfortunately maintaining this level of backwards compatibility is not something we can do as a volunteer service. Backwards compatibility is expensive

db0 avatar Feb 05 '24 08:02 db0