ComfyUI
ComfyUI copied to clipboard
comfyanonymous
feat: SetArea for models or LoRAs
LoRA tends to have side effects.
For example, a fashion LoRA can affect the face or picture style, and so on.
To avoid this, there's an (A1111) extension hako-mikan/sd-webui-lora-block-weight, which limits the activation of layers selectively.
However this also has side effects and requires a lot of experiments to find a suitable combination.
Ideally, I think the best possible idea would be enabling SetArea for models and LoRAs, like conditioning.
(Not block weight)
So for instance, fashion LoRA can be only applied to body, not face.
How do you think?
Is this technically possible?
Thanks :)
Actually.. I'd implemented this once. https://github.com/comfyanonymous/ComfyUI/pull/303
However, it is expected that if the engine is overhauled for SDXL support, it will no longer be compatible, resulting in the postponement of the PR implementation. Now that the SDXL support work has been completed, it appears that maintenance work for PR will be possible, but it seems that I will have to take another look at the internal structure of the engine. lol.
@ltdrdata Thank you! That's also a really good node!
However, you misunderstood what I meant.
However this also has side effects and requires a lot of experiments to find a suitable combination. Ideally, I think the best possible idea would be enabling SetArea for models and LoRAs, like conditioning.
This means I imagine something like this :)
Ah.. I just mentioned about lora block weights.
Maybe area wise lora application requires quite modification of sample.
Impact Pack allows you to apply lora/checkpoints to separate areas with no overlap with the TwoSamplersForMask nodes. It isn't perfect, but works
Using the TwoSamplersForMask node itself gives errors, but you can use the Iterative Upscale node with upscale_factor set to 1.001. 1.0 doesn't work. It will round the dimensions to multiples of 8 so unless you make some insane resolutions on the first pass it will keep the size as you set it.
You will also need to use the openpose controlnet to keep the character consistent and this requires some extra work to implement in a usable way.
It would be awfully convenient if someone happened to have already figured it out, wouldn't it?
https://civitai.com/models/21100?modelVersionId=104147
There are some other custom nodes used too, so make sure to check for them in About this version
Using the TwoSamplersForMask node itself gives errors, but you can use the Iterative Upscale node with upscale_factor set to 1.001. 1.0 doesn't work.
What specific error are you encountering? Providing a description will be helpful in addressing the bug and assisting with the fix. It's possible that the type errors you encountered a few days ago were already fixed. Please give it another try on latest version and see if the issue persists.
What specific error are you encountering? Providing a description will be helpful in addressing the bug and assisting with the fix. It's possible that the type errors you encountered a few days ago were already fixed. Please give it another try on latest version and see if the issue persists.
I don't remember what it was, but it seems to be fixed now as I can't reproduce it. Thank you.
Taking inspiration from this, I have added a node TwoAdvancedSamplersForMask to the Impact Pack.
https://www.youtube.com/watch?v=S1ksTNs5VX0