ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard

[feature request] node for region limited LoRAs / couples

Open dcsan opened this issue 2 years ago • 1 comments

Is there a comfy node for this? I searched the issues but couldn't see anything.

I've seen ways to compose LoRAs so you can have them apply to only part of the output render This allows things like "couple shots" where you apply a different lora to left or right halves of an image, so maintaining separation on the two LoRAs. when i do this with a simple lora pipeline and two loras with different trigger phrases, there seems to be a lot of leakage between characters or the output loses it's likeness. This method of separation seems more promising for rendering "social scenes" with multiple subjects.

here are some relevant notes / plug-ins for the A1111 ecosystem:

https://github.com/hako-mikan/sd-webui-regional-prompter https://github.com/opparco/stable-diffusion-webui-two-shot

https://stable-diffusion-art.com/regional-prompter/#Region_prompting_with_ControlNet https://github.com/opparco/stable-diffusion-webui-composable-lora

older: latent couple https://github.com/opparco/stable-diffusion-webui-two-shot

https://www.youtube.com/watch?v=vZ3W62dxuXI&ab_channel=fofraiartandexperiments https://twitter.com/fofrAI/status/1640837861783007237

dcsan avatar Jul 11 '23 17:07 dcsan

Have a look at https://github.com/comfyanonymous/ComfyUI_examples/tree/master/area_composition

neecapp avatar Jul 11 '23 18:07 neecapp

Is there any solution to implement functionality similar to Regional Lora? • https://civitai.com/models/21100/comfyui-multi-subject-workflows?modelVersionId=87255 • This uses a mask to generate two samples, with only the latest two regions.

carl10086 avatar Sep 07 '23 11:09 carl10086

Is there any solution to implement functionality similar to Regional Lora? • https://civitai.com/models/21100/comfyui-multi-subject-workflows?modelVersionId=87255 • This uses a mask to generate two samples, with only the latest two regions.

Wow, it was initially created as an experimental node, and I was curious about how useful it would be. This is the first time I've seen a workflow applying it to this extent. I had created TwoSamplers and considered MultiSamplers as well, but I had forgotten about it while contemplating how to structure the nodes for convenience. I've had some ideas come to mind based on my experiences during this time, so I should explore improvements.

ltdrdata avatar Sep 07 '23 11:09 ltdrdata