sd-forge-couple icon indicating copy to clipboard operation
sd-forge-couple copied to clipboard

[FEATURE REQUEST] TWO-SHOT Drawing masks (from Latent couple)

Open FemBoxbrawl opened this issue 9 months ago • 9 comments

I was wondering if you could add an option to also use color coded masks (like in the old Stable-diffusion two shot), where each color is a specific area that you prompt.

The thing is, the current Latent couple extension does not work with forge. SO I was wondering if you could also add it to Forge couple? The reason i am asking is because with the tiles, you are limited to what goes wear, while with the color masking, you can have seperated areas as a mask (lets say some complex fight scene where the head and arm is visible but the torso is not, and then another guy is blocking that view, so you'd have to somehow mask it so that prompt belong to the same one.

Basically a color coded masks that can be "seperate" but would still follow the prompt.

FemBoxbrawl avatar May 01 '24 14:05 FemBoxbrawl

Not sure if I follow

Doesn't the Advanced mapping already do this?

Haoming02 avatar May 02 '24 04:05 Haoming02

Not sure if I follow

Doesn't the Advanced mapping already do this?

You are right, the tile does work to some extent, . But I meant like a "masking" option, where you can draw over specific areas, so for example 2 seperate "regions" for the same line/prompt. (to be able to make Overlaps, etc.

FemBoxbrawl avatar May 04 '24 01:05 FemBoxbrawl

Not sure if I follow

Doesn't the Advanced mapping already do this?

they mean a mask you can draw, for even more control

rktvr avatar Jun 07 '24 10:06 rktvr

any plans for being able to draw a mask?

rktvr avatar Jul 23 '24 17:07 rktvr

After I finish revamping Advanced mode to support the new Forge I guess

Still need a good way to store multiple "layers" of images after you draw the mask though...

Haoming02 avatar Jul 29 '24 07:07 Haoming02

Aight, time to think of a way to implement Mask mode to the UI...

Haoming02 avatar Aug 13 '24 08:08 Haoming02

Awesome, can't wait for the update! Does Forge Couple work on new Forge now? I wasn't sure if I need to reinstall it from here or wait for the integrated version inside of Forge to be updated.

I'm SUPER excited for masking mode as this was pretty much the only feature Regional Prompter had that Forge Couple did not.

Thank you so much for all your work!

queenofinvidia avatar Aug 16 '24 01:08 queenofinvidia

Does Forge Couple work on new Forge now?

Yes. It should work for the new Forge now. (Only SD 1.5 and SDXL, but not Flux)

Haoming02 avatar Aug 16 '24 03:08 Haoming02

~~Slowly going insane from working with Gradio 3 Image component...~~

Haoming02 avatar Aug 22 '24 03:08 Haoming02

Hello, please give the mask branch a try~

Also supports the new Forge~

Haoming02 avatar Aug 26 '24 09:08 Haoming02

Hello, please give the mask branch a try~

Only works for old Forge (Gradio 3) right now...

playing around with it at the moment, needs a way to clear the actual canvas. when i "Save mask" it doesn't clear the canvas, so it ends up duplicating everything and i don't see where to clear it.

using it on reForge with gradio 3.41.2

rktvr avatar Aug 26 '24 10:08 rktvr

Oh, click the Create Empty Canvas button to clear the canvas.

Haoming02 avatar Aug 26 '24 10:08 Haoming02

Oh, click the Create Empty Canvas button to clear the canvas.

doesn't work, when i click it the anvas stasy the same. is probably because of reForge though.

rktvr avatar Aug 26 '24 10:08 rktvr

doesn't work, when i click it the anvas stasy the same. is probably because of reForge though

Wait, really? Let me try.

Haoming02 avatar Aug 26 '24 10:08 Haoming02

i'll assume that any errors are because of the (somewhat?) old version of reForge i'm using. I'll see if anything changes if I update it though I don't remember if they started using gradio 4 in the newer reForge.

rktvr avatar Aug 26 '24 10:08 rktvr

Huh, it's working for me on reForge though. Using the main branch.

Let me try dev_upstream branch real quick.

Haoming02 avatar Aug 26 '24 10:08 Haoming02

apparently this is the version i'm on: https://github.com/Panchovix/stable-diffusion-webui-reForge/commit/a75e69cb3528726e11a41f23fdec745a9f03d795

I'll update and re-check everything again. it was likely just because of that version of reforge

rktvr avatar Aug 26 '24 10:08 rktvr

Clicking the Create Empty Canvas button empties the canvas for me, on the latest commit of both main and dev_upstream branch of reForge.

Haoming02 avatar Aug 26 '24 10:08 Haoming02

Just updated and it does indeed clear now! Thank you for cour continued work and support on this add-on! 🥳

rktvr avatar Aug 26 '24 10:08 rktvr

There was code added code to OG forge for Latent couple

It is a bit late but "online loras" are currently available now: https://github.com/lllyasviel/stable-diffusion-webui-forge/blob/2f1d04759f36122cbb3553f0f0758755790624c7/backend/operations.py#L14 - lllyasviel

View “online lora” as a module similar to ControlLoRA so that it is moved to GPU together with model when sampling, achieving significant speedup and perfect low VRAM management simultaneously. - lllyasviel

the code hasn't been ported over to reforge yet, and i'm not sure if it just help with Latent couple or 100% fixes Latent couple.

metapea avatar Aug 27 '24 22:08 metapea