[Feature Request]: Regional Prompter - latent mode support

[wardensc2]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Hi Illyasviel

Please implement Regional Prompt Control and Tiled Noise Inversion from multidiffusion upscaler into your WebUI Forge. These features are really neccessary for us to draw multi objects with multi prompt in 1 picture and also upscale with much more detailed than just Hires Fix.

Thank you in advance

Proposed workflow

1. Go to ....
2. Press ....
3. ...

Additional information

No response

[tetsuoo-online]

I second that. I also wish I could use my favorites upscalers, I have no idea why this isn't even possible in the very limited integrated version in Forge. Or at least allow to add the original MultiDiffusion extension... right now it's automatically deactivated if I try to install it, not compatible I guess :/ That tool was essential for me in A1111, it's a big loss in Forge

[catboxanon]

Tiled Diffusion/Multidiffusion issue is already open here: https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/191

I'll repurpose this issue for Regional Prompter instead.

[rktvr]

would be nice to have the regional prompter/latent couple extensions work in Forge. not sure why they don't.

[arcusmaximus]

The Regional Prompter addon partially works in Forge. Specifically, it starts failing with SDXL at "high" resolutions, which makes me suspect some kind of Forge-specific memory optimization kicks in that breaks the regional prompting.

I did some tests on my 8GB 3070 with text-to-image mountain ADDCOL catgirl (no negative prompt) at 1820x1024, with the Regional Prompter settings at their defaults (so Divide Ratio = 1,1). Hypertile, Multidiffusion, NeverOOM are all inactive. The console shows no message about Tiled VAE being used.

A1111 / SD1.5 (revAnimated_v122): Works A1111 / SDXL (dreamshaperXL_v21TurboDPMSDE): Works

Forge / SD1.5: Works Forge / SDXL: Doesn't work - only shows a mountain. If I change the prompt to catgirl ADDCOL mountain, it only shows a catgirl. If I change the resolution to 1024x1024, it shows both (in the correct positions). If I change it to 2048x2048, it again only shows a catgirl.

There's also a ticket for this in the Regional Prompter repo, should communication with its author be needed: https://github.com/hako-mikan/sd-webui-regional-prompter/issues/295

[Heather95]

@arcusmaximus The author has already communicated with Regional Prompter's author. Forge's author said 2 weeks ago, "i will take a look soon and try to give more hacks to lora compute." It seems Latent mode does not work for now. hako-mikan/sd-webui-regional-prompter#299

[arcusmaximus]

Yes, I saw that. I should clarify that the tests I mentioned were done with Attention mode, not Latent mode - so, there are more problems than what's been discussed so far.

[zaqhack]

Just here from 515 (Closed). Thanks for noticing us, Senpai.

[catboxanon]

Retitled this issue to better reflect the actual enhancement request. Attention mode is supported (albiet currently broken, but will be fixed when this PR is merged upstream). Latent mode is not yet supported and will require work on Forge's end. See https://github.com/hako-mikan/sd-webui-regional-prompter/issues/299#issuecomment-1932310940 for further context.