stable-diffusion-webui-forge icon indicating copy to clipboard operation
stable-diffusion-webui-forge copied to clipboard

Published 20 hours ago • verified publisher icon lllyasviel

Add back ControlNet multi-inputs upload tab

Open huchenlei opened this issue 1 year ago • 22 comments

Description

Closes https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/98

IP-Adapter face_id and photomaker verified to be working with multi-inputs. InstantID not working with multi-inputs now. @lllyasviel PTAL if we can somehow make multi-inputs work for InstantID. In sd-webui-controlnet, the approach is approximation via averaging the insightface embedding.

Screenshots/videos:

image

Checklist:

huchenlei avatar Feb 15 '24 04:02 huchenlei

Thanks!

strawberrymelonpanda avatar Feb 15 '24 08:02 strawberrymelonpanda

this will be taken a look later since it needs more considerations

lllyasviel avatar Feb 17 '24 16:02 lllyasviel

this will be taken a look later since it needs more considerations

Please move the Batch Folder and Batch Upload together under a Batch tab and restore the Multi Input tab to avoid confusion. It is very important for IP Adapters like FaceID Portrait which requires 5 input images.

StellarBeing25 avatar Feb 17 '24 17:02 StellarBeing25

this will be taken a look later since it needs more considerations

Please move the Batch Folder and Batch Upload together under a Batch tab and restore the Multi Input tab to avoid confusion. It is very important for IP Adapters like FaceID Portrait which requires 5 input images.

also my (and many others) ip adapter face id workflow requires a set of image as base. maybe show multi-input only, where it actually works -if that's the problem?

a3nima avatar Feb 17 '24 18:02 a3nima

I've been using the PR - it works great, exactly as expected. Please consider merging this soon, it's very useful for IP Adapter workflows.

strawberrymelonpanda avatar Feb 18 '24 00:02 strawberrymelonpanda

Like to also report this PR is working great in conjunction with Face ID IP Adaptor, greatly improved likeness using multiple source images.

saunderez avatar Feb 20 '24 08:02 saunderez

Was trying to use instant id today when i noticed I was unable to use multiple image inputs like I can on the regular controlnet extension.

Why remove?

ScionoicS avatar Feb 20 '24 21:02 ScionoicS

@sohamaa in #344 raises a valid point: The "Multiple Images" tab is shown in the screenshots of the README.

image

strawberrymelonpanda avatar Feb 21 '24 03:02 strawberrymelonpanda

I added it back to my own installation.

934853942985934

Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

CRCODE22 avatar Feb 21 '24 05:02 CRCODE22

I added it back to my own installation.

Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

JaB43a avatar Feb 25 '24 00:02 JaB43a

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

Everything you need to know is at the link below.

https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

saunderez avatar Feb 25 '24 12:02 saunderez

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

526453453425123423

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

CRCODE22 avatar Feb 26 '24 11:02 CRCODE22

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them. I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images. Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

526453453425123423

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)

Thank you anyway :)

JaB43a avatar Feb 26 '24 18:02 JaB43a

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them. I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images. Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files 526453453425123423 Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)

Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces.

Here it does work:

Moving model(s) has taken 0.68 seconds 2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. To load target model SDXL

CRCODE22 avatar Feb 27 '24 03:02 CRCODE22

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them. I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images. Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files 526453453425123423 Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears) Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces.

Here it does work:

Moving model(s) has taken 0.68 seconds 2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. To load target model SDXL

I haven't used Photomaker much, the one I use is InstantID, and it is with InstantID that it doesn't work.

JaB43a avatar Feb 27 '24 10:02 JaB43a

Please consider the merge, it is extremely useful for FaceID ipadapter and InstantID

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them. I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images. Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files 526453453425123423 Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears) Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces. Here it does work: Moving model(s) has taken 0.68 seconds 2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. To load target model SDXL

I haven't used Photomaker much, the one I use is InstantID, and it is with InstantID that it doesn't work.

It is working for me with InstantID though

banjaminicc avatar Feb 27 '24 16:02 banjaminicc

Multi-image tab doesn't seem to work on inpainting (possibly all of img2img). The tab shows, but it doesn't have any effect on the image.

webinksupport avatar Feb 28 '24 13:02 webinksupport

this will be taken a look later since it needs more considerations

why is it not implented yet?

a3nima avatar Mar 06 '24 13:03 a3nima

Please implement, this is such a good quality of life setting

lhucklen avatar Mar 09 '24 06:03 lhucklen

I would also like to see this re-implemented. I'm very surprised it was removed in the first place.

PeanutButterSkeletonCrew avatar Mar 15 '24 10:03 PeanutButterSkeletonCrew

I added it back to my own installation. Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them. I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images. Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

526453453425123423

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

While this still works, it would be nice to have the multi upload function AND the "use mask" function working together. I have to switch to single image to get the mask drawing/use mask feature atm. which makes them unusable together.

a3nima avatar Mar 23 '24 15:03 a3nima

Any updates on this? @lllyasviel

lhucklen avatar Mar 23 '24 19:03 lhucklen

Btw I've found that with the code provided by @huchenlei I do get an error, when trying to use controlnet inpaint. multi-inputs for ip-adapter still works though.

a3nima avatar Mar 27 '24 14:03 a3nima

I really need multiple inputs for Face ID. Really hope this can be added back.

gohan2091 avatar May 01 '24 15:05 gohan2091

huchenlei merged commit 61a2a9d into lllyasviel:dev2

I've been asking for this pretty much since Forge launched, ha. Thanks @huchenlei!👍

strawberrymelonpanda avatar May 03 '24 05:05 strawberrymelonpanda

huchenlei merged commit 61a2a9d into lllyasviel:dev2

I've been asking for this pretty much since Forge launched, ha. Thanks @huchenlei!👍

I just tried it on dev2 branch and I can't get controlnet/ipadapter to work with multi-inputs. In fact I can't get ipadapter to work at all on the dev2 branch even with a single image input.

GatorsUF avatar May 03 '24 13:05 GatorsUF

i get this error

*** Error loading script: controlnet.py Traceback (most recent call last): File "G:\SD F\sd forge\webui\modules\scripts.py", line 525, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "G:\SD F\sd forge\webui\modules\script_loading.py", line 13, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "G:\SD F\sd forge\webui\extensions-builtin\sd_forge_controlnet1\scripts\controlnet.py", line 15, in from lib_controlnet.utils import ( File "G:\SD F\sd forge\webui\extensions-builtin\sd_forge_controlnet1\lib_controlnet\utils.py", line 9, in from modules_forge.forge_util import HWC3 ModuleNotFoundError: No module named 'modules_forge.forge_util'

tealytee avatar Aug 10 '24 17:08 tealytee