[Bug]: Multidiffusion integrated + controlnet integrated errors

[GodEmperor785]

Checklist

• [ ] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [X] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

I checked multiple possible combinations of Multidiffusion Integrated + Controlnet Integrated (tile model) and there are some combinations that fail with errors. Lets start with what works:

1. Multidiffusion Integrated works well without controlnet with both SD1.5 and SDXL
2. MultiDiffusion (method name) + SD1.5 + controlnet tile_resample - works well

However other combinations cause problems, from what I checked:

1. Mixture of Diffusers + SD1.5 + controlnet any preprocessor = "error_1"
2. MultiDiffusion + SD1.5 + controlnet tile_colorfix+sharp = "error_2"
3. MultiDiffusion/Mixture of Diffusers + SDXL + controlnet any preprocessor = "error_3"

I'm adding console logs as files to this issue (in "Additional information" section).

In the extension for normal webui (https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) problematic scenarios 1 and 2 worked fine - multidiffusion extension worked on SD1.5 with all combinations of controlnet

For SDXL controlnet tile I used these: https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite models named "bdsqlsz_controlllite_xl_tile_anime_X" - they seem to work fine in Ultimate SD Upscale extension For SD1.5 I used the standard controlnet tile model "control_v11f1e_sd15_tile"

In Multidiffusion Integrated for SDXL I set tile size = 1024, other settings were left at defaults. For SD1.5 all settings were left at defaults.

All tests were made in img2img tab with processing image upscaled in extras with "resize by" = 1, image size was 2304x3456

There were some other issues made for multidiffusion extensions but none seems to describe current situation

Can we expect support for mentioned scenarios (at least for those that worked in original extension)?

Also I noticed that slider "Tile Batch Size" in Multidiffusion Integrated has allowed range of values from 1 to 8192, this looks like a much too big number for batch size. Or am I missing something and this is intentional?

If it matters I installed webui-forge from the "one-click installation package" in readme file (I also ran update.bat so forge version should be up to date)

Steps to reproduce the problem

1. Go to img2img
2. Add input image
3. set "resize by" to 1 (this shouldn't matter actually, I just did testing with this number)
4. enable Controlnet Integrated with tile model (preprocessors described in "What happened?" section)
5. enable Multidiffusion Integrated and choose method as described in "What happened?" section
6. generate image

What should have happened?

Process the image with Multidiffusion Integrated + Controlnet Integrated (tile model) without errors and produce output image

What browsers do you use to access the UI ?

Brave

Sysinfo

sysinfo-2024-02-11-12-43.json

Console logs

Console logs added as files in "Additional information" section, let me know if I should instead just paste them here

Additional information

error_1-mixture_sd15_controlnet.txt error_2-multidiffusion_sd15_controlnet_tile_colorfix+sharp.txt error_3-sdxl.txt

[Zhincore]

I have the same issues :(

[Adahm83]

I have noticed that "MultiDiffusion Integrated" with "ControlNet Integrated" error depends on the resolution of the image.

Example with SDXL model

• Txt2Img image output is 1024x1344 (model used: RealCartoonXL_v6)
• Img2Img tab settings: MDIntegrated = Enabled, ControlNet = Enabled / Tile, (preprocessor: tile_resample, Model: bdsqlsz_controlllite_xl_tile_realistic)

a) Resize by: 2 (from 1024x1344 to 2048x2688) -> will result in error: TypeError: 'NoneType' object is not iterable b) Resize by: 1.5 (from 1024x1344 to 1536x2016) -> will work !! c) anything other scale ratio will result in TypeError: 'NoneType' object is not iterable, both higher or lower values (e.g 1.2/1.3/1.4/1.6/..../2.8/3.0/4.0) d) disabling MD extension and leaving only CN enabled will result in TypeError: 'NoneType' object is not iterable e) disabling CN extension and leaving only MD will work

So, the problem is between "CN" extension and output resolution / scaling ratio, not MD.

Edit: ....after further testing I found that it's also related to Denoising strength !?! but not on Steps. Meaning that what I described in point b) Resize by: 1.5 (from 1024x1344 to 1536x2016) -> was a PASS with 40 steps, and denoise: 0.5 (DPM++ 2M Karras)

• changed denoise to 0.8 -> FAIL (also 0.7/ ... /0.64 FAIL), minimum value 0.63 -> PASS
• increased steps to 100 -> PASS (with denoise 0.63)

I hope this will help find the source of the issue.

[Zhincore]

I don't think it's dependent on parameters, the real error is AttributeError: 'dict' object has no attribute 'get_control' (at least for me and OP), which seems to be a programming error to me

[Adahm83]

I don't think it's dependent on parameters, the real error is AttributeError: 'dict' object has no attribute 'get_control' (at least for me and OP), which seems to be a programming error to me

Well... my testing lead to different conclusion. And I was talking about TypeError: 'NoneType' object is not iterable which seem to be the same as attached *.txt examples. This error depends on mentioned parameters, at least in my environment.