Support for Multi-Diffusion Upscaler

[LindezaBlue]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

I have spent the last 2 months working with other members and the developers of the extension "Multdiffusion-upscaler" and so far there is an insane amount of support to have this extension implemented or available for Forge users. There is so much requests for it the developers are getting flooded with requests to have it work... but from what I can tell it's not on their end, it seems forge has it hard locked out.

(link to authors post) https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/issues/350#issuecomment-2058241530

Extension in forge being locked: (cannot be enabled, and resets back to disabled upon restart.)

This extension is the reason a lot of people have not switched to Forge since it is highly memory efficient and works really well at upscaling and regional prompting. I myself am only use forge due to the speed it provides compared to A1111, however the only thing that I keep going back to A1111 is that there is no support for this one extension.

If at all possible could this be added as a usable extension?

Thanks!~

Proposed workflow

• Enable it in the backend so that the extension can be used and worked on by the authors of the extension.

Additional information

Link to Extension: https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111

[Vigilence]

Agreed!

[PeterLorre]

Would LOVE for this to work!

[muwaaaah]

Me too, I can't use multidiffusion upscale!!! The only thing that i need to run a1111 is Multidiffusion upscale.

[Vigilence]

Only a stripped down version is integrated, which doesn't function the same as the original. Something is better then nothing, but atm the original author and friends want to improve the extension for forge and can't since its being blocked.

---

From: Nathan Harris @.> Sent: Wednesday, April 17, 2024 10:28 PM To: lllyasviel/stable-diffusion-webui-forge @.> Cc: Vigilence @.>; Comment @.> Subject: Re: [lllyasviel/stable-diffusion-webui-forge] Support for Multi-Diffusion Upscaler (Issue #672)

Isn't this already integrated?

Screenshot.2024-04-17.192728.png (view on web)https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/648059/841d9502-1672-4aa3-8e87-43bc4910229a

— Reply to this email directly, view it on GitHubhttps://github.com/lllyasviel/stable-diffusion-webui-forge/issues/672#issuecomment-2062878750, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFNZ5AVUHIYBGVQM5NF67EDY54VUTAVCNFSM6AAAAABGKKRKDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRSHA3TQNZVGA. You are receiving this because you commented.Message ID: @.***>

[LindezaBlue]

I would be nice to not have the stripped down version for sure. I've tried it and it doesn't give results that are even remotely good. I also want to be able to use the region control that comes with Multi-Diffusion. That thing is a God send!

[altoiddealer]

This 100% should happen, unfortunately I haven't seen any evidence that Forge development even has a pulse right now?

[LindezaBlue]

This 100% should happen, unfortunately I haven't seen any evidence that Forge development even has a pulse right now?

Like they haven't worked on any Commits/Updates to the program?

[altoiddealer]

If you go to Discussions tab you’ll see people have been asking repeatedly for a dev to make any sort of comment about development status but reaponse is nothing but crickets

[LindezaBlue]

If you go to Discussions tab you’ll see people have been asking repeatedly for a dev to make any sort of comment about development status but response is nothing but crickets

Well, if that is true/the case atm, that just sucks. We really need a webui that renders fast like Forge and that has support for all A1111 extensions. If support is indeed dropped, I hope someone else picks up the pieces and develops a new webui that's actually maintained.

[nosdregamon]

I just searched for the better part of an hour, why this extension didn't show up (without a noticeable error message). Would be to bad, if Forge couldn't run this.

[LindezaBlue]

I just searched for the better part of an hour, why this extension didn't show up (without a noticeable error message). Would be to bad, if Forge couldn't run this.

It was the same for me, and that's why I'm here, also bouncing back and forth between the discussions for the extension and hopefully Forge as well. But it seems like Forge Developers are dead quite atm. Not even a peep.

[GuruVirus]

I just searched for the better part of an hour, why this extension didn't show up (without a noticeable error message). Would be to bad, if Forge couldn't run this.

It was the same for me, and that's why I'm here, also bouncing back and forth between the discussions for the extension and hopefully Forge as well. But it seems like Forge Developers are dead quite atm. Not even a peep.

The author has been working on this repo instead this week: https://github.com/lllyasviel/IC-Light

[Vigilence]

This doesn’t need the Illyasviel. It just needs the code blocking the extension removed, which hopefully anyone why the know how can do.

[NikkeTryHard]

+999999999999999999

[nosdregamon]

This doesn’t need the Illyasviel. It just needs the code blocking the extension removed, which hopefully anyone why the know how can do.

To achieve this, disable the inbuilt extension in the settings. Apply and close Forge.

Then open "modules_forge/config.py" with a text-editor. The file only has 4 lines, so it will not be complicated.

Change

always_disabled_extensions = [
    'sd-webui-controlnet',
    'multidiffusion-upscaler-for-automatic1111'
]

to

always_disabled_extensions = [
    'sd-webui-controlnet',
    # 'multidiffusion-upscaler-for-automatic1111'
]

(the # makes Python ignore the whole line) Then you can start Forge and install/run the full extension. No clue how smooth this will run.

[realmalikov]

Native upscaler support would be highly appreciated.

[miro-ux]

@nosdregamon this does seem to allow you to enable the extension but the tab is still nowhere to be found under img2img:

(Edit) Errors from console:

ControlNet preprocessor location: c:\SD\webui\models\ControlNetPreprocessor
*** Error loading script: tilediffusion.py
    Traceback (most recent call last):
      File "c:\SD\webui\modules\scripts.py", line 525, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "c:\SD\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\tilediffusion.py", line 71, in <module>
        from tile_methods.abstractdiffusion import AbstractDiffusion
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_methods\abstractdiffusion.py", line 1, in <module>
        from tile_utils.utils import *
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_utils\utils.py", line 16, in <module>
        from tile_utils.typing import *
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_utils\typing.py", line 9, in <module>
        from ldm.models.diffusion.ddpm import LatentDiffusion
    ModuleNotFoundError: No module named 'ldm'

---
*** Error loading script: tileglobal.py
    Traceback (most recent call last):
      File "c:\SD\webui\modules\scripts.py", line 525, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "c:\SD\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\tileglobal.py", line 13, in <module>
        from tile_methods.abstractdiffusion import AbstractDiffusion
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_methods\abstractdiffusion.py", line 1, in <module>
        from tile_utils.utils import *
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_utils\utils.py", line 16, in <module>
        from tile_utils.typing import *
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_utils\typing.py", line 9, in <module>
        from ldm.models.diffusion.ddpm import LatentDiffusion
    ModuleNotFoundError: No module named 'ldm'

---
*** Error loading script: tilevae.py
    Traceback (most recent call last):
      File "c:\SD\webui\modules\scripts.py", line 525, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "c:\SD\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "c:\SD\webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\tilevae.py", line 68, in <module>
        from ldm.modules.diffusionmodules.model import AttnBlock, MemoryEfficientAttnBlock
    ModuleNotFoundError: No module named 'ldm'

[mr-september]

@nosdregamon this does seem to allow you to enable the extension but the tab is still nowhere to be found under img2img:

+1 same here. This is honestly so childish and unlike @lllyasviel's usual behaviour. He just keeps closing issues mentioning this without resolutions nor explanations. I'd rather be able to do my work 20% slower than not do it at all. Can only leave Forge.