[Feature Request]: Add Kohya's Deep Shrink Hires. fix

[CCpt5]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Add an option to use a new method for "hires fixing" as discovered/outlined by Kohya (training master) on twitter on November 13, 2023.

https://twitter.com/kohya_tech/status/1724273551937786164 [Announcement of discovery/breakdown] https://gist.github.com/kohya-ss/3f774da220df102548093a7abc8538ed [Code]

Translated tweets:

Kohya Tech - @kohya_tech @kohya_tech - Nov 14 - [Attached photos] Yesterday, I tried to find a method to prevent the composition from collapsing when generating high resolution images. The first attached image is 4 images normally generated at 2688x1536, and the second image is generated by applying the same seed. Highres fix is ​​not applied in either case. Taverns and towns are usually pretty good, but I think you'll find that they're more stable when applied.

--

The idea is extremely simple; we know that composition is determined by timesteps that are close to noise'' and composition is determined by deep parts of U-Net,'' so we just reduce the latents in that part. I have placed the differences realized by the sd-scripts generation script."

--

Proposed workflow

1. Go to ....
2. Press ....
3. ...

Additional information

Extension even would be great if anyone reading this is savvy in that regard.

[wcde]

Another way to “make mutants faster”...

[catboxanon]

Note that this only applies to SDXL, at least currently. https://twitter.com/kohya_tech/status/1724275270582956443 https://twitter.com/kohya_tech/status/1724275005213536478

ComfyUI node if anyone wants to try it out for the time being. Seems to work for SD and SDXL. https://gist.github.com/laksjdjf/487a28ceda7f0853094933d2e138e3c6

Edit: Changed issue title to reflect name. https://twitter.com/kohya_tech/status/1724558590185513013

[wcde]

I implemented it as extension, it works with 1.5 too. https://github.com/wcde/sd-webui-kohya-hiresfix

[AndreyRGW]

I implemented it as extension, it works with 1.5 too. https://github.com/wcde/sd-webui-kohya-hiresfix

You are straight up the fastest hand in the neural west!

[AndreyRGW]

I implemented it as extension, it works with 1.5 too. https://github.com/wcde/sd-webui-kohya-hiresfix

I don't understand how this script works yet. Do I have to have the normal hires fix enabled or not? Without the normal hires fix, the pictures change as if they were seeded differently, but the resolution is not increased.

[wcde]

@AndreyRGW just set resolution whatever you want and generate with enabled extension.

[AndreyRGW]

@AndreyRGW just set resolution whatever you want and generate with enabled extension.

Would it be better if you could add the ability to disable this extension for the second pass (hires fix)? For example, generate at 512x768 resolution in the first pass and disable this extension in the second pass to avoid distortions?

upd: like controlnet:

[2blackbar]

this conflicts with adetiler and makes adetailer corrupt faces best settings so far for me for stop 6 and 14, depth 4 and 6,scale 1

[Scr4tchproof]

this conflicts with adetiler and makes adetailer corrupt faces best settings so far for me for stop 6 and 14, depth 4 and 6,scale 1

Thanks, that fixed my confetti issue (even gen was getting confetti cause of the noise).