zoom_enhance several issues, detailed testing.

[Dawgmastah]

This is sort of a combined issue, for reference, Im running the most recent version have disabled all other extensions except controlnet and the images browser and I have renamed the unprompted folder _unprompted as specified in reddit.

What Im finding is that I need a particular set of settings so the extension works, kind of well, and it will still not save the final combined image as it should.

(as an aside, sometimes the extension will start looping recursively on itself, it will start generating faces it finds on the new faces and keep for a while until cancelled, the result is something like an infinity mirror, I could not replicate this reliably, just aborted when I saw it)

Tests run (same setting, same seed, using the auto include in this prompt option for faster testing. Using no controlnet for starters, 640x960 image with hires fix upscale by 1.2, mask to find face and "Namehere face" as replacement

#1: Include original, unenhanced... - unchecked Include original image in out... - unchecked Unload txt2mask model after... - unchecked

Result: Zoomenhance will run and will generrate 2 images in output folder, the enhanced face, and the original image as if zoomenhance had not run, nothing on tmp folder chosen manually on settings.

#2: Include original, unenhanced... - checked Include original image in out... - unchecked Unload txt2mask model after... - unchecked

Result: Zoomenhance will run and will generate 2 images in output folder, the enhanced face, and the original image as if zoomenhance had not run. However, the final correct combined image will appear as previewable as a result, together with the uncombined image and can be saved with rightclick (pressing save will error out). The final combined image will be in the tmp folder chosen manually on settings

Using a batch size higher than 1 will do weird stuff, starting by it generating way more faces than needed, so processing time skyrockets:

-With a batch size of 2 in the output folder will be 4 enhanced faces, , with a pair looking... more degraded, as if it was run on the initial pair. (blurriness,color issues). The uncombined images will be on the output folder as well.

--With grids on, The output window in the webui will have a total of 7? images. A grid of the uncombined images (I dont recall how to turn this off), then each of them separately, then a grid of the first combined image correct and the other not,then a grid with the second image combined as well, the each image separately. It seems the images beyond #1 degrade further and further (the first image is fine, then the second is already horrendous). There are more faces generated in the out folder than used. Correct combined image can be found in the manually chosen tmp folder.

--With grids off, most of these issues go away (so, incompatibility with grids or maybe a patch needed?) However with a batch size of 2, the second image enhanced face is much worse than expected. If I go manually and retry that seed with batch size of 1, the result improves a ton, so I'm guessing there is something wrong batch size >1. Correct combined image can be found in the manually chosen tmp folder.

#3: Include original, unenhanced... - checked Include original image in out... - checked Unload txt2mask model after... - unchecked

Zoomenhance will run, 3 images will be present in the preview plane, two copies of the uncombined, and the combined. In the tmp folder there will be an uncombined and combined set, andin the output folder the UNcombined and the enhanced face

With a batchsize of 2 (grids were left off for this test), similar as above, but as noted, the 2nd face is much worse than the first.

#4: Include original, unenhanced... - unchecked Include original image in out... - checked Unload txt2mask model after... - unchecked

This produces sets of uncombined images everywhere, visualizer, output folder and tmp folder. Set of enhanced faces also present in output folder

Right now Ill set only the Include original, unenhanced checkmark, use batch size of 1, but use the generate forever button as a sort of a workaround on the batchsize issues, Ill also just grab my output files from the tmp folder.

It sometimes gets stuck on the recursiveness loop, and have to stop it manually though.

Also, I havent done extensive testing, but seems to not play nice with the new controlnets (or any help is appreciated on this). With the new openpose (full) noise is inserted along he face.

[p3iurhgmc98h]

Are you using the wizard, or entering commands manually? I've found that the wizard doesn't work reliably for zoom_enhance, and can even interfere with directly entered commands.

FYI the looping issue is much more noticeable when you have two consecutive zoom_enhance directives.

[Dawgmastah]

Im using the wizard with the "auto include in prompt", since it was way faster to test isntead of generating, copying and pasting

[ThereforeGames]

Hi @Dawgmastah,

Thank you for your detailed feedback!

I have released a new version of Unprompted that includes a couple speculative fixes for [zoom_enhance]:

1. The "auto-include" result will now include a conditional check for batch index (zoom_enhance only needs to run once for the entire batch):

[if batch_index=0]{after}{{zoom_enhance mask="face" replacement="face" mask_sort_method="left-to-right" upscale_method="Nearest Neighbor" downscale_method="Lanczos" blur_size=0.03 cfg_scale_min=3.0 denoising_max=0.65 mask_size_max=0.3 min_area=50.0 contour_padding=0.0 upscale_width=512.0 upscale_height=512.0}}{/after}[/if]

2. I updated the postprocess() routine to hopefully resolve the seemingly-infinite loops that could occur.

Feel free to let me know if you continue experiencing problems with this shortcode. Thank you!

[Dawgmastah]

I had issues having zoom_enhance to run at all, had to revert back

[freke70]

It no longer worls for me. IT will run once, then give these errors:

Running script process: /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py: AttributeError ╭──────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────╮ │ /home/me/automatic/modules/scripts.py:422 in process │ │ │ │ 421 │ │ │ │ script_args = p.script_args[script.args_from:script.args_to] │ │ ❱ 422 │ │ │ │ script.process(p, *script_args) │ │ 423 │ │ │ except Exception as e: │ │ │ │ /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:791 in process │ │ │ │ 790 │ │ for idx, unit in enumerate(params_group): │ │ ❱ 791 │ │ │ unit = self.parse_remote_call(p, unit, idx) │ │ 792 │ │ │ if not unit.enabled: │ │ │ │ /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:634 in parse_remote_call │ │ │ │ 633 │ │ │ │ ❱ 634 │ │ unit.enabled = selector(p, "control_net_enabled", unit.enabled, idx, strict=True │ │ 635 │ │ unit.module = selector(p, "control_net_module", unit.module, idx) │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ AttributeError: 'int' object has no attribute 'enabled' 100%|███████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 22.13it/s] Running script process: /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py: AttributeError ╭──────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────╮ │ /home/me/automatic/modules/scripts.py:422 in process │ │ │ │ 421 │ │ │ │ script_args = p.script_args[script.args_from:script.args_to] │ │ ❱ 422 │ │ │ │ script.process(p, *script_args) │ │ 423 │ │ │ except Exception as e: │ │ │ │ /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:791 in process │ │ │ │ 790 │ │ for idx, unit in enumerate(params_group): │ │ ❱ 791 │ │ │ unit = self.parse_remote_call(p, unit, idx) │ │ 792 │ │ │ if not unit.enabled: │ │ │ │ /home/me/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:634 in parse_remote_call │ │ │ │ 633 │ │ │ │ ❱ 634 │ │ unit.enabled = selector(p, "control_net_enabled", unit.enabled, idx, strict=True │ │ 635 │ │ unit.module = selector(p, "control_net_module", unit.module, idx) │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ AttributeError: 'int' object has no attribute 'enabled'

No idea what's going on, I tried to disable most extensions I don't need, but this keeps coming back. After it fails like this, it won't work at all anymore.

[ThereforeGames]

Hi @freke70,

Hmm, could you let me know what your ControlNet settings look like? Here are mine:

Do you have "Allow other script to control this extension" enabled?

[freke70]

Hi thanks, Everything was solved in a different issue. Version 9.01 made everything work. So, no more problems on my end. Thanks again!

[Dawgmastah]

On my end, 3 last versions fail to enhance the image with 9.01, zoomenhance runs, generates just an enhanced face and unenhanced image on the output folder, and unenhanced image on temp folder. (with option include original)

No controlnet, no hires

[Dawgmastah]

Scrolling on the console I found: (not sure at what point this happened)

File "X:\STABLEDIFFUSION\AUTOMATIC111\extensions_unprompted/shortcodes\stable_diffusion\zoom_enhance.py", line 373, in run_atomic fixed_image = self.Unprompted.shortcode_objects["img2img"].run_atomic(set_pargs,None,None) File "X:\STABLEDIFFUSION\AUTOMATIC111\extensions_unprompted/shortcodes\stable_diffusion\img2img.py", line 94, in run_atomic return img2img_images[0] IndexError: list index out of range

[ThereforeGames]

Hi @Dawgmastah,

Hmm, based on the error message, the img2img() procedure completed successfully, but it did not return an image object for some reason. I'll add a debug message in the next update so we can look at more diagnostic info.

There's another thing that stands out to me about your console output: your Unprompted path looks incorrect. It says extensions_unprompted as opposed to the expected extensions\_unprompted. It still appears to be reading the files in question, though. What OS are you on?

[Dawgmastah]

Hi Its windows 10, I renamed the folder to _umprompted, maybe this is why It has failed to output normally and I had to get the merged images from tmp folder? (an issue with the (_)

I also noticed an unusual backwards slash, is this expected?

X:\STABLEDIFFUSION\AUTOMATIC111\extensions_unprompted**/**shortcodes\

I am currently without access to the computer until saturday though, but I can help debug Sunday onwards

[ThereforeGames]

I see - I'm on Windows 11, the path handling is identical as far as I know. Linux is another story.

I also noticed an unusual backwards slash, is this expected?

No, I don't recall seeing that in my console before. I suppose the first thing to try is renaming your folder back to unprompted and give txt2img another shot (make sure to disable ControlNet for this test.)

[Sporking]

See also https://github.com/ThereforeGames/unprompted/issues/199 regarding the "batch size" effects mentioned in this bug description.