[Bug]: Color quality loss when adetailer is fed the image using skip img2img

[mprz1024]

Checklist

• [ ] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [X] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

The adetailer extension can be used in img2img with an option to skip the img2img process, just to inpaint hands or whatever you need inpainted, and inpainting only the detected area. When you do that, even with the option to inpaint only masked, the image fed to it will have some visible colour saturation loss across all the image. I reported this bug in adetailer but they claim it's the image adetailer is being fed, not a bug in adetailer ( https://github.com/Bing-su/adetailer/issues/524 ).

The issue doesn't happen if you let img2img happen, or if you use inpainting manually

Original image:

With [X] Skip img2img: red colours damaged:

Steps to reproduce the problem

1. Produce an image with saturated colours, and take it to img2img.
2. With the Bing-su adetailer extension installed, enable it in img2img, click skip img2img, use a model to detect whatever you want to enhance, make sure inpaint only masked is checked, and generate.
3. Compare the result with the original image: colour loss.

What should have happened?

Colours not altered outside inpainted areas.

What browsers do you use to access the UI ?

Google Chrome

Sysinfo

Will attach

Console logs

################################################################
Launching launch.py...
################################################################
Python 3.11.8 (main, Mar 26 2024, 12:26:07) [GCC 13.2.0]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
#######################################################################################################
Initializing Civitai Link
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.11.8 (main, Mar 26 2024, 12:26:07) [GCC 13.2.0]
Civitai Link revision: 115cd9c35b0774c90cb9c397ad60ef6a7dac60de
SD-WebUI revision: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5

Checking Civitai Link requirements...

#######################################################################################################
Check mmcv version...
Your mmcv version 2.0.1 may not work with mmyolo.
or you need to fix version restriction of __init__.py of mmyolo manually, to use mmcv 2.1.0 with mmyolo.
Check mmengine version...
your mmengine version is 0.8.5
Launching Web UI with arguments: -f --port 7777 --listen --enable-insecure-extension-access --theme dark --unload-gfpgan --skip-load-model-at-start --xformers --skip-torch-cuda-test --no-download-sd-model
==============================================================================
You are running torch 2.0.1+cu118.
The program is tested to work with torch 2.1.2.
To reinstall the desired version, run with commandline flag --reinstall-torch.
Beware that this will cause a lot of large files to be downloaded, as well as
there are reports of issues with training tab on the latest version.

Use --skip-version-check commandline argument to disable this check.
==============================================================================
=================================================================================
You are running xformers 0.0.20.
The program is tested to work with xformers 0.0.23.post1.
To reinstall the desired version, run with commandline flag --reinstall-xformers.

Use --skip-version-check commandline argument to disable this check.
=================================================================================
Civitai Helper: Get Custom Model Folder
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 24.3.5, num models: 24
ControlNet preprocessor location: /opt/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2024-03-29 16:36:08,421 - ControlNet - INFO - ControlNet v1.1.442
2024-03-29 16:36:08,491 - ControlNet - INFO - ControlNet v1.1.442
Civitai: API loaded
 Total 8 mmdet, 0 yolo and 3 mediapipe models.
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/bbox/face_yolov8n.pth!
 You can enable model validity tester in the Settings-> μ DDetailer.
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/bbox/face_yolov8s.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/bbox/hand_yolov8n.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/bbox/hand_yolov8s.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/bbox/mmdet_anime-face_yolov3.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/segm/mmdet_dd-person_mask2former.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/segm/yolov5_ins_n.pth!
SUCCESS - success to load config for /opt/stable-diffusion-webui/models/mmdet/segm/yolov5_ins_s.pth!
 Total 8 valid mmdet configs are found.
 You can disable validity tester in the Settings-> μ DDetailer.
Check config files...
Done
loading.. cn_modole
 - ControlNet extension sd-webui-controlnet found: scripts.external_code
 - default µ DDetailer model= bbox/face_yolov8n.pth [882ebbb6]
2024-03-29 16:36:10,453 - ControlNet - INFO - ControlNet UI callback registered.
 - default µ DDetailer model= bbox/face_yolov8n.pth [882ebbb6]
Civitai Helper: Set Proxy: 
Loading weights [6e8e316b68] from /opt/stable-diffusion-webui/models/Stable-diffusion/Anime-DivineE_9__CS2_S30_Euler_CFG8.safetensors
Creating model from config: /opt/stable-diffusion-webui/configs/v1-inference.yaml
Applying attention optimization: xformers... done.
Model loaded in 0.8s (create model: 0.3s, apply weights to model: 0.4s).
Running on local URL:  http://0.0.0.0:7777

To create a public link, set `share=True` in `launch()`.
Civitai: Check resources for missing info files
Civitai: Check resources for missing preview images
Startup time: 12.5s (prepare environment: 3.0s, import torch: 2.2s, import gradio: 0.6s, setup paths: 0.9s, initialize shared: 0.1s, other imports: 0.3s, load scripts: 2.2s, scripts before_ui_callback: 0.9s, create ui: 1.8s, gradio launch: 0.3s).
Civitai: Found 0 resources missing preview images
Civitai: No preview images found on Civitai
Civitai: Found 0 resources missing info files
Civitai: No info found on Civitai
Authorization required, but no authorization protocol specified

img2img: hand
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  5.58it/s]
Total progress:   0%|                                                                                                                   | 0/1 [00:00<?, ?it/s]
0: 384x640 2 hands, 4.9ms
Speed: 1.4ms preprocess, 4.9ms inference, 0.8ms postprocess per image at shape (1, 3, 384, 640)
tiled upscale: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 13.11it/s]
tiled upscale: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:02<00:00, 13.38it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.60it/s]
tiled upscale: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 13.37it/s]
tiled upscale: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:02<00:00, 13.33it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.60it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:20<00:00, 20.07s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:20<00:00, 20.07s/it]

Additional information

This happened using 2 different nVidia GPUs and 3 different driver versions, including the latest at the time of writing.

[light-and-ray]

Because it actually doesn't skip img2img, it uses 1 step with 0 denoising, as I know, and the image still goes through vae

[light-and-ray]

Fuck stop pasting the whole sys info inside issues! 😡

[mprz1024]

Because it actually doesn't skip img2img, it uses 1 step with 0 denoising, as I know, and the image still goes through vae

Oh so it's a VAE artifact of going in and out? I'm surprised it's not more noticeable. Is there a way this could have been done better in adetailer, so that it doesn't go through img2img for real and just straight into inpainting?

The extension does complain if you try to use it with skip from the inpainting tab, but there's a bigger interest in this which is to fix whole batches of poorly-drawn hands. In my use case, I have A1111 generate a hundred pictures (with dynamic prompts, variations, etc.) then I select the 10% best, but some of them have bad hands and I want to mass fix them without running adetailer on the 100% of unselected pictures from the start.

Moved sysinfo here, sorry: sysinfo.json