[Bug]: Inpainting, desaturated inpainted area, possibly darkening inpainted area

[gitmylo]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

I inpainted an image, and it looked fine color wise in the live preview, but once the image gets outputted there is a discoloration on the inpainted area, the area is slightly less (but still noticably) saturated. When using a mask from a file, it appear to be even darker. example image where i inpainted just the eyes.

Steps to reproduce the problem

1. Go to img2img
2. Go to Inpaint
3. Upload an image
4. Draw a mask on the image
5. Hit Generate
6. Look at the output image, The masked area is discolored.

What should have happened?

Just like in the live preview, The inpainted area should not be desaturated, but consistent with the rest of the image.

It looks like what's happening is that somehow the inpainting mask is pasted over the image after generating

Commit where the problem happens

22bcc7be428c94e9408f589966c2040187245d81

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Mozilla Firefox, Google Chrome

Command Line Arguments

--xformers --no-half-vae --disable-safe-unpickle --api --cors-allow-origins=localhost

List of extensions

I have the following extensions, however the same bug occurs when i have all of them disabled:

• ABG_extension
• Abysz-LAB-Ext
• a1111-sd-webui-lycoris
• a1111-sd-webui-tagcomplete
• asymmetric-tiling-sd-webui
• ebsynth_utility
• infinite-zoom-automatic1111-webui
• loopback_scaler
• multidiffusion-upscaler-for-automatic1111
• openOutpaint-webUI-extension
• posex
• sd-dynamic-prompts
• sd-webui-controlnet
• sd-webui-cutoff
• sd-webui-keyframer
• sd-webui-llul
• sd-webui-model-converter
• sd-webui-riffusion
• sd-webui-stopmotion
• sd-webui-supermerger
• sd-webui-text2video
• sd_dreambooth_extension
• shift-attention
• stable-diffusion-webui-anti-burn
• stable-diffusion-webui-prompt-travel
• stable-diffusion-webui-tokenizer
• stable-diffusion-webui-two-shot
• ultimate-upscale-for-automatic1111
• unprompted
• video_loopback_for_webui

Console logs

Already up to date.
venv "B:\stable-diffusion-webui2\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Installing torch and torchvision
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu117
Requirement already satisfied: torch==1.13.1+cu117 in b:\stable-diffusion-webui2\venv\lib\site-packages (1.13.1+cu117)
Requirement already satisfied: torchvision==0.14.1+cu117 in b:\stable-diffusion-webui2\venv\lib\site-packages (0.14.1+cu117)
Requirement already satisfied: typing-extensions in b:\stable-diffusion-webui2\venv\lib\site-packages (from torch==1.13.1+cu117) (4.5.0)
Requirement already satisfied: requests in b:\stable-diffusion-webui2\venv\lib\site-packages (from torchvision==0.14.1+cu117) (2.28.2)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in b:\stable-diffusion-webui2\venv\lib\site-packages (from torchvision==0.14.1+cu117) (9.4.0)
Requirement already satisfied: numpy in b:\stable-diffusion-webui2\venv\lib\site-packages (from torchvision==0.14.1+cu117) (1.23.3)
Requirement already satisfied: certifi>=2017.4.17 in b:\stable-diffusion-webui2\venv\lib\site-packages (from requests->torchvision==0.14.1+cu117) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in b:\stable-diffusion-webui2\venv\lib\site-packages (from requests->torchvision==0.14.1+cu117) (1.26.15)
Requirement already satisfied: idna<4,>=2.5 in b:\stable-diffusion-webui2\venv\lib\site-packages (from requests->torchvision==0.14.1+cu117) (2.10)
Requirement already satisfied: charset-normalizer<4,>=2 in b:\stable-diffusion-webui2\venv\lib\site-packages (from requests->torchvision==0.14.1+cu117) (3.1.0)

[notice] A new release of pip available: 22.2.1 -> 23.0.1
[notice] To update, run: B:\stable-diffusion-webui2\venv\Scripts\python.exe -m pip install --upgrade pip
Installing xformers
Installing requirements for Web UI
Installing None
Installing onnxruntime-gpu...
Installing None
Installing opencv-python...
Installing None
Installing Pillow...

current transparent-background 1.2.3

Installing requirements 1 for Infinite-Zoom

Stop Motion CN - Running Preload
Set Gradio Queue: True

Installing sd-dynamic-prompts requirements.txt


Initializing Riffusion



Stop Motion CN - Running Preload
Set Gradio Queue: True
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: 926ae204ef5de17efca2059c334b6098492a0641
Successfully installed accelerate-0.18.0 fastapi-0.94.1 gitpython-3.1.31 google-auth-oauthlib-0.4.6 transformers-4.26.1

Does your project take forever to startup?
Repetitive dependency installation may be the reason.
Automatic1111's base project sets strict requirements on outdated dependencies.
If an extension is using a newer version, the dependency is uninstalled and reinstalled twice every startup.

# Your version of xformers is 0.0.16rc425.
# xformers >= 0.0.17.dev is required to be available on the Dreambooth tab.
# Torch 1 wheels of xformers >= 0.0.17.dev are no longer available on PyPI,
# but you can manually download them by going to:
https://github.com/facebookresearch/xformers/actions
# Click on the most recent action tagged with a release (middle column).
# Select a download based on your environment.
# Unzip your download
# Activate your venv and install the wheel: (from A1111 project root)
cd venv/Scripts
activate
pip install {REPLACE WITH PATH TO YOUR UNZIPPED .whl file}
# Then restart your project.
[!] xformers version 0.0.16rc425 installed.
[+] torch version 1.13.1+cu117 installed.
[+] torchvision version 0.14.1+cu117 installed.
[+] accelerate version 0.18.0 installed.
[+] diffusers version 0.14.0 installed.
[+] transformers version 4.26.1 installed.
[+] bitsandbytes version 0.35.4 installed.



Installing requirements for Unprompted - img2pez
Installing requirements for Unprompted - pix2pix_zero

Launching Web UI with arguments: --xformers --no-half-vae --disable-safe-unpickle --api --cors-allow-origins=localhost --reinstall-torch --reinstall-xformers
Stop Motion CN - Running Preload
Set Gradio Queue: True
Loading Unprompted v7.9.1 by Therefore Games
(SETUP) Initializing Unprompted object...
(SETUP) Loading configuration files...
(SETUP) Debug mode is False
Loading weights [212cdc3715] from B:\stable-diffusion-webui2\models\Stable-diffusion\Merges\aaomi-anything-abyssorangemix-inpainting.ckpt
Creating model from config: B:\stable-diffusion-webui2\configs\v1-inpainting-inference.yaml
LatentInpaintDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.54 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(3): badhands5, badprompt, easynegative
Model loaded in 15.1s (load weights from disk: 9.1s, create model: 0.5s, apply weights to model: 1.4s, apply half(): 1.5s, move model to device: 1.2s, load textual inversion embeddings: 1.5s).
LAB
CUDA SETUP: Loading binary B:\stable-diffusion-webui2\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.dll...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 35.1s (import torch: 4.3s, import gradio: 0.8s, import ldm: 0.4s, other imports: 1.0s, load scripts: 6.3s, load SD checkpoint: 15.4s, create ui: 6.1s, gradio launch: 0.2s, scripts app_started_callback: 0.4s).
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:03<00:00,  4.05it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 16/16 [00:03<00:00,  4.73it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 16/16 [00:03<00:00,  5.39it/s]

Additional information

No response

[gitmylo]

forgot to mention, when using inpaint upload, the image becomes really dark, and also is during the generation with live preview.

[gitmylo]

the slight discoloration was caused by not using a vae, whoops.

however, there's still an issue when using the mask from an uploaded file, as i said, the image becomes really dark, this is also during the generation. it does this on masked content fill, but not on original.

you can see this when you put the denoising on 0, on fill, it's a really dark version of the original. is this intended behavior?

[msampson3d]

I do think there is still a problem introduced some time after the gradio upgrade. I'm seeing different behavior in img2img and inpainting compared to commits a few months back.

During inpainting, the entire image will sometimes shift in overall brightness over the course of the generation. The generated portion will match, but when the inpainted portion gets applied to the original image, it will be darker or lighter than the original image since the generation was applied a version of the image with an altered brightness.

No idea when this got introduced, but it's a major downgrade in inpainting.

[JasXSL]

I'm definitely still having this issue. Lots of light mismatch halos around anything I inpaint

[gitmylo]

I'm definitely still having this issue. Lots of light mismatch halos around anything I inpaint

Make sure you're using the appropriate VAE for the model, with the right VAE you won't be able to notice the discoloration.