[Bug]: inpanting model 2.0 add in the end stages latend nothing

[rewixx]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

os windows browser chrome runing https://github.com/TheLastBen/fast-stable-diffusion

the new 2.0 model gives in the inpanted areas returns latend nothig with faint image

Steps to reproduce the problem

1. get inpanting model https://huggingface.co/stabilityai/stable-diffusion-2-inpainting
2. put in the model folder for image generating from text
3. inpaint a part of a image put a promt
4. generare

What should have happened?

the inpainting result should be less like latend nothing

Commit where the problem happens

https://github.com/TheLastBen/fast-stable-diffusion

What platforms do you use to access UI ?

Other/Cloud

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

No response

Additional information, context and logs

[mclsugi]

I don't have 2.0 inpainting model, but I also have this problem with inpainting model v1.5 on my end with both latent masking.

@ThereforeGames , perhaps this got something to do with the neglected processing.py in your merge-pull that you've mentioned in other issue?

[rewixx]

it has less latend nothing

[ThereforeGames]

@mclsugi I'm late to this party, but yes, it looks like the same issue I mentioned.

TL;DR colors become desaturated when using an inpainting model with partial conditioning mask strength.

The fix is to replace lines 195-200 of modules/processing.py with a copy of the else condition below it:

conditioning_mask = source_image.new_ones(1, 1, *source_image.shape[-2:])

I haven't submitted another PR about this because, frankly, I don't understand what the original code does or why my fix works. But it does work.

[mclsugi]

conditioning_mask = source_image.new_ones(1, 1, *source_image.shape[-2:])

If we remove the previous code with that particular code, does it mean that we should have treat the Inpainting-models just like any other models, no?

Just a hunch, maybe there's a misunderstanding from. both the model's maker and the coders. Both sides try to adapt to each other, which in the end, screw things up.

Just imagine, perhaps the model maker thought, "So far, models are implemented like this, so we should modify our previous model with so we can have a proper model for inpainting." After the model was released, now it's the coders turn to assumed, "This is a different kind of model! We should learn it characteristic and modify our codes to implement this particular model!". And because of that, it doesn't implemented properly.

Worse part is, it won't threw no errors. If it eventually got fixed (assuming what I'm assumed is TRUE) that many users who already getting used to the wrong implementation will go berserk just like the case of HiresFix. 😄😄😄

Just my opinion though.