sd-webui-controlnet
sd-webui-controlnet copied to clipboard
Mikubill
Inpainting masked results in different resolution
Using inpaint with inpaint masked and only masked options results in the output being distorted.
I'm not sure how the resize modes are supposed to work, but sometimes even with the same settings the results are different. Like one in 30 times the output will actually match the same size somehow.
Changing width and height sometimes affects it and sometimes doesn't. Every time I thought I found the settings that work the next time I load a new image the same settings don't work anymore.
Maybe it's a bug or maybe I'm doing something wrong with how I have it set up?
For example, in this case the face is larger than the original.
I have added a clearer example of the issue. Same settings. You can see that the head is greatly enlarged.
And then same result but a bit better after altering the input image size. So it is somehow related to image sizes, but no idea how. The distortion seems to happen espeiakll downwards/sideways.
Also results for changing to outter fit. (never found a case where this one is decent)
And resize.
Same thing happens if you crop the image in img2img. In this case, rather than processing the cropped area, it's processing the whole image, resizing to the new resolution and applying the map over the cropped area. Thus the map and the image are completely out of sync.
So maybe rather than just resizing, there should be some cropping involved in the process?
I can now say with almost full certainty that it is related to Mask blur. A value of 0 does not distort the image, while increasing it distorts the image for each extra pixel. @Mikubill
Yep, was just looking for this issue's writeup. Looks like it is in fact the mask blur isn't offsetting properly. I'm guessing it's supposed to offset properly on the x and y axis equal to the blur, but it's definitely not factoring in the outside mask properly.
It seems to be processing the image at the larger size but then it puts it back in at the same location at the original unmasked size.
Here's an example of a bookshelf because it has the best contrast of straight lines. Used Hed and Canny for detail. You can see the zoomed shelf is higher and lower meaning the x and y aren't properly scaled but are centered properly.
It also doesn't properly center when the mask + padding is on an image border. Instead of throwing away the overflow, it pushes the image over to place it which messes with the justification.
Here it pushed the entire mask down to fit rather than centering like it does in the middle of the image.
--Edit: Ok, fixed the images. My mistake. Some of the images were fuzzy.
Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests.
Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests.
Yes, I have the same issue
Same issue here! I am thinking maybe try turning off mask blur and using Poisson Blending to process the boundary?
On Sat, Apr 15, 2023 at 8:36 AM Mark @.***> wrote:
Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests.
+1 For me it generates whole new image with controlnet, like im using text2image
— Reply to this email directly, view it on GitHub https://github.com/Mikubill/sd-webui-controlnet/issues/288#issuecomment-1509832531, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.***>
Ive not had that issue. Controlnet works, i just can’t do a mask blur. I always have to use mask padding instead.
On Sat, Apr 15, 2023 at 8:36 AM Mark @.> wrote: Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests. +1 For me it generates whole new image with controlnet, like im using text2image — Reply to this email directly, view it on GitHub <#288 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.> Ive not had that issue. Controlnet works, i just can’t do a mask blur. I always have to use mask padding instead.
sry, I didn't mention a thing:
- It doesn't work with open pose models (background completely changes)
Huh. I’ll give it a go. What settings for inpaint and control net when it’s being run? Also, what is the preprocessor generating for your image?
On Sat, Apr 15, 2023 at 12:21 PM Mark @.***> wrote:
On Sat, Apr 15, 2023 at 8:36 AM Mark @.> wrote: Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests. +1 For me it generates whole new image with controlnet, like im using text2image — Reply to this email directly, view it on GitHub <#288 (comment) https://github.com/Mikubill/sd-webui-controlnet/issues/288#issuecomment-1509832531>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.> Ive not had that issue. Controlnet works, i just can’t do a mask blur. I always have to use mask padding instead.
sry, I didn't mention a thing:
- It doesn't work with open pose models (background completely changes)
— Reply to this email directly, view it on GitHub https://github.com/Mikubill/sd-webui-controlnet/issues/288#issuecomment-1509899481, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U3R7V3II7KGNHZDPWLXBLKJFANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.***>
Huh. I’ll give it a go. What settings for inpaint and control net when it’s being run? Also, what is the preprocessor generating for your image? … On Sat, Apr 15, 2023 at 12:21 PM Mark @.> wrote: On Sat, Apr 15, 2023 at 8:36 AM Mark @.> wrote: Anyone else having issues doing inpainting with controlnet enabled these days, with all the most recent updates? The masks are getting ignored when I enable a controlnet in my latest tests. +1 For me it generates whole new image with controlnet, like im using text2image — Reply to this email directly, view it on GitHub <#288 (comment) <#288 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU https://github.com/notifications/unsubscribe-auth/A4GG6U7SVBQ26VHK2JRI2W3XBKP47ANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.> Ive not had that issue. Controlnet works, i just can’t do a mask blur. I always have to use mask padding instead. sry, I didn't mention a thing: - It doesn't work with open pose models (background completely changes) — Reply to this email directly, view it on GitHub <#288 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4GG6U3R7V3II7KGNHZDPWLXBLKJFANCNFSM6AAAAAAVELHXDU . You are receiving this because you commented.Message ID: @.**>
Default settings of img2img inpaint, but with a painted area to replace
Controlnet preprocessor - openpose, model - openpose (tried t2iadapter_openpose_sd14v1 and control_v11p_sd15_openpose), same image as inpaint image set
Also I tried both "Inpaint area"s: fill whole and only masked