Marigold icon indicating copy to clipboard operation
Marigold copied to clipboard

Fix upsampling misalignments after resizing

Open nandometzger opened this issue 9 months ago • 1 comments

In the current implementation, we have the following problem: If the user input an image of odd shape (either not divible by 8 or aspect ratio which don't allow to resize to a multiple of 8) the code does something unexpected: It (or the stable diffusion backbone) cut's of the boarders to make it divisible by 8. After that we just resample it to original resolution. This can cause a misalignment (depht vs RGB) of up to 7 pixels in the extrem case at the bottom right corner. I propose to modify the resize_max_resolution function to slightly change the aspect ratio (which will be at max 768/7~=1% off from the true ratio), but this will in turn fix the misalignment when upsampling to match the resolution.

For now, I set the div8 boolean to false, so changes will not have any effects. Does any of the test-datasets have edges that are not divisible by 8? because that could mess with the evaluation protocol and the results are not exactly reproduceable anymore. If not, I propose to change div8 to True

nandometzger avatar May 13 '24 07:05 nandometzger

code quality check failed :'(

nandometzger avatar May 13 '24 07:05 nandometzger

Maybe it is better to take the padding route from diffusers

toshas avatar Jun 11 '24 05:06 toshas