Image is resized when using tool 'sketch'

[Darius-H]

Describe the bug

I have an image with shape (height=80,width=848), and I want to use the 'sketch' tool to generate mask for it. However, whether or not I use the gr.Image().style(height = 80, width = 848) or set shape parameters gr.Image(shape=(848,80)), the image is resized automatically which is not my desire.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

without .style

import gradio as gr
import os


def image_mod(input_image):
    init_image = input_image["image"]
    init_mask = input_image["mask"]
    return init_mask

with gr.Blocks() as demo:
    with gr.Column():
        i_image = gr.Image(tool='sketch',shape=(848,80))# .style(height = 80, width = 848)
        o_image = gr.Image(shape=(848,80))# .style(height = 80, width = 848)
        button = gr.Button()

    button.click(image_mod,inputs=i_image,outputs=o_image)


if __name__ == "__main__":
    demo.launch()

Screenshot

without .style:

with .style: The image even cover the button and other components.

Logs

$ python sketch.py 

Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

System Info

gradio version: 3.11.0
system: windows 11
python version: 3.9.7

Severity

annoying

[freddyaboulton]

Hi @Darius-H !

To make sure I understand, you're saying the mask should also be resized to be the shape passed to gr.Image?

[Darius-H]

Hi @Darius-H !

To make sure I understand, you're saying the mask should also be resized to be the shape passed to gr.Image?

I mean, you can see that in the first screenshot, the first Image component (which uses tool='sketch')is longer than the Run Button and exceeds the screen. But the second Image component which doesn't use tool='sketch' show the image correctly, and the image is resized to fit it. So when i set tool='sketch', the image passed to the Image component doesn't fit.

[abidlabs]

I tested this issue and it still seems to a problem even after PR #3277. The basic problem is that when an image is uploaded, the sketch tool is selected (or the color-sketch tool), and a fixed shape is provided, then the image is resized in a weird way. Let me give an example:

import gradio as gr

gr.Interface(
    lambda x:(x["image"], x["mask"]),
    inputs=gr.Image(tool="sketch", shape=(100, 100)),
    outputs=["image", "image"],
).launch()

produces this:

As you can see, the problem is only with the input image component. The data is being processed and displayed in the output just fine.

Note that this problem does not happen if a shape is not hardcoded:

import gradio as gr

gr.Interface(
    lambda x:(x["image"], x["mask"]),
    inputs=gr.Image(tool="sketch"),
    outputs=["image", "image"],
).launch()

or if the source is a canvas:

import gradio as gr

gr.Interface(
    lambda x:x,
    inputs=gr.Image(tool="sketch", source="canvas", shape=(100, 100)),
    outputs="image",
).launch()

It does happen if the the tool is color-sketch instead of sketch:

import gradio as gr

gr.Interface(
    lambda x:x,
    inputs=gr.Image(tool="color-sketch", shape=(100, 100)),
    outputs="image",
).launch()

and it seems that in this case, the data that is sent through is also affected: