[Bug]: Img2Img Inpainting/Sketching - Mask/sketch cannot be cleared

[CodeHatchling]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

When using the inpainting feature (specifically, inpaint sketch) you cannot clear the mask/sketch without opening a new tab. In order to control the generation it is sometimes required that you make small changes one-at-a-time. However, due to a bug with the painting feature, this is very cumbersome to do as it requires closing and reopening the tab, reconfiguring your prompt and settings, etc.

Additionally, the feature is generally very buggy/unpolished and this is only one of the issues I have good reproduction steps for. Other issues include:

• When passing an image over from text2img, it will show a blank white canvas after some initialization steps.
• It will log UnidentifiedImageError messages and fail to generate anything.
• The canvas can meander around the entire screen and get lost under the page.
• Controls disappear off-screen when zooming in.

I will attach the console logs that occurred while reproducing these problems.

Steps to reproduce the problem

1. In Text2Image, generate an image. As an example, write "A person walking a dog".
2. Click the "Send image and generation parameters to img2img tab" button.
3. In the inpainting menu, click "Copy image to: [inpaint sketch]"
4. Inpaint the dog with a red brush, change prompt to "A person walking a red dog".
5. Click inpaint (whether the generation actually looks decent doesn't matter for testing).
6. After it finishes generating, click the "Send image and generation parameters to img2img tab" button to make further edits.
7. Press "Copy image to: [inpaint sketch]" again.
8. On the canvas, press the "Clear" button.
9. Paint over the person's face.
10. Change the prompt to "A woman walking a red dog".
11. Click inpaint (whether the generation actually looks decent doesn't matter for testing).
12. Notice that it inpaints both the woman and the dog despite clearing the canvas. It doesn't clear the original sketch/mask.

What should have happened?

On step 12, only the woman's face should be regenerated, as it is the only thing painted over with a mask in the second inpainting edit.

Sysinfo

sysinfo-2023-11-18-21-47.txt

What browsers do you use to access the UI ?

Google Chrome

Console logs

[progress bars]

100%|██████████████████████████████████████████████████████████████████████████████████| 60/60 [00:25<00:00,  2.38it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [00:25<00:00,  2.35it/s]
Traceback (most recent call last):█████████████████████████████████████████████████████| 60/60 [00:25<00:00,  1.99it/s]
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1429, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1239, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\components\image.py", line 274, in preprocess
    im = processing_utils.decode_base64_to_image(x)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\processing_utils.py", line 59, in decode_base64_to_image
    img = Image.open(BytesIO(base64.b64decode(image_encoded)))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\PIL\Image.py", line 3298, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x000002B6D1FB9DF0>
Traceback (most recent call last):
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1429, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1239, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\components\image.py", line 274, in preprocess
    im = processing_utils.decode_base64_to_image(x)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\processing_utils.py", line 59, in decode_base64_to_image
    img = Image.open(BytesIO(base64.b64decode(image_encoded)))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\PIL\Image.py", line 3298, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x000002B811ACD580>
Traceback (most recent call last):
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1429, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1239, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\components\image.py", line 274, in preprocess
    im = processing_utils.decode_base64_to_image(x)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\processing_utils.py", line 59, in decode_base64_to_image
    img = Image.open(BytesIO(base64.b64decode(image_encoded)))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\PIL\Image.py", line 3298, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x000002B7C397D3F0>
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.35it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.64it/s]

[progress bars]

Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [00:25<00:00,  2.35it/s]
Traceback (most recent call last):█████████████████████████████████████████████████████| 60/60 [00:25<00:00,  2.17it/s]
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1429, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1239, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\components\image.py", line 270, in preprocess
    assert isinstance(x, dict)
AssertionError
Traceback (most recent call last):
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1429, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1239, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\Steve\Downloads\sd.webui\system\python\lib\site-packages\gradio\components\image.py", line 270, in preprocess
    assert isinstance(x, dict)
AssertionError
100%|██████████████████████████████████████████████████████████████████████████████████| 60/60 [00:23<00:00,  2.58it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [00:23<00:00,  2.54it/s]

[progress bars]

Additional information

No response

[derpina-ai]

found this bug too, my workaround is to delete the inpainting image when i go to img2img. I did not need to do the opposite until now, but I guess that can work too.

[songproducer]

This is too small and doesn't look like it's the right resolution either. It only happens in Safari. Chrome works okay.