[Bug]: In sketch and inpaint sketch mode, FPS drops

[Igrom4ik]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

In sketch and inpaint sketch mode, the FPS drops and it is impossible to draw - the movements become abrupt! Only refreshing the page helps.

https://user-images.githubusercontent.com/31959570/218333718-08ab1a31-1c1a-4a53-a8ab-a2b3b3656d5d.mp4

Steps to reproduce the problem

1. Go to - export the image to inpaint, then to Sketch / Inpaint Sketch mode
2. Press left mouse button to draw the mask.

What should have happened?

Masking should work as smoothly as inpaint mode.

Commit where the problem happens

ea9bd9fc7409109adcd61b897abc2c8881161256

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

Launching Web UI with arguments: --api --xformers --autolaunch

List of extensions

stable-diffusion-webui-Prompt_Generator ultimate-upscale-for-automatic1111

Console logs

venv "venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: ea9bd9fc7409109adcd61b897abc2c8881161256
Installing requirements for Web UI

Launching Web UI with arguments: --api --xformers --autolaunch
Checkpoint  not found; loading fallback deliberate_v11.safetensors [d8691b4d16]
Loading weights [d8691b4d16] from E:\SD_Portable\models\Stable-diffusion\deliberate_v11.safetensors
Creating model from config: E:\SD_Portable\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(1): CharTurner
Textual inversion embeddings skipped(7): AnimeScreenCap, AnimeScreenCap-90, Chadeisson, Kipaki-150, UlukInkSketch2, VikingPunk-63, VikingPunk
Model loaded in 2.9s (create model: 0.2s, apply weights to model: 0.4s, apply half(): 0.5s, move model to device: 0.3s, load textual inversion embeddings: 1.4s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
100%|██████████████████████████████████████████████████████████████████████████████████| 22/22 [00:04<00:00,  5.18it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 22/22 [00:03<00:00,  6.28it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 22/22 [00:02<00:00,  7.58it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 22/22 [00:02<00:00,  7.66it/s]
Closing server running on port: 7860
Restarting UI...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
100%|██████████████████████████████████████████████████████████████████████████████████| 22/22 [00:02<00:00,  7.50it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 22/22 [00:02<00:00,  7.61it/s]
Closing server running on port: 7860
Restarting UI...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Closing server running on port: 7860
Restarting UI...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Closing server running on port: 7860
Restarting UI...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Loading weights [9e2c6ceff3] from E:\SD_Portable\models\Stable-diffusion\f222.ckpt
Applying xformers cross attention optimization.
Weights loaded in 32.3s (load weights from disk: 31.6s, apply weights to model: 0.2s, move model to device: 0.5s).
100%|██████████████████████████████████████████████████████████████████████████████████| 22/22 [00:02<00:00, 10.93it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 11.11it/s]

Additional information

No response

[ClashSAN]

Try with firefox for now

[Igrom4ik]

Try with firefox for now

I checked in other browsers. The result is as follows: Opera, Firefox - all work fine Cent browser, Yandex browser - there is a problem.

[Sarastro72]

I am having the same issue with firefox (haven't tried any other browser for now) on an M1 mac. Furthermore I am getting a rather scary warning in the console log.

Everything else works fine, even normal inpainting, but for sketch inpainting I get this every time. Image size is 512x768

Total progress: 100%|██████████████████████████████████████████████████████████████████████████| 279/279 [05:02<00:00,  1.08s/it]
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
Traceback (most recent call last):
  File "/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1013, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 911, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/components.py", line 1492, in preprocess
    im = processing_utils.decode_base64_to_image(x)
  File "/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/processing_utils.py", line 53, in decode_base64_to_image
    content = encoding.split(";")[1]
IndexError: list index out of range
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
/Volumes/Villevalla/projects/stable-diffusion-webui/venv/lib/python3.10/site-packages/PIL/Image.py:3167: DecompressionBombWarning: Image size (100663296 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(
100%|████████████████████████████████████████████████████████████████████████████████████████████| 31/31 [00:35<00:00,  1.13s/it]```

[doomhanner]

I have the same issue. Trying to send an image to either sketch tab causes the page to lag for several seconds, expands the image to several times normal size, then image breaks, page stops lagging, but no generations (txt2img or img2img) work until the page is reloaded.

[lyuro]

I have the same problem, but it seems that Chrome is the reason, after switching to Edge, everything is fine

[Dragoy]

As a temporary solution I propose to copy the picture to the buffer and paste it to the canvas with Ctrl+C. Then the fps doesn't drop

[rsboarder]

In my case, I had image in In Paint. Clearing image in In Paint made Sketch working

[joyblob]

I had the same issue. Check your browser screen resolution. It should be 100%. I have a 2k monitor and a resolution of 2560x1440, but I upscaled the screen to 125%. Then I returned the original windows screen resolution to 100% and the sketch worked fine. And don't forget to remove the img2img or inpaint settings in the launch .bat file if you have set them.

[Terpus]

I use 4k resolution for my desktop with 200% scaling in Windows display settings. I discovered that switching to 100% scaling resolves the issue. By the way, I mainly use Opera GX. If I use Automatic1111 in Chrome with the same scaling and resolution, I don't have this issue.