[Bug]: checking for availability of extensions results in crash of python process on Exception

[genglinxiao]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Upon click the "Load From" button in the Available tab of the "Extension" tab, and with the URL set to default, https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui-extensions/master/index.json, after a little while, I got an error message from the system: Python process terminated unexpectedly. (Rough translation from a Chinese message)

Steps to reproduce the problem

1. Start the webUI;
2. Go to the Extension tab;
3. Go to the Available sub tab;
4. Click on the "Load from" button;
5. Wait a while.

What should have happened?

The webui should be able to load all available extensions from github and list them in a table.

Sysinfo

sysinfo-2023-09-09-01-20.txt

What browsers do you use to access the UI ?

Mozilla Firefox

Console logs

-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer'
./webui.sh: line 255: 70084 Abort trap: 6           "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"

Additional information

Image generation seems unaffected.

[marcobazzani]

I have the same issue after installed SDXL and sdxl vae extension (removing them didn't fix) it worked one time but now loading plugins like written in this ticket or running a generation task crash the whole process in the same way

-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer' ./webui.sh: line 255: 44856 Abort trap: 6 "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"

Macos 13.5.2 on M2 Pro latest master version of this repo

[marcobazzani]

I have the same issue after installed SDXL and sdxl vae extension (removing them didn't fix) it worked one time but now loading plugins like written in this ticket or running a generation task crash the whole process in the same way

-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer' ./webui.sh: line 255: 44856 Abort trap: 6 "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"

Macos 13.5.2 on M2 Pro latest master version of this repo

I found a way to avoid the problem, run with --loglevel DEBUG and wait until there is no more logs printed looks like if you click generate or load from url too soon the webui crashes

[genglinxiao]

I have the same issue after installed SDXL and sdxl vae extension (removing them didn't fix) it worked one time but now loading plugins like written in this ticket or running a generation task crash the whole process in the same way -[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer' ./webui.sh: line 255: 44856 Abort trap: 6 "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@" Macos 13.5.2 on M2 Pro latest master version of this repo

I found a way to avoid the problem, run with --loglevel DEBUG and wait until there is no more logs printed looks like if you click generate or load from url too soon the webui crashes

After seeing this, I simply restart the webui and waited an extra minute after the model loaded message appeared, and yes, I can load the available extensions with no problem.

I guess this makes it a low prio bug.

[neal3000]

I am getting this issue too, even shutting down my computer and restarting does not make it go away

[iloveicedgreentea]

confirming waiting an extra 1-5 minutes avoids the crash

[thander]

confirming, Mac OS 14.0 (23A344), Apple M1 pro

[JohnFlyIII]

--loglevel DEBUG and waiting until all messages are done (a few minutes) does work.

(log is just there to help see that the system is "done", there's nothing special about running with debug logging)

[xvoland]

2023-10-14 20:32:58 STATUS [modules.shared_state] Ending job task(ujlcqv8wzh15szp) (1520.43 seconds)
-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer'
./webui.sh: line 255: 61593 Abort trap: 6           "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"
/opt/homebrew/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

--loglevel DEBUG and waiting until all messages are done (a few minutes) does work.

Following the @boxofnotgoodery advice, the problem disappeared

-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer'

However, A1111 will fall on macOS M2

pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu --upgrade

I updated PyTorch and it became very slow :(

My current solution:

./webui.sh --opt-split-attention-v1 --use-cpu all

or

--skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interrogate --opt-sdp-attention

I chose the second option and it's works perfect for Version: v1.6.0-245-gd4255506 Commit hash: d4255506ff41c382df68b926fd5d7e9e188d3dc2

[codejedi-ai]

I also have the same issue. When I try to run a model I would get [IOGPUMetalCommandBuffer validate]:215: failed assertion commit an already committed command buffer'`

[xvoland]

Maybe this will be useful

(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: error: 'mps.matmul' op contracting dimensions differ 2048 & 768
(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: note: see current operation: %5 = "mps.matmul"(%arg0, %4) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<2x77x2048xf16>, tensor<768x320xf16>) -> tensor<2x77x320xf16>
/AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphComputePackage.mm:180: failed assertion `expected a valid model URL'
./webui.sh: line 256: 57108 Abort trap: 6           "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"
/opt/homebrew/Cellar/[email protected]/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[afeiluo]

there is some task running background

[xvoland]

there is some task running background

@afeiluo This happened after start using ControlNet

(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: error: 'mps.matmul' op contracting dimensions differ 2048 & 768
(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: note: see current operation: %5 = "mps.matmul"(%arg0, %4) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<2x77x2048xf16>, tensor<768x320xf16>) -> tensor<2x77x320xf16>
/AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphComputePackage.mm:180: failed assertion `expected a valid model URL'
./webui.sh: line 256: 15443 Abort trap: 6           "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"
/opt/homebrew/Cellar/[email protected]/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[parjanyaacoder]

2023-10-14 20:32:58 STATUS [modules.shared_state] Ending job task(ujlcqv8wzh15szp) (1520.43 seconds)
-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer'
./webui.sh: line 255: 61593 Abort trap: 6           "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"
/opt/homebrew/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

--loglevel DEBUG and waiting until all messages are done (a few minutes) does work.

Following the @boxofnotgoodery advice, the problem disappeared

-[IOGPUMetalCommandBuffer validate]:215: failed assertion `commit an already committed command buffer'

However, A1111 will fall on macOS M2

pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu --upgrade

I updated PyTorch and it became very slow :(

My current solution:

./webui.sh --opt-split-attention-v1 --use-cpu all

or

--skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interrogate --opt-sdp-attention

I chose the second option and it's works perfect for Version: v1.6.0-245-gd4255506 Commit hash: d4255506ff41c382df68b926fd5d7e9e188d3dc2

The second option worked for me on a M2 MacBook with Stable Diffusion model - RPG-v4.safetensors

[gianpaj]

I tried setting this in webui-user.sh

export COMMANDLINE_ARGS="--skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interrogate --opt-sdp-attention"

but didn't work with Juggernaut XL (based on SDXL 1.0)

[Teglgaard]

I had a lot of issue running it on my macbook pro m1 16gb I ended up using a bit of help from the setup viking 1304 has done which optimizes it for mac. https://github.com/viking1304/a1111-setup I also use a controlnet https://github.com/Mikubill/sd-webui-controlnet And it works with that. Unrelated i was trying to do the QR codes from below :) Then you can follow guide in https://www.youtube.com/watch?v=nFfHEmNu0Zk&t=36s

[ryanhalliday]

In case anyone missed the comment by @marcobazzani while reading here:

WAIT BEFORE RUNNING YOUR FIRST GENERATION and the problem goes away

Things are still loading.

[photodarren]

I've got the same issue. Using SDXL. Yesterday (15 Apr 24) was working fine on my M2 Macbook Pro. Today it doesn't. I'll run it up, make a cup of tea, and see if it clears itself. If it says there are items queued then surely there is a way to clear the queue.. I would like to know why it is happening.