sd-webui-controlnet
sd-webui-controlnet copied to clipboard
Mikubill
MPS keeps crashing
I got the ControlNet extension loading fine, but it keeps on crashing when I use scribble:
0%| | 0/20 [00:00<?, ?it/s](mpsFileLoc): /AppleInternal/Library/BuildRoots/9e200cfa-7d96-11ed-886f-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:228:0: error: 'mps.add' op requires the same element type for all operands and results
(mpsFileLoc): /AppleInternal/Library/BuildRoots/9e200cfa-7d96-11ed-886f-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:228:0: note: see current operation: %5 = "mps.add"(%4, %arg2) : (tensor<2x1280xf32>, tensor<*xf16>) -> tensor<*xf32>
zsh: segmentation fault ./webui.sh
/opt/homebrew/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Getting the same error when I use any model
Loading preprocessor: depth, model: control_sd15_depth [fef5e48e] Loaded state_dict from [/Users/philbuck/sd/extensions/sd-webui-controlnet/models/control_sd15_depth.pth] ControlNet model control_sd15_depth [fef5e48e] loaded. 0%| | 0/16 [00:00<?, ?it/s]loc("mps_add"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/0aa643d0-625a-11ed-b319-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<2x1280xf32>' and 'tensor<*xf16>' are not broadcast compatible LLVM ERROR: Failed to infer result type(s). zsh: abort ./webui.sh (base) philbuck@PhilsMacStudio sd % /opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
You can get it working if you use --no-half, but it's obviously a lot slower and uses a lot more memory. Hoping for a solution to this, the tool looks really cool.
my Web UI launches with the argument: --no-half-vae, does that have a different effect than --no-half?
Yeah, I still get the tensor size mismatch with --no-half-vae, only --no-half fixes it. I'm not the most knowledgable on pytorch, but I believe that the issue is when you try to do an operation with a tensor of type float16 and a tensor of size float32. --no-half forces everything to use float32 and fixes the issue, but at a significant cost to performance.
Hmm, --no-half unfortunately doesn't fix it for me. I get a whole different set of errors.
Sure, systems specs are Mac Studio M1 Ultra Running Ventura 13.1 Error is super long:
ControlNet model control_sd15_openpose [fef5e48e] loaded. Error running process: /Users/philbuck/sd/extensions/sd-webui-controlnet/scripts/controlnet.py Traceback (most recent call last): File "/Users/philbuck/sd/modules/scripts.py", line 386, in process script.process(p, *script_args) File "/Users/philbuck/sd/extensions/sd-webui-controlnet/scripts/controlnet.py", line 270, in process input_image = HWC3(image['image']) TypeError: 'NoneType' object is not subscriptable
0%| | 0/20 [00:00<?, ?it/s]
Error completing request
Arguments: ('task(5q0lfqy3o0p0qe2)', 'Dog', '', [], 20, 0, False, False, 1, 1, 7, -1.0, -1.0, 0, 0, 0, False, 512, 768, False, 0.7, 2, 'Latent', 0, 0, 0, [], 0, False, 'keyword prompt', 'keyword1, keyword2', 'None', 'textual inversion first', True, 'openpose', 'control_sd15_openpose [fef5e48e]', 1, None, False, 'Scale to Fit (Inner Fit)', False, False, False, 3, 0, False, False, False, False, 'positive', 'comma', 0, False, False, '', 1, '', 0, '', 0, '', True, False, False, False, 0, None, True, None, None, False, 10.0, True, 30.0, True, 0.0, 'Lanczos', 1) {}
Traceback (most recent call last):
File "/Users/philbuck/sd/modules/call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "/Users/philbuck/sd/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/Users/philbuck/sd/modules/txt2img.py", line 56, in txt2img
processed = process_images(p)
File "/Users/philbuck/sd/modules/processing.py", line 486, in process_images
res = process_images_inner(p)
File "/Users/philbuck/sd/modules/processing.py", line 628, in process_images_inner
samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
File "/Users/philbuck/sd/modules/processing.py", line 828, in sample
samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
File "/Users/philbuck/sd/modules/sd_samplers_kdiffusion.py", line 323, in sample
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
File "/Users/philbuck/sd/modules/sd_samplers_kdiffusion.py", line 221, in launch_sampling
return func()
File "/Users/philbuck/sd/modules/sd_samplers_kdiffusion.py", line 323, in
- (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups) didn't match because some of the arguments have invalid types: (NoneType, Parameter, Parameter, tuple, tuple, tuple, int)
- (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups) didn't match because some of the arguments have invalid types: (NoneType, Parameter, Parameter, tuple, tuple, tuple, int)
Hmm. I wonder if we might be running different versions of pytorch. Are you using the mac-specific build discussed here? I recommend using it even though it doesn't resolve this issue, since it provides a ~25% speed boost on MPS.
I was not running that mac-specific build. Thanks for tipping me to that resource. I'm installing now and will try ControlNet again to see if get the same error.
Unfortunately, still running into the same errors even after updating to the mac-specific build. 🤷♂️
And you've pulled the latest Automatic update & sd-webui-controlnet update, correct? Just trying to figure out what could be different in our setup.
Wow, ok I did need to update ControlNet and it's working with --no-half. Thank you!
No worries! Glad it helped. Hopefully we can get a solution that allows us to use float16
With the command line arg --opt-sub-quad-attention and --no-half it runs about twice as fast for me (7-8s/it vs 20s/it). Still terrible but a bit better.
Yes, float16 doesn’t work correctly with MPS on this extension yet. I will try to fix that but I can’t make any guarantees at this point. First though I want to fix the normal and depth map preprocessors returning bad/inconsistent results on MPS.
I have to assume that v21 is now required to use ControlNet. Whenever I use v15 models, it crashes.
I'm also running a Mac Studio M1 Ultra Running Ventura 13.1 and have got the latest versions of everything - Automatic1111, Controlnet, Python, et al. Have applied the suggested fixes that have worked for Philbuck84 as per above but Python still keeps crashing on trying to render using Controlnet every time, with the exact same error warning resulting in the message ending in "There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '"
I'm also running a Mac Studio M1 Ultra Running Ventura 13.1 and have got the latest versions of everything - Automatic1111, Controlnet, Python, et al. Have applied the suggested fixes that have worked for Philbuck84 as per above but Python still keeps crashing on trying to render using Controlnet every time, with the exact same error warning resulting in the message ending in "There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '"
I can confirm now that it is working for me after adding --no-half to the web-user.sh file, COMMANDLINE_ARGS line.
For unknown reason, from today, I kept getting the crashing again. It was working find for many weeks and suddenlty....
Loading preprocessor: none 0%| | 0/20 00:00<?, ?it/s: /AppleInternal/Library/BuildRoots/9e200cfa-7d96-11ed-886f-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: error: 'mps.matmul' op contracting dimensions differ 1024 & 768 (mpsFileLoc): /AppleInternal/Library/BuildRoots/9e200cfa-7d96-11ed-886f-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:39:0: note: see current operation: %3 = "mps.matmul"(%arg0, %2) {transpose_lhs = false, transpose_rhs = false} : (tensor<1x77x1024xf32>, tensor<768x320xf32>) -> tensor<1x77x320xf32> zsh: segmentation fault ./webui.sh jimmygunawan@192-168-1-100 stable-diffusion-webui % /opt/homebrew/Cellar/[email protected]/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
If I use the diff_openpose model, it's working. But the more recent one keeps crashing my webUI automatic1111.