[Metal]: Regression after 1ba30ce: Unsupported Op with nitro models

[newfla]

Up to master-306-2abe945 Chroma models won't work when building for Metal backend with the infamous GGML unsupported op abort signal. Starting from 1ba30ce Chroma model started working but nitro/diff* models crash with the GGML unsupported op signal.

The models files that I'm using are described here: Chroma Nitro

[Green-Sky]

There has been another llama.cpp sync to ggml https://github.com/ggml-org/ggml/commit/84fd368c7b7b11c07089d78457c4992bc6f96b6e . Can you try with an updated ggml submodule?

[newfla]

Unfortunately the regression is still present 😕

[newfla]

Hi, Metal backend seems heavily broken: I have found that sxdl-turbo is now broken.

CMD: ./sd -m /Users/fbiz/.cache/huggingface/hub/models--stabilityai--sdxl-turbo/blobs/e869ac7d6942cb327d68d5ed83a40447aadf20e0c3358d98b2cc9e270db0da26 --vae /Users/fbiz/.cache/huggingface/hub/models--madebyollin--sdxl-vae-fp16-fix/blobs/235745af8d86bf4a4c1b5b4f529868b37019a10f7c0b2e79ad0abca3a22bc6e1 --guidance 0.0 --steps 4 --cfg-scale 1.0 -p "banana"

Relevant stack trace:

[INFO ] model.cpp:2293 - loading tensors completed, taking 11.43s (process: 0.01s, read: 11.22s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.08s) [INFO ] stable-diffusion.cpp:668 - total params memory size = 6558.89MB (VRAM 6558.89MB, RAM 0.00MB): text_encoders 1564.36MB(VRAM), diffusion_model 4900.07MB(VRAM), vae 94.47MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM) [INFO ] stable-diffusion.cpp:721 - running in eps-prediction mode [INFO ] stable-diffusion.cpp:2457 - TXT2IMG [INFO ] stable-diffusion.cpp:881 - attempting to apply 0 LoRAs [INFO ] stable-diffusion.cpp:901 - apply_loras completed, taking 0.00s [ERROR] ggml_extend.hpp:71 - ggml_metal_op_encode_impl: error: unsupported op 'DIAG_MASK_INF' /Users/fbiz/diffusion-rs/sys/stable-diffusion.cpp/ggml/src/ggml-metal/ggml-metal-ops.cpp:140: unsupported op

[newfla]

Is there anything we can do? The CLDawes pull request on the llama.cpp repository is still open and not reviewwd. 😕

[wbruna]

I'd suggest testing the patch, and posting about the results on the PR.