Wagner Bruna

Just tested your fix with a few embeddings and models; it seems to be working now. Thanks!

FWIW, on that same rev different Qwen models (I can't run a 32b) seem to work fine on 128.6.0esr here (Debian 12, amd64, KDE Wayland). Could you check if the...

@evcharger , The Vulkan equivalent is `GGML_VK_VISIBLE_DEVICES`; for instance, `GGML_VK_VISIBLE_DEVICES=1` will make only the second card available to sd.cpp.

I did hit a weird behavior once, that at first I suspected was from a conditioner bug: a specific weight change on a prompt produced a distorted image, then another...

> https://civitai.com/models/372465?modelVersionId=914390 That's not a quantized model, so I'll assume you converted it yourself. > now on my other pc i did run a batch of 10 an all are...

> I did hit a weird behavior once, that at first I suspected was from a conditioner bug: a specific weight change on a prompt produced a distorted image, then...

Note this could have security implications: for instance, for web servers that call the library directly. It may be better to explicitly enable this support through a context flag, even...

Without considering the heuristic itself, and the badly named `GGML_VK_DEVICE`: why would you check an environment variable first (which supposedly could be changed by the user), then promptly override its...

> Is the existing `GGML_VK_VISIBLE_DEVICES` env var sufficient? Or does it not work with how sd uses vulkan? It works fine for me.

I'm kind of a smaller fish here, but I'd be happy to help :-) From the loading latency side, a low-hanging fruit could be support for reloading only parts of...