Getting the following error when I try to run Flux model

[mdkberry]

I am on Windows 10 with latest updates and latest ACLY updates. it was working but I may have deleted a file while clearing down space a few weeks back, if I have I don't know which one and I can't tell from the messaging. Flux is installed and I didnt remove the models as I use them in Comfyui a lot. How can I figure out the issue there are no further clues in the client.log other than full traceback which I can share if it helps.

model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16 model_type FLUX gguf qtypes: Q6_K (169), F32 (73) Attempting to recreate sentencepiece tokenizer from GGUF file metadata... Created tokenizer with vocab size of 256384 Dequantizing token_embd.weight to prevent runtime OOM. CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16 clip missing: ['text_projection.weight'] !!! Exception during processing !!! Error(s) in loading state_dict for T5: size mismatch for shared.weight: copying a param with shape torch.Size([256384, 4096]) from checkpoint, the shape in current model is torch.Size([32128, 4096]).

EDIT: actually the gguf type mentioned above is kind of odd because my flux model is flux1-dev-fp8.safetensors when I look at the settings in the config for built-in. It begins the run when I press generate but fails part way trhough with the above. Same error if I try to setup flux inpainting model I have in the folder too.

EDIT: I have only "t5xxl_fp8_e4m3fn.safetensors" in the clip folder (under comfyui-portable models folder) but a "t5" folder in there too with "google_t5-v1_1-xxl_encoderonly-fp8_e4m3fn" in it

[Acly]

You can you check client.log for which T5 model is used.

If you are using the built-in style with flux1-dev-fp8.safetensors and it has T5 included and is in the checkpoints folder, no external T5 is needed and you shouldn't get this error.

[mdkberry]

in logs I can see no obvious issues other than the one I shared. I have

INFO Found Text Encoder t5 for All: umt5-xxl-encoder-Q6_K.gguf

but I just realised there is no entry mentioning flux1-dev-fp8.safetensors.

In my folders I have it stored in M:\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\unet

There is mention of flux in a number of places but not the main model. Does it need to be in a different folder for Krita to find it?

[Acly]

If you are using flux1-dev-fp8.safetensors that includes T5, it should be in checkpoints folder. See https://github.com/Acly/krita-ai-diffusion/discussions/1176

[mdkberry]

when I tried moving flux1-dev-fp8.safetensors out of the unet folder into the checkpoints folder it was no longer availble in Krita, so I moved it back.

its wierdly selective about what it shows. The image shows what is in my checkpoints folder and my unet folder and it cant find Flux but can find others. If I move flux to \unet folder it finds it. I rebooted Comfyui in case it was that but still not finding Flux-dev-fp8 when it is in checkpoint folder but its clear Krita is looking in both.

[Acly]

It inspects files and tries to filter models it doesn't support.

How big is your file? Where did you get it from? You need to know whether it's a checkpoint (includes T5) or diffusion model (aka. unet, doesn't include T5). That decides which folder it goes in.

Assuming it is a diffusion model because it doesn't show up in checkpoints, you still need the plugin to find the correct T5 model. You said it found umt5-xxl-encoder-Q6_K.gguf - I don't know what that is. Maybe try to find and remove it, so it picks up the t5xxl_fp8_e4m3fn.safetensors, that should work.

[mdkberry]

You found the cause. But I have to delete umt5-xxl-encoder-Q6_K.gguf if I want to run Krita.

Here is what is going on.

umt5-xxl-encoder-Q6_K.gguf I downloaded as part of a recent VACE workflow for using Quantstack's VACE Quantized models in comfyui. In the provided workflow that gguf model is used by the "GGUF clip loader" node, and the node can reach the model from either the clip or text_encoders folder.

But if that file is in either location, Krita wont run Flux. It just errors, picking that GGUF encoder instead of the other options with the messages I showed already.

[ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes gguf qtypes: Q6_K (169), F32 (73) Attempting to recreate sentencepiece tokenizer from GGUF file metadata... FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [DONE] [ComfyUI-Manager] All startup tasks have been completed. Created tokenizer with vocab size of 256384 Dequantizing token_embd.weight to prevent runtime OOM. CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16 clip missing: ['text_projection.weight'] !!! Exception during processing !!! Error(s) in loading state_dict for T5:

I am going to try to use the comfyui workflow with a different encoder model, and if I can, then will remove that GGUF model from the "clip" or "text_encoders" folders.

At least I know the cause and workaround at this point. That workflow with that GGUF Clip loader node, will probably trip more people up, it is only recently out. I dont have any other GGUF files in those folders, only safetensors.

Thanks for your help, and thank you for your time on this.

[Acly]

You can also put the T5 model you want the plugin to use in a krita subfolder (within the folder it's currently in). That way it should be prioritized over others.

I'm guessing the umt5-xxl is a bit different, there's a lot of models called T5 in various ways and it's difficult to know what's inside.

[mdkberry]

turns out I use the umt5-xxl-encoder-Q6_K.gguf in a couple of other workflows, but at least now i know I can swap it out when I need flux in Krita. I'll try the approach you suggest. closing this for now. thanks again.