[Bug] t5 encoder problem

[zwukong]

i use it in flux comfyui workflow, the textencoder node seems needing three files,clip-L t5 and svdq-t5. The memory usage is huge,if we can only use clip-L.safetensor and svdq-t5.safetensors(3G file only),it will be great.

[xueqing0622]

use int4 Loading checkpoint shards from combine these two model very slow

why just use svdq-t5.safetensors

[lmxyy]

Hi, we are optimizing the RAM usage for loading our T5 encoder. You can also use the FP8 text encoder by choosing the corresponding file and set the T5 precision to BF16 for now.

[xueqing0622]

hope use t5 only svdq-t5.safetensors 2.99g

[xueqing0622]

use dual clip gguf clip can run! so it is another way to save vram.

[lmxyy]

yeah, it is compatible.

[xueqing0622]

hello, after use version 0.2, use gguf clip loader easy to out of memory, clean vram and out of memory on the second queue

use nunchaku text encoder loader is ok, and plz support support only use 2.9g t5 model when use int4_model

[lmxyy]

#320 should fix this issue.

[tazztone]

why are these files still needed? model-00001-of-00002.safetensors model-00002-of-00002.safetensors

[lmxyy]

fixed in v0.3.0

[tazztone]

btw remember to use the v2 version node of nunchaku text encoder. 🤝