comfyanonymous
Support XLabs IPAdapter
Feature Idea
Add support for XLabs IPAdapter https://huggingface.co/XLabs-AI/flux-ip-adapter
Existing Solutions
Now, it requires custom nodes from XLabs. https://github.com/XLabs-AI/x-flux-comfyui
Other
We also need XLabs custom nodes to use the Unet loader and GUFF models. https://github.com/comfyanonymous/ComfyUI/issues/4482
What's the benefits of using Unet Loader ? With a checkpoint loader we can use a lot more features from other packages
The Unet loader has several benefits:
-
The Unet loader allows to use the ( larger ) fp16 versions of the Flux models and the T5 text encoder, which give better quality than fp8. You can also combine a fp16 model with a fp8 text encoder and vice versa. https://comfyanonymous.github.io/ComfyUI_examples/flux/#regular-full-version
-
The Unet allows the use of Flux finetunes, without the T5 encoder, because it can be loaded separately. This saves almost 5GB per Flux model you have, considering the checkpoints usually all have the same T5 encoder in fp8 format. ( e.g. see the size difference between the Unet (11.08 GB) and Checkpoint (15.91 GB ), both fp8 versions, of this finetune ). https://civitai.com/models/652009/fluxrealisticv1
-
The Flux GGUF versions ( Q4, Q8, etc... ) are also loaded via a custom Unet loader. They give similar quality as fp16 or fp8 formats, with smaller sizes. https://github.com/city96/ComfyUI-GGUF
Oh, right ! Perhaps the Comfy UI team wants to wait and see if @cubiq is going to release it's custom nodes for this, like on previous models. I'm OK with that !
https://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/687