[question] UNet Models

[MoonMoon82]

What can I do with them? Where do I get them? SDXL related?

[MaddyAurora]

A model checkpoint that usually ends in ckpt or safetensors that we all usually use, like those you can download from civitai or the oficial SD 1.5, all are comprised of 3 actual models.

The model contains a Unet model, a CLIP model and a VAE model.

If you separate them, you can load that individual Unet model similarly how you can load a separate VAE model.

The Unet is the neural network model that generates the image in the latent space.

[MoonMoon82]

@MaddyAurora Thank you for your detailed feedback ! I already noticed that some checkpoints come with more or less usefull VAEs, so I tended to use a separate VAE file independent of the checkpoint.

But I'm still not sure why there are separate loaders for Unet and Clip models. I never saw a "pure" Unet or Clip modelfile to download on ie civitai or huggingface. And for which purpose would it be useful to get Unet and Clip models separated from each other?

[comfyanonymous]

UNET Loader is for loading these files (unet files in diffusers format): https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main/unet

CLIP Loader is for loading these files: https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main/text_encoder

They are in advanced so you should only use them if you really need to.

[samiede]

Does it support sdxl unets as well?

[comfyanonymous]

Yes, it should support both SDXL and SDXL refiner unets.

[Feihei]

Is the unet loader only looking inside the models/unet path? Is it possible to specify another path for unet models in extra_model_paths.yaml ?

[joshgura]

unet loader is not launching a file dialog in order to select the unet. should it? same for the Load CLIP node. maybe the files are supposed to be in a default location, if so i apologize.

[ltdrdata]

unet loader is not launching a file dialog in order to select the unet. should it? same for the Load CLIP node. maybe the files are supposed to be in a default location, if so i apologize.

Fundamentally, ComfyUI operates as a server-based structure. Except for cases like image uploads from your computer, it uses files that are already stored on a remote server. Therefore, it only allows access to files in specific paths within the server