comfyanonymous
[Feature Request] SDXL on 16G RAM (model unload)
Currently SDXL base+refiner can`t be used at the same time if you have only 16G of RAM. Separate model works perfectly. Constantly switching between models is very annoying since you need to change workflow and pictures. However, loading model from SSD only takes about 10 seconds. This time is comparable to one iteration on my GTX 1650 ti (~8 s).
Possible solution: Can base model be "unloaded" from RAM after first stage is finished and loaded back at the next generation? (obviously, same is need for refiner model) Loading model from SSD is safe and much more faster than using swap on HDD. Something like extra node "unload" with trigger activation or an "unload model after generation" parameter at the sampler node would be fine.
I would like an Unload node as well. It would make many tasks, like troubleshooting, much more convenient.
I have finally found possible solution.
torch.Tensor object has set_() method. This allows to overwrite tensor value with same dtype but different shape and data. This methos is inplace (it changes instance, not just copy) and also frees unused memory. This allows to replace all model weights with just [0] using only variables that is passed to node.
I will try to code "Unload" node and model reloading after change.
This looks like crutch so RAM leaks possibe.
After some testing yes, this works. Although, my implementation is too bad. Another (perhaps an easier one) solution is to create node for model replacement. Sadly, I can`t write proper code with my poor knowledge of ComfyUI.
At least you tried. I'm a new user to the UI so I'm too unfamiliar with the project as a whole.
After I finish using some functions, the video memory is always occupied, which prevents me from completing subsequent operations. Is there a way to put the calculated model somewhere else? I'm referring to plug-ins like segment_anyting that are needed at the beginning but not needed later.
I have been running into problems today using 3 different SD 1.5 models alongside ESRGAN-4x upscale. Not possible to use anything else beside the workflow
A node to unload a model (sd, upscale, clip, facerestore,...) would be really great
Fooocus program do it is why you can run base + refiner on little config. I thinks the good way is a node for replace the current loaded model with new path and model output node
ComfyUI-Manager has a "Unload models" button (in the "Manage" pop up). However it doesn't seem to really unload everything.
Yeah, it was only recently added, but I've also noticed it doesn't get everything.
For now, better than nothing.
