lllyasviel
It's too time-consuming to reload flux model every time generate an image
It's just swapping it from VRAM to SYSRAM, which is negligible in speed difference, while integral to keeping from having bugs and also for general memory management to keep garbage from collecting. Pretty much any system memory should be able to swap that in under a second.
It's just swapping it from VRAM to SYSRAM, which is negligible in speed difference, while integral to keeping from having bugs and also for general memory management to keep garbage from collecting. Pretty much any system memory should be able to swap that in under a second.
yet it's saying, 5 to 6 seconds yea?
It takes much longer if you're just on the edge of using all your system ram. For me it adds almost 1 minute. If parts of the model were to be kept in VRAM, that would be a huge relief for my DRAM.
It takes much longer if you're just on the edge of using all your system ram. For me it adds almost 1 minute. If parts of the model were to be kept in VRAM, that would be a huge relief for my DRAM.
I had found a few optimization settings I didn't know about before, and if I use the taesd vae the model stays in memory most times and reloads a 10th of the time it seems. Quality takes a bit of a hit though but fiddling with parameters help mitigate that.
Still doesn't seem like this should be happening though.
I set it to Swap Method:Async,Swap Location:Shared,now I don't have to reload the model every time!
I get a hang which adds about 1 min + to the average render time when the LORA weight is changed using NF4
version: [f2.0.1v1.10.1-previous-329-g128a7932] commit/128a793265c1e43da66e3b3382bda9327c03fd52) • python: 3.10.6 • torch: 2.3.1+cu121 • xformers: N/A • gradio: 4.40.0 • checkpoint: [b7e5569f8c]
I set it to Swap Method:Async,Swap Location:Shared,now I don't have to reload the model every time!
nice, i have to blindly tweak a lot of things to reduce this, I'm no expert in this field so its hard to know what to tweak on my specific system
my latest issue has been with the kmodel loading and unloading, something I changed helped this a lot but i can't even recall what it was i tried so many things