stable-diffusion-tensorflow icon indicating copy to clipboard operation
stable-diffusion-tensorflow copied to clipboard

Load model without the image size

Open ulysses500 opened this issue 2 years ago • 4 comments

The current version of TF Stable Diffusion needs the image width and height to load the model.

def get_models(img_height, img_width, download_weights=True):

Indeed, the graph is built again if there is another image with a different size.

Is it possible to load the model without specifying the width and height? It would save a lot of time...

ulysses500 avatar Dec 02 '22 13:12 ulysses500

You do need to specify the width and height, but there's no reason to do it again for every generation if nothing changes. I re-wrote the code on my repo to check if anything has changed before generating again. No need to re-compile if it's already good to go!

You can also look into saving the currently compiled model and loading that instead.

soten355 avatar Dec 04 '22 19:12 soten355

No problem indeed if the images have the same size.

But what if want to use the model for many images having different sizes? I don't want to load the graph again and again.

ulysses500 avatar Dec 05 '22 07:12 ulysses500

Unfortunately the model needs to have an image size. If your computer can handle the memory use, you could create multiple classes that house pre-compiled models of different image dimensions, but I would imagine that would max out at 2 or 3 classes on the best machines.

soten355 avatar Dec 05 '22 21:12 soten355

Yes, I've thought about it: it is a very good workaround but it cannot cover all cases in a clean way and it uses additional memory :-/

ulysses500 avatar Dec 06 '22 10:12 ulysses500