Inference Speed

[blessedcoolant]

I wanted to compare the inference speeds to https://github.com/harubaru/waifu-diffusion/

I noticed that while the first inference is around the same speed, the waifu-diffusion repo infers the iterations that follow at a much faster speed than this one does.

I'm already using --skip_grid and --small_batch.

Any idea why that is the case and what we could change to get the same output?

Additionally what are the chances of being able to add weighted prompts like this repo does: https://github.com/lstein/stable-diffusion

[basujindal]

Hi, you can use a larger batch size to reduce the inference time per image. Although I am working on reducing the inference time further, it's not very straightforward. Hoping to see positive results in a day or two. Also, thank you for the idea of weighted prompts; I will add this feature as soon as possible.

[basujindal]

Hi, I have added an optional argument --turbo. Using it will reduce the inference time to 25 sec per image for txt2img and 15sec per image for img2img (excluding the time to load the model once) at the expense of around 1GB VRAM.

Although this is not the optimization I was taking earlier. That is still in the works.