Add queuing (different from batching)

[ghostsquad]

Is your feature request related to a problem? Please describe. Sometimes I have separate prompts I'm writing on, I'd like to be able to open multiple tabs and queue up prompt generations

Describe the solution you'd like A queue, similar to that of Midjourney

Describe alternatives you've considered Just waiting...

Additional context N/a

[xram64]

This is already (kind of) how it works. If I open up two tabs and hit generate in each one, it processes one and then the other, although it bugs out the UI a bit. For instance, if Tab 1 is generating something, clicking generate in Tab 2 seems to make the progress images for Tab 1 show up in Tab 2's gallery. The progressbar also seems to completely break in both. But, it seems like these are just cosmetic issues because once both tabs are finished generating, they go back to showing their correct gallery results (at least as far as I've seen). So the feedback might be missing, but it still technically works.

This still only lets me start two generations at a time usually, but adding something like demo.queue(concurrency_count=5) to webui.py before the demo.launch() call (just above line 105) allows for more, although it doesn't fix the UI issues.

To be clear, this is just a hack, and the Web UI doesn't seem to be designed for queuing. I'm not sure what it would take to implement properly, but I'm guessing it'd have something to do with the state = State() instance created in modules/shared.py, and multiple instances would have to be made to keep track of different sessions.

[ghostsquad]

Does the generation happen in parallel though? Cause I don't have enough gpu memory to do too much in parallel. Ideally, similar to batching, I'd like to queue up 10's or 100's of operations, such as when I can let it process overnight.

[ghostsquad]

This is already (kind of) how it works. If I open up two tabs and hit generate in each one, it processes one and then the other, although it bugs out the UI a bit. For instance, if Tab 1 is generating something, clicking generate in Tab 2 seems to make the progress images for Tab 1 show up in Tab 2's gallery. The progressbar also seems to completely break in both. But, it seems like these are just cosmetic issues because once both tabs are finished generating, they go back to showing their correct gallery results (at least as far as I've seen). So the feedback might be missing, but it still technically works.

This still only lets me start two generations at a time usually, but adding something like demo.queue(concurrency_count=5) to webui.py before the demo.launch() call (just above line 105) allows for more, although it doesn't fix the UI issues.

To be clear, this is just a hack, and the Web UI doesn't seem to be designed for queuing. I'm not sure what it would take to implement properly, but I'm guessing it'd have something to do with the state = State() instance created in modules/shared.py, and multiple instances would have to be made to keep track of different sessions.

Ok, so I guess the TLDR on this is improve state management so you can see what's going on and have a more fluid experience?

[millerpb]

Just noticed this as well. I decided to try out the share feature and let my family generate some images from my PC. When multiple people tried generating an image at the same time, there was a memory error. I was surprised it didn't just queue up the requests.

Would love to see this supported. 🤞

[Dusty12345]

I really love the queueing feature on the cmdr2 ui, would love to see it implemented in AUTOMATIC 1111 web ui!

[Kryptortio]

So I made a queue system using a "user script" (also let's you save sets of settings). It works kind of like and addon, you can get it here:

https://github.com/Kryptortio/SDAtom-WebUi-us

It could be added into the main repo but they might prefer a solution more tightly integrated with the normal interface (I'm not a Python dev so I built it like this instead).

I just completed it and have not tested it a lot so there is bound to be some bugs but it seems to be working pretty well now.

[ComproLoper]

So I made a queue system using a "user script" (also let's you save sets of settings). It works kind of like and addon, you can get it here:

https://github.com/Kryptortio/SDAtom-WebUi-us

It could be added into the main repo but they might prefer a solution more tightly integrated with the normal interface (I'm not a Python dev so I built it like this instead).

I just completed it and have not tested it a lot so there is bound to be some bugs but it seems to be working pretty well now.

I just used your addon to generate 1090 images over 2 hours, and it worked flawlessly. That is, 109 batches of 10 images added to the queue as I came up with prompts, there was no problem adding more when the queue was already running.

[ghostsquad]

I'll install it tonight and provide feedback! Thank you for your hard work and contribution!