AUTOMATIC1111
stop current generation if change from step to step is practically zero
Is your feature request related to a problem? Please describe. sometimes, the image will no more evolve from step to step. sometimes as soon as the 20th step (personal experience). But it still goes all the way to the last step unless click on stop button.
Describe the solution you'd like It would be time and ressources saving if generation was automagically stopped when there is no longer discernable changes from one gen to the other. a threshold could be set in the settings by the user.
Describe alternatives you've considered click the stop button. But sometimes i am not in front of the screen, expecially if batch generation, and some other time i am unsure about the rate of change from step to step so letting the process complete. But that is not satisfying.
Additional context Add any other context or screenshots about the feature request here.
the control could be executed every n (user-defined) steps until, eventually, completion of the generation process. That would also help user to determine if the image is still "evolving" significantly at the end, opening the door to apply user-defined schedule in order to ultimately get to the point of no more significant change per step.
There must be some ressources on the net about determination of two-image difference... might look that up later.
Did you try "DPM Adaptive" sampler? It is doing exactly this (but often chooses to perform 40-60 steps).
Did you try "DPM Adaptive" sampler? It is doing exactly this (but often chooses to perform 40-60 steps).
Ohhh ! Yes I tries it and noticed the number of steps variation. However I did not have time to investigate why exactly. So I didn't know that, TY.
Well then... Would I be too greedy, asking the possibility of other "adaptative" samplers ? or simply a script for making any sampler "adaptative", within the limits of its own schedule. I am not in position to write it at the moment. I am not sure at all to even be capable of writing it. But if nobody does it, I may give it a try next week.
while something as simple as a normalized absolute difference between the images of two consecutive steps can be used, this would have little to do with the actual structure of the image.
You can go to something more complex like the MSE or the PSNR, but the problem of lack of control over the morphological changes would continue, since it takes into account more the changes at the pixel value level. it is necessary to control the changes as perceived by the human eye.
the simplest metric that takes this into account is the SSIM. its value ranges from 0 to 1 (being 1 for two totally identical images)
In the OpenCV library, which is installed with webui (for Python), this function is included (also the PSNR), so there is no need to implement it from scratch.
there are other somewhat more complex metrics that take into account other morphological aspects
Suppose we have a function that compares two images to determine their changes and differences, and takes into account the structural changes and not only those of the pixel values. if the metric is linear (or can be converted to linear) it can be taken as a %, and the user might decide that if the last generated image compared with the previous step image (or every three, or four...) doesn't change by more than 5%, execution is stopped. If the metric is logarithmic (for example the PSNR is like this), values greater than 33dB correspond to differences almost imperceptible to the eye, 35-36 are practically invisible.
something like the "stop on learning plateau" when training neural networks
Sometimes, letting the user shoot himself in the foot is a good thing. Too much of this and that will lead to extreme amounts of complexity in the program. Better let this be, start using the "skip" / "interrupt" button.
Sometimes, letting the user shoot himself in the foot is a good thing. Too much of this and that will lead to extreme amounts of complexity in the program. Better let this be, start using the "skip" / "interrupt" button.
This would just be a slider, and I really want it. It would allow achieving a high quality in just the right number of steps automatically, without having to observe changes, or wait too long, or trial-error multiple times. This stop if there are hardly any changes is widely used in some iterative automatic image adjustment algorithms as a stop point, and as I mentioned before, as a stop during the long training of neural networks
Related: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1113#issuecomment-1258446573
Originally, I also thought that each step just takes the previous one and improves on it, where each new step brings a "constant" amount of improvements. My thinking was that if 10 steps brought 10u amount of details, then surely 40 steps would bring 40u of details, and I could stop the process anywhere in the middle to get, for example, 25u amount of details.
But in reality it behaves more like a sub-division of the distance between the noise and the final image. So imagine the noise is at distance 0 and the final image is at distance 1. When you set steps=10, then each step advances by 0.1. When you set steps=40, then each step advances by 0.025.
More steps doesn't get you "further", it gets you to the same destination but using smaller steps, which in theory helps the definition of the final image.