stable-diffusion-webui Different outputs since WebUI update

Hi, Since I've updated the WebUI, the outputs are slightly different.

All parameters are identical according to PNG Info:

Prompt
CFG scale
Sampler (Euler a)
Steps
Seed
Model hash

I've tried enabling and disabling "Use old emphasis implementation" and / or "Enable quantization in K samplers ..." in the settings but it didn't restore my original outputs.

WebUI commit hashes: old: 53651696dc1492f3b7cc009d64a55532bc787aa7 new: 594cbfd8fbe4078b43ceccf01509eeef3d6790c6

Example outputs: old, new

Oct 17 '22 10:10 CutterXYZ

In case it matters: COMMANDLINE_ARGS=--medvram --precision full --no-half

Oct 17 '22 21:10 CutterXYZ

I know you're not using xformers here, but in this issue where I recognized a similar issue, it was mentioned by some of the developers of these optimizations that because of rounding to support lower VRAM usage, generation may not be as deterministic.

Oct 17 '22 23:10 JustMaier

The first commit to change the output is 615b2fc9ce8cb0c61424aa03655f82209f425d21

Oct 21 '22 22:10 CutterXYZ

A friend mentioned that using xformers could make things non-deterministic, and that there were a lot of references to it on the repo issues here. Wanting to understand a bit more about it, and link a bunch of potentially related issues together, I tried to find as many issues as I could that seemed to be related to xformers and the potential for it to be causing non-deterministic / unstable / inconsistent results:

The following may potentially be related (ordered by issue number):

Issues:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1999
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2005
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2197
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2293
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2423
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2528
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2943
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3054
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3476
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3967
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/4011

Discussions:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/2418
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/2592
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/2705
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/2849
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/2958
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/3461

I also came across this thread in the xformers repo, which while I can't guarantee is related, am wondering if it might be:

https://github.com/facebookresearch/xformers/issues/219

And a question I raised on a PR in the diffusers repo:

https://github.com/huggingface/diffusers/pull/532#issuecomment-1297711826
- A friend just mentioned that apparently using xformers tends to lower the quality of generated images, and apparently even causes StableDiffusion to generate different images for the same seed/settings. I haven't looked too deeply into things to try and validate that, but I was wondering if a) that is something that is currently known/documented, and b) if that's likely to be fixed at any point, or if it's just 'the price we pay' for the benefits it adds. Would any of that be a concern with integrating things here? I did stumble upon this issue on the xformers repo, but not sure if it would be the same/similar root cause as my friend described: https://github.com/facebookresearch/xformers/issues/219

Nov 01 '22 04:11 0xdevalias