stable-diffusion
stable-diffusion copied to clipboard
CompVis
Blue or black artifacts in many images
When generating an image there is a high chance of the appearance of small blue or black blobs, this is mainly noticeable in simpler images such as smooth textures, however it can appear in much more complex images as well, I've noticed this using txt2img, img2img and in-painting.
I have replicated this problem 100% reliably on a number of different local installations and online generators, from other people reporting this issue it appears to effect all operating systems and GPU's, different samplers, steps, CFG, etc have practically no effect on it.
"A sheet of white paper texture"
Credit: u/mikenew02 - reddit
Credit: abesmon - github
Credit: hd-x - github
This creates quite a bit of additional work to correct so I'd be very happy if this could be fixed if possible.
This problem appears to be related to prompt complexity and image depth, flat textures are a particular problem with the artefact appearing roughly between 40% to 90% of the time depending on the amount of detail, this makes stable diffusion very unreliable with texture generation, simple prompts also seem to be a major cause regardless of the image complexity.
I just encountered this today, usually I run a quick test to make sure SD is working after a git pull and sure enough blue spots with a simple prompt like "1girl" but it seems limited to simple 1-2 word prompts such as what I provided. If you were to use a prompt like "1girl, blonde hair" then the problem seems to not happen or very rarely. So basic take away is avoid very basic prompts.
I've had the same issues.. you all need to update to the latest VAE. Problem gone :)
I've had the same issues.. you all need to update to the latest VAE. Problem gone :)
I tested again with v1-5-pruned.ckpt [a9263745] with vae-ft-mse-840000-ema-pruned.ckpt [ c6a580b13a5bc05a5e16e4dbb80608ff2ec251a162311590c1f34c013d7f3dab] which is the latest as far as I'm aware. Things like landscapes appear significantly better with perhaps less than 1 in 10 having a noticeable artifact, however textures are still a big problem unfortunately.
Have you followed these instructions? I recommend to configure usage for all models - how to is in the description. https://www.reddit.com/r/StableDiffusion/comments/yaknek/you_can_use_the_new_vae_on_old_models_as_well_for/
Have you followed these instructions? I recommend to configure usage for all models - how to is in the description. https://www.reddit.com/r/StableDiffusion/comments/yaknek/you_can_use_the_new_vae_on_old_models_as_well_for/
Yes the latest VAE is definitely being loaded.
Prompt: a sheet of white paper texture
Seed: 784215236
CFG: 7
It's definitely a lot better, this took several tries to get something so bad, in most cases now it's not noticeable unless you're really looking for it, so for most practical purposes it may no longer be an issue.
Great. Thanks for the follow-up and the extensive testing. Appreciated by a nerdish mind like me :)
im having this issue with a hypernetwork trained on huggingface 1.4 without any vae, and i've successfully mitigated it by adding "blue spot, blue dot" to my negative prompts. its hacky, but it works. my model was trained on an artstyle where this will basically never disclude proper style creation. if you wanted pictures of earth from space, maybe not great.
This is definitely still an issue. I can't produce any textures without having these artifacts in them. Changing models, VAEs, samplers, steps or any other settings seems to have no effect.
