invoke-ai
[bug]: Completely broken final steps ruining all faces and prompts.
Is there an existing issue for this?
- [X] I have searched the existing issues
OS
Linux
GPU
cuda
VRAM
8gb
What version did you experience this issue on?
3.0.0+a0
What happened?
Prompts used to be flawless 1-2 days ago, then I got the conjunction issue from my previous bug, but seems like you fixed it, I can prompt again, but now for some reason I am seeing tons of weird steps that are in negative space colors in between regular steps, and then everything still looks "fairly normal" until the final 2-5 steps no matter what and it turns what looks like normal faces into complete messes. Happens with any complex prompt involving people mostly, about 95% of prompts come out like this now across multiple models.
Screenshots
Additional context
So I crashed my system by trying to do a 6000px prompt the other day then couldn't prompt anymore at all due to previous bug in my history. This was 2ish days ago? Seems like maybe what happened to me was just a coincidence, because they apparently patched it, and my prompting does work with the above issue.
I've re-installed models etc to see if that's the issue. I haven't re-installed anything else as I am not sure where to start frankly, and don't want to dive into a completely fresh install of InvokeAI as I am running through an UNRAID Docker and it was pretty complex to get it working so well as it has been.
Contact Details
Hey @AmericanPugX - To be clear, what version was your "previously" example generated on? What were settings, and what was the same with your "now" generation?
There have been reports that the Diffusers pipeline has some sensitivity on certain samplers - Reducing CFG or switching to a different sampler like Euler typically resolves the above.
To be clear, what version was your "previously" example generated on? What were settings, and what was the same with your "now" generation?
There have been reports that the Diffusers pipeline has some sensitivity on certain samplers - Reducing CFG or switching to a different sampler like Euler typically resolves the above.
Yeah no definitely want to be super clear:
Both are using my identical models, resolution, steps, CFG, etc - I've attempted every single setting separately in both directions to see if they fix the issue, isolated everything, used same and diff seeds, Euler, DPMPP, etc.
I have tried diff models, resolutions, everything all isolated as the only change to continue to have the same issue.
To be clear, using every single style of CFG, resolution ratio, etc, I used to get fairly amazing images that I could fine tune about 90% of the time, now no matter what I do, even my "safe" settings for god-tier images, they all almost entire come out like the new version about 95% of the time under every single scenario, model, setting, etc.
So there's definite something that's happened, my worst guess is something with the VAEs have changed? I am using global settings, so could something have happened somewhere along the line?
I've really confidently tested around every aspect of the pipeline to feel like if my actual install hasn't been corrupted, it definitely isn't some sort of settings issue.
There's no question what you're experiencing is not a localized issue - My question is largely "where did this change". What version were you on previously?
Sorry, wasn't trying to sound defensive, just feel like being as clear as possible might help.
I'm unsure what version I would've been on, as I don't know where to see my update history for Invoke?
The issues started happening yesterday, when I finally fixed the Conjunction issue which I assume was fixed by updating. I was online using a script that automatically keeps me on the main branch of InvokeAI every time I boot up which is daily, so I would've been fine for example on whatever the updated branch was 3-4 days ago? If there's an easier way for me to see this info let me know.
Making a quick (embarrassing) video here to show the live render in case that helps figure out where it's going wrong.
Realized I started typing a response and didn't hit the big green button!
Deleted your video - No worries.
We have seen some difference in generation since our move to Diffusers, but this happened a while back, which is why I find the timing odd. If you try with the Euler sampler, at 30-40 steps, do you still run into issues? It's seemingly an oddity that happens at lower step counts, higher configs, or with specific samplers (often due to different step count norms for each sampler)
We'll still be investigating this issue in the meantime - just sharing workarounds. If you can help find a pattern here, I'm sure it'd be helpful :)
Haha all good - if it was a while back I doubt that's what is affecting me. At 30-40 steps the same issues happen unfortunately, sometimes a stray one comes out with a good face but then back to worse.
Wish I could do more to try and isolate the issue, will mess around with it more.
Another example/test of old prompt w/ all settings identical. Tried 40 steps as well, still showing this corruption. Added a picture of the steps just before it goes haywire.. it really does seem to be generationally heading to a better pictured.. then suddenly the final few images go completely bonkers.. like it gets way better before it gets worse which feels very specific.
Alright I am sorry to spam, but I think it might be related to negative prompts. Here's my findings:
Obviously the negative prompts affecting most the image ruin most of it, vs the negative prompt that only focuses on skin only shows issues on the skin. Hmmm??
Now that is interesting- @damian0815 any idea why that would be causing this to happen? Is there a dynamic between the unconditioning and the sampler (as I’ve seen Euler, for example, seem to sidestep the issue)
no idea. there was a PR recently that implemented a bunch of new samplers and renamed some, maybe that’s the culprit? id suggest someone who can reproduce this run git bisect to identify the commit where it broke
@damian0815 These are with legacy samplers (e.g., KLMS), and this issue has been happening for a while now
Check your Image Generation Parameters output if you're using the webui. If you see a \n embedded in your prompt string there, then find that place in your input and remove it. This is what I was experiencing as well, with my negative prompts being taken as positive prompts. Upon removing the \n this was improved. Started happening for me in 2.3.5post2 for prompts that were working fine with the embedded new line.
