Using a certain wildcard seems to inject parameters into image generation that ruin images - particularly with Pony v6.

[a-l-e-x-d-s-9]

trafficstars

It's very strange problem, but I was able to pinpoint and reduce it to a single wildcard with 2 lines. This problem discoverable mainly with Pony v6 checkpoint - probably due to Pony v6 being particularly sensitive to the change. Please use __prompt_test__ wildcard - it has 2 prompts: prompt_test.txt

Use the following parameters for the image and batch size 3:

__prompt_test__
Negative prompt: open mouth, bad quality, poor quality, disfigured, jpg, toy, bad anatomy, missing limbs, missing fingers, ugly, scary, watermark
Steps: 20, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 7, Seed: 10, Size: 640x960, Model hash: 67ab2fd8ec, Model: ponyDiffusionV6XL_v6StartWithThisOne, RNG: CPU, Template: __prompt_test__, Negative Template: "open mouth, bad quality, poor quality, disfigured, jpg, toy, bad anatomy, missing limbs, missing fingers, ugly, scary, watermark", Emphasis: No norm, Version: v1.10.1

And you should get image that look like this: Notice the artifacts and very distorted image that you get. If you replace the prompt with the prompt it supposedly using: illustration, digital art, wild life, nature, trees, grass, bushes, flowers, mountain, wild nature, digital painting, medium shot, sharp focus, focused, cgsociety, analytical art, detailed, popart style, (cherrmous style:0.8), You will get this image instead: We can see a proper flowers, and whole image is lacking artifacts. The image that uses wildcard has artifacts that indicate something very wrong with image parameters. I think it's related to: Auto 1111 Attention/Emphasis Auto 1111 Alternating Words Which are used in my wildcards prompts. I have a lot of complex prompts in wildcard file that using emphasis, alternating words and other structures supported by Auto 1111. I suspect that Dynamic Prompts confusing the syntax with something else and injecting parameters into generated images - causing artifacts in the results. It's not something that noticeable with most checkpoints, but Pony v6 seems to be particularly vulnerable to these undesired changes. The problem exist with Auto 1111 and Forge. And it's not a new issue. This issue not influence 100% of the seeds, and some seeds seem to work ok. With the provided parameters, those are a few seeds that showing the issue: 10,14,15,18,19.

[a-l-e-x-d-s-9]

I tried to simplify the prompts in the wildcard - doesn't seem to help. But using batch size 1 seems to make a fine image each time.

[a-l-e-x-d-s-9]

@teamolhuang I appreciate the attempt to help. Yes, the file has two lines - two prompts, I wrote it in the bug report. The exact seeds that have the issue specified, particularly seed 10 - which clearly has prompt from line 1. Both of the prompts from file are working perfectly fine when written directly in the prompt. The issue appears only when dynamic prompts extension is used. I explained exactly how to reproduce it, and I already reproduced it on another system with another user.

[DarkPulse]

@a-l-e-x-d-s-9 I think that Dynamic Prompts has real problems with values with weights in them in Wildcard files.

I've had other stuff break, and every single time I got generated some sort of random noise pattern or glitchy part, it was either token length (meaning inserting some BREAK here and there, but your prompt is well short of that, so it's not likely to be it), or a weight value inside of some parentheses.

Try nuking that 0.8 at the end of the prompt, and see if it works then.

Last but not least, it could be due to the oddball resolution - it's not ideal for PonyXL. The "sweet spot" for generation is generally if you add the numbers to both sides and come up to 2x the "baseline" width/height. For SD 1.5, this was 512+512 = 1024; for SDXL/Pony/Flux this is 1024 + 1024 = 2048. 640x960 can still work, but it only totals out to 1600, so you're a bit short - it might work, but it could also glitch out as there's not enough noise in the right places that the model is expecting.

Try to get the numbers to add up to 2048 if at all possible; while it's not a guarantee (it depends on how well the model itself was trained), having the correct multiple usually means that there won't be any issues due to noise (or if they are, it will be fairly minimal and more based on the generator giving you a bad run). Give 864x1184 a shot.

[a-l-e-x-d-s-9]

I've tried to remove weights - it hasn't helped. It's not a matter of image size, image with the same size and the same prompt, without using Dynamic Prompts are generated always without issues, with all other parameters being the same. The only thing that helped with Dynamic Prompts was to use batch size 1. So the problem is probably related to some edge case overriding or calculating wrong parameters for images in case of batch size above 1.

[DarkPulse]

There's definitely some weirdness if you've got ADetailer installed and running batches (it's ruining embedding generation info in images/writing them to CSVs), so I wonder if that is tangentially related to this. No real idea though.

[akx]

Last but not least, it could be due to the oddball resolution - [...]

For that, you may be interested in my res-helper tool – the SDXL trained resolutions are the ones with a golden background.

[2dameneko]

Same problem on Forge and reForge. NoobAI Vpred 1.0 model, 2-3 wildcards like {tag1|tag2|} in prompt. When using batch size >1 (e.g 8), after a while mostly glitchy blocky images start to be generated. Disabling batch size (set to 1) solves the problem, but it would be nice if the problem'll be solved w/o disabling batch size > 1. Enabling/disabling/unistalling Adetailer has no effect on the issue. It continues to reproduce.

Updated: Figured out the cause of the problem: if one or more {tag1|tag2|} constructs are used. The important place is the 3rd position, empty tag. A glitch occurs if promt has such wildcards. If you edit promt so that there are no empty tags in the wildcards, the breakage does not happen.

So, to reproduce the problem, you need to take promt with several such constructs and set infinite generation with batch size 8. Usually by 3-4 batches you already start seeing glitchy images.