Changing the order of wildcards results in different prompts

[uwidev]

I was experimenting with prompt ordering my outputs seemed to get very different results than expected despite using the same seed. It seems that wildcards seem to be chosen differently somehow depending on the ordering of the wildcards in the prompt. I have no idea if this is intended behavior or not. If anything, it disallows consistency in prompt contents when re-organizing the prompt. I am using this on reForge.

Replicating

Setup

Commit de056ff8 Seed: 234 [x] Fixed seed [x] Automatically purge wildcard cache on every generation [x] Disable de-duplication of wildcards before processing [x] Disable sorting of wildcards before processing [x] Ignore whitespace in prompts: All newlines, tabs, and multiple spaces are replaced by a single space

I'm pretty sure ticking those boxes didn't do anything for this issue, I tried on/off and nothing changed.

Generate with two different prompts. Prompt A

masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo,
{red|blue} hair,
{portrait|full body},

Prompt B

masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo,
{portrait|full body},
{red|blue} hair,

Notice the wildcards swapped places.

Generated Output

Resulting prompt: masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo, portrait, blue hair,

Resulting prompt: masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo, red hair, full body,

Expected

Should've been very similar prompts, just with with only switched positions, not a completely differently chosen prompt. Maybe there was a setting or something I missed?

[songib]

For me the extension didn't work at all, it pulled the name of the files instead of what was inside it. idk what happen there.

[uwidev]

For me the extension didn't work at all, it pulled the name of the files instead of what was inside it. idk what happen there.

Make sure you're pathing correctly to wildcards and they can be detected under the wildcards tab. Also make sure you're using __wildcard__ format, or whatever you set as your parser wrapper.

And please keep discussion related to the issue.

[d-ellis]

That sounds expected. Your input changed, so your output changed. Re-ordering your wildcards means you're changing the order in which they're resolved. Using the same seed will result in the same output, assuming the parameters haven't changed, but in this case they have.

[uwidev]

So even if the seed is the same, the way the prompts are chosen are coupled on their ordering in the prompt itself?

I thought that prompts chosen only based on the seed. If it works like this, then I find it odd because intuitively I would think it's solely based on the seed, not its ordering to other wildcards.

I ticked off "force sorted ordering before processing", and unfortunately the prompt still changed. I was thinking that this might've been a way to force consistent resolution of wildcards but it didn't seem to work.

Is this actually intended behavior? I would understand some change because ordering, but to select a completely different prompt is strange.

[d-ellis]

Well, when you seed a random number generator, you're just giving it a starting point for the sequence of numbers that it will generate. So lets say your seed 234 will generate the sequence 1, 2, 3, 4, 5, .... The first time a random number is generated, it returns 1, the second time it returns 2, and so on. (This is a massive simplification, but illustrates the point)

So when you gave it the prompt:

masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo,
{portrait|full body},
{red|blue} hair,

When it encountered the first place to make a choice, the random number generator returned 1, which meant it resolved to portrait. When it encountered the second place, the generator returned 2, which meant it resolved to blue.

Now consider the prompt where the order has changed:

masterpiece, best quality, good quality, newest, very awa, absurdres, highres, 1girl, solo,
{red|blue} hair,
{portrait|full body},

When it first made a choice, the number generator returned 1 (again), which now resolves to red. When it came to the second choice, the generator returned 2 (again), which is now full body.

I don't know the underlying code being used, but that's exactly how I would expect it to work, and it seems that's how it is working.

[uwidev]

That doesn't make sense. I'm using a static seed 234 and have fixed seed ticked. As per README.md

If the fixed seed checkbox is checked: The same seed is used for all images and prompts. This means the same image is generated repeatedly (this is useful for combinatorial generation).

Even if it's using a different seed, the position that it chooses should be the same (unless it also depends on the contents of the wildcard itself).

~~Here's what I tried with your explanation.~~

{black hair|red hair}, {black eyes|red eyes}

~~Let's say for this generation, the first prompt chose the 1st position (black) and the second prompt choose the 2nd position (red), resulting in black hair, red eyes.~~

~~Now if I switch the prompt ordering, I should expect it to choose the same positions, but instead it chooses the opposite positions, choosing 2nd position then 1st, resulting in black eyes, red hair. This is with fixed seed ticked.~~

Let me do some more testing.

Also Disable sorting of wildcards before processing is a bit vague. Is this referring to all of the wildcard prompts being sorted (i.e. ordered {B} {C} {A} -> {A} {B} {C} and being assigned seeds, 1, 2, 3) or sorting the contents of the wildcard itself (i.e. {B, C, A} -> {A, B, C})?

If the former, I'd assume, then even if I changed the wildcard ordering, it should remain consistent. If the latter, I guess that's for combinatorial generation or something.

[uwidev]

Okay, I think I understand more about how seeds chosen and used to pick a prompt from a wildcard. However, this means that fixed seed is not working as intended?

This test right here.

{black|red|blue} hair,
{black|red|blue} eyes,

If it uses a fixed seed and picks 1st position for the first prompt, I would expect it to also choose the 1st position for the second prompt. But instead it chooses something else entirely. Isn't fixed seed supposed to lock what is chosen since the seed is the same across all prompts? Or is it another misunderstanding on my part?

[timwillhack]

I believe you are missing the point of it. I believe its the starting seed for the full prompt. If you used seed '3' and ran that SAME prompt more than once it would evaluate the same each time it runs. Its useful for rerunning the same prompt to produce the same image for instance from like an old generation that you want to change like max shift or something like that

[uwidev]

I believe you are missing the point of it. I believe its the starting seed for the full prompt. If you used seed '3' and ran that SAME prompt more than once it would evaluate the same each time it runs. Its useful for rerunning the same prompt to produce the same image for instance from like an old generation that you want to change like max shift or something like that

Yes, that's the same thing I said just worded differently. But it doesn't work like that. Despite having fixed seed ticked, generating a grid with different seeds results in completely different prompts chosen.

[timwillhack]

Different seeds would change the entire selection process for EACH set of brackets in your full prompt. When you say for each prompt are you referring to the full generation prompt or are you calling each set of brackets a prompt? I don't think you can get what you want with how it works unless I'm misinterpreting you.

[uwidev]

So when I say "chosen prompt", I'm referring to what the wildcard chooses. And yes, changing the seed is would change the entire selection prompt for each bracket as you are saying. However, "Fixed Seed" is supposed to lock the selection from the first seed generation and apply that across the grid(?).

For example, let's say I have a grid to generate seeds 1, 2, and 3. We have a wildcard as follows: {blue|red|green}. If for seed 1, the chosen prompt (wildcard) is "red", then with fixed seed enabled, it should be "red" for seeds 2, and 3, correct? This is what "Fixed Seed" is supposed to do(?), except it doesn't to work that way, or do anything at all?

The original issue, is that altering the ordering of wildcards (not their contents, the entire wildcard itself with another wildcard) results in completely different generations. I was under the impression that wildcards were sorted before being assigned seeds to choose the prompt from wildcard, but that option doesn't seem to do anything. Hence, why I'm looking at the other options and seeing if they solve the original issue.

[timwillhack]

If you set seed to anything, lets say '1' and have the following prompt "{blue|red|green} {light|cube|carpet}" in this fake example lets say seed 1 picks "blue carpet". If you leave the seed to '1' and run the same generation again, it will select 'blue carpet' again. If you change the seed to anything else it will completely randomize the full prompt (every set of brackets would potentially be different than seed 1) again and might become 'red cube' on seed 2 for instance. If you kept the seed at '2' again it would select 'red cube' again. I hope that clarifies why your example doesn't work the way you think it should: 'For example, let's say I have a grid to generate seeds 1, 2, and 3. We have a wildcard as follows: {blue|red|green}. If for seed 1, the chosen prompt (wildcard) is "red", then with fixed seed enabled, it should be "red" for seeds 2, and 3, correct?' You cannot (that I know of) lock in a seed for each set of brackets, just the entire prompt with as many sets of brackets as you use. I hope that brings clarity to your issue. The 'apply across the grid'

if you are in A1111 and want the same seed for the full grid you can probably do something like set seed to "1,1,1,1" for a 2x2 grid. I am in comfy so I haven't touched sdforge/a1111 in a while.