UnivAICharGen icon indicating copy to clipboard operation
UnivAICharGen copied to clipboard

Negative prompt tensor size mismatch during batch generation

Open TxcTrtl opened this issue 1 year ago • 0 comments

I'm using version 12d04605 It is currently possible to generate tensor size mismatch errors during batch generation, if there are negative prompts in the wildcards.

Example: I have the following static negative prompt: (abstract paint), (3d, realistic:1.3), (low quality, worst quality:1.4), (bad anatomy), extra digit, fewer digits, (extra arms:1.2), bad hands, by (bad-artist:0.6), bad-image-v2-39000, Asian-Less-Neg -> this amounts to ~43 tokens according to the A1111 UI.

I also have the following somewhere in my positive prompt: **(melee weapon, sword, axe, mace, blade, dagger, modern technology, modern weapons, automatic rifle, backpack, cigarette, jetpack, metal, cars:1.2),**

If I would add this to the static negative prompt, it would bring it over the 75 token limit (~77), but I want to keep it in the positive prompt, as it is used as part of a wildcard.

If I run batch generation (batch size > 1), the following error occurs fairly frequently: RuntimeError: The expanded size of the tensor (77) must match the existing size (154) at non-singleton dimension 0. Target sizes: [77, 768]. Tensor sizes: [154, 768]

I believe that the root cause of the issue is indeed that the negative prompts of the individual images in the batch have different tensor sizes. Manual workaround for the issue is to insert BREAK at the end of the static negative prompt, so that it always creates a new block, for the dynamically inserted negative prompts, regardless of how long the static negative prompt is.

However, I suggest an improvement of the extension to handle such cases. It should be possible to preprocess negative prompts in case of batch generation, and align the tensor sizes of the negative prompts, eg. by inserting 'BREAK' at the end of negative prompts that are below the maximum tensor size, until they match.

TxcTrtl avatar Oct 08 '23 12:10 TxcTrtl