[Bug]: adding to negative prompt from extension can fail if batch size > 1

[rubberbaron]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

I modified the wildcards extension, which modifies the positive prompt, to also modify the negative prompt. Because it is random, it may add different amount of things to each item within a batch.

Probably the trigger for the bug is when one item in the batch uses 75 or less negative prompt words, and another item in the batch uses more than 75 negative prompt words.

There errors are generally one of these two:

• RuntimeError: The expanded size of the tensor (77) must match the existing size (154) at non-singleton dimension 0. Target sizes: [77, 768]. Tensor sizes: [154, 768]
• RuntimeError: The expanded size of the tensor (154) must match the existing size (77) at non-singleton dimension 0. Target sizes: [154, 768]. Tensor sizes: [77, 768]

Presumably the fix is to replicate the logic in https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/prompt_parser.py#L249 for the negative prompt.

Steps to reproduce the problem

1. Modify wildcards.py or some other extension to randomly add ",cow, bird, dog, cat, goat, snake" to the negative prompt
2. Use any prompt, e.g. "forest"
3. Use a negative prompt like "a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a" that brings the negative prompt close to 75 words.
4. Run with a batch size > 1, until the randomness produces negative prompts both above and below 75 in the same batch

What should have happened?

Shouldn't fail.

Commit where the problem happens

commit: 6cff4401

What platforms do you use to access the UI ?

No response

What browsers do you use to access the UI ?

No response

Command Line Arguments

xformers

List of extensions

wildcards

Console logs

Error completing request
Arguments: ('task(40iarytany9fnt5)', 'forest __test__', 'a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a', [], 1, 0, False, False, 1, 4, 7, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, False, 'none', 'None', 1, None, False, 'Scale to Fit (Inner Fit)', False, False, False, False, False, '', 1, '', 0, '', 0, '', True, False, False, False) {}
Traceback (most recent call last):
  File "C:\ai\sdweb\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "C:\ai\sdweb\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "C:\ai\sdweb\modules\txt2img.py", line 52, in txt2img
    processed = process_images(p)
  File "C:\ai\sdweb\modules\processing.py", line 485, in process_images
    res = process_images_inner(p)
  File "C:\ai\sdweb\modules\processing.py", line 629, in process_images_inner
    samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
  File "C:\ai\sdweb\modules\processing.py", line 824, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "C:\ai\sdweb\modules\sd_samplers.py", line 544, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "C:\ai\sdweb\modules\sd_samplers.py", line 447, in launch_sampling
    return func()
  File "C:\ai\sdweb\modules\sd_samplers.py", line 544, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "C:\ai\sdweb\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\ai\sdweb\repositories\k-diffusion\k_diffusion\sampling.py", line 145, in sample_euler_ancestral
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "C:\ai\sdweb\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ai\sdweb\modules\sd_samplers.py", line 318, in forward
    uncond = prompt_parser.reconstruct_cond_batch(uncond, self.step)
  File "C:\ai\sdweb\modules\prompt_parser.py", line 223, in reconstruct_cond_batch
    res[i] = cond_schedule[target_index].cond
RuntimeError: The expanded size of the tensor (77) must match the existing size (154) at non-singleton dimension 0.  Target sizes: [77, 768].  Tensor sizes: [154, 768]

Additional information

No response