[bug]: Crashes when prompt is longer than 597 characters

[kasbah]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS

Linux

GPU

cuda

VRAM

8GB

What happened?

When the prompt is longer than 597 characters, inoke.py crashes with a CUDA error.

This prompt works (597 characters):

caaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaat

while this one crashes (598 characters):

caaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaat

Screenshots

No response

Additional context

Log of failed attempt: log.txt

Contact Details

No response

[jasoa]

I'm experiencing a similar crash on longer prompts on windows. It's happening on a prompt longer than 386 characters for me. log_win.txt

[svumme]

same problem on windows!

[i3oc9i]

Confirmed in MacOs with version 2.2.4 and last commit 474220ce8e43773c35ad19d281293243c1d2f0ff

The max working prompt on CLI is 337 characters long.

''' this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt this is a very very very long prompt1234 '''

[DrakenZA]

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasGemmStridedBatchedExFix( handle, opa, opb, m, n, k, (void*)(&falpha), a, CUDA_R_16F, lda, stridea, b, CUDA_R_16F, ldb, strideb, (void*)(&fbeta), c, CUDA_R_16F, ldc, stridec, num_batches, CUDA_R_32F, CUBLAS_GEMM_DEFAULT_TENSOR_OP)

When prompt too long.

[Barafu]

It is probably at least connected with issue #1908

[i3oc9i]

It does not crash with last commit 64339af2dcda21aa79843f7f1bf5d1ede9d52406

[nelsonre]

I've updated my install, and it stills crashes on MacOS when the prompt is 337 chars or more:

index 77 is out of bounds for dimension 0 with size 77


Traceback (most recent call last):
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/backend/invoke_ai_web_server.py", line 1125, in generate_images
    self.generate.prompt2image(
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/generate.py", line 492, in prompt2image
    results = generator.generate(
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/generator/base.py", line 98, in generate
    image = make_image(x_T)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/generator/txt2img.py", line 42, in make_image
    samples, _ = sampler.sample(
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/ksampler.py", line 226, in sample
    K.sampling.__dict__[f'sample_{self.schedule}'](
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/k_diffusion/sampling.py", line 128, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/ksampler.py", line 52, in forward
    next_x = self.invokeai_diffuser.do_diffusion_step(x, sigma, uncond, cond, cond_scale)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/shared_invokeai_diffusion.py", line 107, in do_diffusion_step
    unconditioned_next_x, conditioned_next_x = self.apply_standard_conditioning(x, sigma, unconditioning, conditioning)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/shared_invokeai_diffusion.py", line 123, in apply_standard_conditioning
    unconditioned_next_x, conditioned_next_x = self.model_forward_callback(x_twice, sigma_twice,
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/ksampler.py", line 38, in <lambda>
    model_forward_callback=lambda x, sigma, cond: self.inner_model(x, sigma, cond=cond))
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/k_diffusion/external.py", line 114, in forward
    eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/k_diffusion/external.py", line 140, in get_eps
    return self.inner_model.apply_model(*args, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/ddpm.py", line 1441, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/ddpm.py", line 2167, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/diffusionmodules/openaimodel.py", line 806, in forward
    h = module(h, emb, context)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/diffusionmodules/openaimodel.py", line 88, in forward
    x = layer(x, context)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/attention.py", line 271, in forward
    x = block(x, context=context)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/attention.py", line 221, in forward
    return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/diffusionmodules/util.py", line 159, in checkpoint
    return func(*inputs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/attention.py", line 226, in _forward
    x += self.attn2(self.norm2(x.clone()), context=context)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/modules/attention.py", line 199, in forward
    r = self.get_invokeai_attention_mem_efficient(q, k, v)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/cross_attention_control.py", line 295, in get_invokeai_attention_mem_efficient
    return self.einsum_op_mps_v1(q, k, v)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/cross_attention_control.py", line 250, in einsum_op_mps_v1
    return self.einsum_lowest_level(q, k, v, None, None, None)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/cross_attention_control.py", line 229, in einsum_lowest_level
    self.attention_slice_calculated_callback(attention_slice, dim, offset, slice_size)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/shared_invokeai_diffusion.py", line 69, in <lambda>
    lambda slice, dim, offset, slice_size, key=key: callback(slice, dim, offset, slice_size, key))
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/shared_invokeai_diffusion.py", line 61, in callback
    saver.add_attention_maps(slice, key)
  File "/Users/nelsonre/invokeai/.venv/lib/python3.10/site-packages/ldm/models/diffusion/cross_attention_map_saving.py", line 31, in add_attention_maps
    maps = maps[:, :, self.token_ids]
IndexError: index 77 is out of bounds for dimension 0 with size 77

[github-actions[bot]]

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

[einarpetersen]

Trying to generate from command line and after tweaking prompt to optimize results I am gettin this regarding prompt length Fetching 15 files: 0%| | 0/15 [00:00<?, ?it/s] Fetching 15 files: 100%|██████████| 15/15 [00:00<00:00, 80867.04it/s] | Default image dimensions = 512 x 512

Model loaded in 1.65s Textual inversions available: Setting Sampler to k_lms (LMSDiscreteScheduler) #"Commander Wilson, standing tall and resolute in front of his spacecraft, clad in his futuristic suit, determined to save humanity, face set in steely resolve, high resolution, realistic art by Brian Sum [bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated, disfigured, kitsch, ugly, oversaturated, grain, low-res, deformed hands, deformed, blurry, poorly drawn face, mutation, extra limb, missing limb, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, ugly, disgusting, poorly drawn, childish, malformed fingers, mutilated, mangled, old, surreal, pixelated, distortion, clipped] --facetool -G0.9 -ft gfpgan -cfg_scale -C 7.5 --outdir /home/id/data/MyPythonPrograms/a_dir-somewhwr/e___images". Patchmatch initialized Prompt is 9 token(s) too long and has been truncated Prompt is 6 token(s) too long and has been truncated Prompt is 8 token(s) too long and has been truncated

Generating: 0%| | 0/1 [00:00<?, ?it/s] - Generating using CPU - Debian 11 - Linux