[bug]: Sliced Attention: `TypeError: '>' not supported between instances of 'str' and 'int'`

[psychedelicious]

Is there an existing issue for this problem?

• [X] I have searched the existing issues

Operating system

Linux

GPU vendor

Nvidia (CUDA)

GPU model

No response

GPU VRAM

No response

Version number

main

Browser

ff

Python dependencies

No response

What happened

With this config:

attention_type: sliced
attention_slice_size: auto # the default

I get this error:

[2024-08-02 11:44:05,459]::[InvokeAI]::ERROR --> Error while invoking session 0a817e9c-5106-4d76-918c-f73e332292c7, invocation 7bb9e6e1-6f7b-434d-bd19-7cf5ee9dbad5 (denoise_latents): '>' not supported between instances of 'str' and 'int'
[2024-08-02 11:44:05,460]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 129, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 289, in invoke_internal
    output = self.invoke(context)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/denoise_latents.py", line 757, in invoke
    return self._old_invoke(context)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/denoise_latents.py", line 986, in _old_invoke
    result_latents = pipeline.latents_from_embeddings(
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/stable_diffusion/diffusers_pipeline.py", line 354, in latents_from_embeddings
    self._adjust_memory_efficient_attention(latents)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/stable_diffusion/diffusers_pipeline.py", line 184, in _adjust_memory_efficient_attention
    self.enable_attention_slicing(slice_size=slice_size)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1635, in enable_attention_slicing
    self.set_attention_slice(slice_size)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1651, in set_attention_slice
    module.set_attention_slice(slice_size)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/diffusers/models/unets/unet_2d_condition.py", line 813, in set_attention_slice
    if size is not None and size > dim:
TypeError: '>' not supported between instances of 'str' and 'int'

The root cause is "auto" attention size is erroneously set to "balanced".

---

"balanced" attention slice size is an invoke-only value. diffusers has"auto" | "max" | int. There's a bug where we don't fall back to the diffusers-supported value of "auto":

https://github.com/invoke-ai/InvokeAI/blob/main/invokeai/backend/stable_diffusion/diffusers_pipeline.py#L180-L184

The elif on L182 should be an if to allow the fallback to work. This is already corrected in the custom attention processor:

https://github.com/StAlKeR7779/InvokeAI/blob/be84746e6783a9259e8cbde96d78fbb8514058bb/invokeai/backend/stable_diffusion/diffusion/custom_attention.py#L67-L71

However, it still feels awkward that both config values of "auto" and "balanced" mean the same thing. Probably we should deprecate one.

We'll also need to update this util:

https://github.com/invoke-ai/InvokeAI/blob/main/invokeai/backend/util/attention.py#L11-L33

What you expected to happen

worksies

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

[psychedelicious]

Also the custom attn processor in mps_fixes.py needs to be updated.