InvokeAI
InvokeAI copied to clipboard
invoke-ai
Backport MemoryEfficientCrossAttention from Stability-AI/stablediffusion
this is just a copy and paste from original https://github.com/Stability-AI/stablediffusion codebase. it effectively enables cross attention optimizations via https://github.com/facebookresearch/xformers on invokeai (my 3090 ti goes from 10it/s to 18it/s. still the time from interaction to result is slightly slower than https://github.com/AUTOMATIC1111/stable-diffusion-webui, there's probably more room for optimizations.
xformers is also supported by the attention models in diffusers, so this is something we get with #1583.
Or more accurately: that diffusers PR uses xformers if it is installed. Installing it is still a piece of work that will need to be added to the release packages.
@keturn installing xformers is as easy as conda install xformers -c xformers/label/dev, at least on linux (pre-compiled binaries!).
@keturn installing xformers is as easy as
conda install xformers -c xformers/label/dev, at least on linux (pre-compiled binaries!).
It's quite the headache on Mac and Windows where most of the user base is. So gotta work through it.
ok reworked the PR to only include MemoryEfficientCrossAttention and is only activated if xformers is present. tested with and without on my linux machine and works, can't test on other platforms however. btw totally supporting switching to diffusers (honestly ldm codebase is not the greatest) but i think some will appreciate this patch meanwhile diffusers work is ready.
@knoopx;
installing xformers is as easy as
conda install xformers -c xformers/label/dev, at least on linux (pre-compiled binaries!).
Not everyone uses Conda
thanks for the contribution, but i think we should hold off on this until diffusers gets merged.
The following part of the code must be changed before this can be merged: https://github.com/invoke-ai/InvokeAI/blob/ce17051b28c7d08560c4f0f102888633933ae83d/ldm/models/diffusion/cross_attention_control.py#L344 Also, regarding xFormers installation, check this https://github.com/invoke-ai/InvokeAI/pull/2360#issuecomment-1396178695
