ComfyUI
ComfyUI copied to clipboard
comfyanonymous
How to force ComfyUI to use xformers?
When I installed comfy it showed loading xformers [version] when I started it. I started messing with the flags because I had trouble loading the refiner, however I was not able to turn on xformers after. There is no flag to turn on xformers just to disable it. When I load it even with no flags I still don't see xformers showing up, How do I fix it? How to force Comfy to use xformers?
I do have xformers installed. pip install xformers shows requirement already satisfied, I am on Ubuntu, RTX 3060 GPU with 16 RAM.
If it's not using xformers it either means it's not installed or you launched it with a parameter like: --use-pytorch-cross-attention
@comfyanonymous
I am not using any flags. However it seems to be using pytorch cross attention anyway. I have not edited any config files.
python main.py
Total VRAM 12050 MB, total RAM 15951 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
Using pytorch cross attention
Starting server
To see the GUI go to: http://127.0.0.1:8188
Xformers was being used when I first installed it. To show xformer is installled.
(sd) (base) sonnet@sonnet-MS-7A70:/media/sonnet/AAAA42E0AA42A8A1/Ai/ComfyUI$ pip install xformers
Requirement already satisfied: xformers in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (0.0.20)
Requirement already satisfied: numpy in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (1.25.1)
Requirement already satisfied: pyre-extensions==0.0.29 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (0.0.29)
Requirement already satisfied: torch==2.0.1 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (2.0.1)
Requirement already satisfied: typing-inspect in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from pyre-extensions==0.0.29->xformers) (0.9.0)
Requirement already satisfied: typing-extensions in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from pyre-extensions==0.0.29->xformers) (4.7.1)
Requirement already satisfied: filelock in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.12.2)
Requirement already satisfied: sympy in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (1.12)
Requirement already satisfied: networkx in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.1)
Requirement already satisfied: jinja2 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.1.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.99)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.101)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (8.5.0.96)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.10.3.66)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (10.9.0.58)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (10.2.10.91)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.4.0.1)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.4.91)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (2.14.3)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.91)
Requirement already satisfied: triton==2.0.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (2.0.0)
Requirement already satisfied: setuptools in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->xformers) (67.7.2)
Requirement already satisfied: wheel in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->xformers) (0.40.0)
Requirement already satisfied: cmake in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from triton==2.0.0->torch==2.0.1->xformers) (3.27.0)
Requirement already satisfied: lit in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from triton==2.0.0->torch==2.0.1->xformers) (16.0.6)
Requirement already satisfied: MarkupSafe>=2.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from jinja2->torch==2.0.1->xformers) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from sympy->torch==2.0.1->xformers) (1.3.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from typing-inspect->pyre-extensions==0.0.29->xformers) (1.0.0)
ITs not using xformers like it should so you wont see speed gain Should be half the speed, with xformers 1.5 secs, without xformers 3 secs, its like this for me in auto1111 but here it just does not work even if it says its installed
Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.
I'm running a 3060, and just got started in ComfyUI (coming from A1111). Xformers has a significant performance improvement. Would love to see this fixed in a standard way. 🙂
i guess the only fix is for developers of standalone comfyui include xformers by default, or make it search and remember path for already installed version on launch (since its already checks for it on install ¯_(ツ)_/¯). Automatic1111 also does this: i have ver 22 instaled but it uses ver 20 in \side-pakcages folder which it includes by default (you could just copy from a1111 folder since you have it)
Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.
Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.
so there is no way of making it "faster' woth SDP or XFORMERS?
Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.
so there is no way of making it "faster' woth SDP or XFORMERS?
You can install xformers manually. Then ComfyUI will use xformers automatically. However, using xformers doesn't offer any particular advantage because it's already fast even without xformers.
I did some thread necromancy to find this. It might be worth pointing that out in the loading log on server start, because I was scratching my head as to 'why wasn't it using xformers' followed by 'but why is it using cross attention even though i didn't flag it'.
We all have FOMO on speed :) Good to know we're not missing anything out!
Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.
Dear friend, you've just saved my mind! I've almost lost my faith by this day! Thank you so much! Dancing)))
On my machine, today, with fully up-to-date everything (pytorch 2.4, CUDA 12.4), generations take roughly 2-to-3x slower without xformers. It is not the case that xformers is somehow no longer needed.
That being said, there is definitely a need to have a force-xformers flag. I have just finished compiling/installing xformers, only for Comfy to decide not use it for some unknown reason. However, when I run Comfy with the --quick-test-for-ci flag, it confirms that xformers is available to the system.
Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.
Or within a Stability Matrix Environment with installed A1111: copy it from there ...
On my machine, today, with fully up-to-date everything (pytorch 2.4, CUDA 12.4), generations take roughly 2-to-3x slower without xformers. It is not the case that xformers is somehow no longer needed.
That being said, there is definitely a need to have a force-xformers flag. I have just finished compiling/installing xformers, only for Comfy to decide not use it for some unknown reason. However, when I run Comfy with the --quick-test-for-ci flag, it confirms that xformers is available to the system.
There is still no support for CUDA 12.4 of xformers, have you downgraded to 12.1 or stayed in 12.4 ?
There is still no support for CUDA 12.4 of xformers, have you downgraded to 12.1 or stayed in 12.4 ?
https://github.com/facebookresearch/xformers/issues/1079#issuecomment-2314133170
Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.
@comfyanonymous I use ComfyUI Portable for Windows. I'm trying to use a florence model called "MiaoshouAI/Florence-2-large-PromptGen-v15" and I keep getting the error:
ValueError: The checkpoint you are trying to load has model type florence2 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
It looks like some things still need xformers to be installed. I think it might make sense to add xformers back into the install.
For now, I have done as a previous commenter suggested and copied both xformers folders from my AppData\Local\Programs\Python\Python310\Lib\site-packages directory and pasted them into my \python_embedded\Lib\site-packages folder. Now, Comfy properly runs xformers at startup.
Having said all this... I really have no idea what I'm talking about. 😆 I am probably WILDLY off-base about what's causing this issue. Any help would be appreciated.
