🔴 xformers not found! Please install it before trying to use it.

[IronWolve]

Describe the bug

Did both pip install xformers, and xformers in /text-generation-webui directory. Evan ran the installer again.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Xformers

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

xformers missing

Screenshot

No response

Logs

🔴 xformers not found! Please install it before trying to use it.
Loading the extension "api"... Ok.
Loading the extension "sd_api_pictures"... Ok.
Loading the extension "gallery"... Ok.
Loading the extension "send_pictures"... Starting KoboldAI compatible api at http://127.0.0.1:5000/api
Ok.

System Info

AMD 5950x, Nvidia 4090, Win11

[Ph0rk0z]

yes but did you install it inside conda or the correct venv?

[IronWolve]

Should be the correct one, I ran microconda first. I'm using the windows installer.

pip shows it installed.

xformers 0.0.18

[Ph0rk0z]

maybe this check fails on windows

try:
        import xformers.ops
    except Exception:
        print("🔴 xformers not found! Please install it before trying to use it.", file=sys.stderr)

[Gitbreast]

Same situation, why?

[Ph0rk0z]

Open up a python .. write import xformers.ops, what does it say?

[Gitbreast]

@Ph0rk0z I got this, means I need to upgrade something?

[Ph0rk0z]

Yea, you have CPU pytorch.

[Ketorami]

I have the same problem. My notifications are like this. but when i open the WebUI , I still get error that not found

[Ph0rk0z]

Triton doesn't work on windows I think.

[Gitbreast]

Yea, you have CPU pytorch.

So I need to install a PyTorch for GPU and upgrade Python?

[Ketorami]

You can install PyTorch form this repo https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/8691#discussion-4972715

[Ph0rk0z]

Basically.. you need a cuda pytorch to do GPU inference.

[EnviralDesign]

the xformers library def needs to be installed in the env, which is managed by mamba. I was able to get it working with these steps, hopefully it helps:

1. run micromamba-cmd.bat, this will open a terminal, in the text-generation-webui's env activated already.
2. run pip install xformers
3. close that terminal, and close, then restart webui start-webui.bat

[Ketorami]

the xformers library def needs to be installed in the env, which is managed by mamba. I was able to get it working with these steps, hopefully it helps:

1. run micromamba-cmd.bat, this will open a terminal, in the text-generation-webui's env activated already.
2. run pip install xformers
3. close that terminal, and close, then restart webui start-webui.bat

Can you elaborate? Where is "text-generation-webui's env"? in oobabooga-windows\installer_files? And how i run micromamba-cmd.bat?

[EnviralDesign]

The env and the mamba stuff is here:

I'm on windows, so for me the micromamba-cmd.bat is located here. I was able to double click and it opened up ready to use.

[Ketorami]

Sad TT

[EnviralDesign]

that's weird! I'm not sure. fyi for context I used the one click installer to set everything up, not sure if it would behave differently if you installed it another way or manually.

[tonyzehs]

I used the new installer zip, did a fresh install in a directory with no spaces. That worked for me.

[Jonseed]

After installing xformers, I get the Triton not available message, but it will still load a model and the webui. It also says "Replaced attention with xformers_attention" so it seems xformers is working, but it is not any faster in tokens/sec than without --xformers, so I don't think it is completely functional. What is Triton?

[tonyzehs]

After installing xformers, I get the Triton not available message, but it will still load a model and the webui. It also says "Replaced attention with xformers_attention" so it seems xformers is working, but it is not any faster in tokens/sec than without --xformers, so I don't think it is completely functional. What is Triton?

Search Triton in the existing issues on github, I think someone answered it being a Linux specific thing. And yeah, xformers is working for you now.

[Jonseed]

If it is specific to Linux, why is it trying to run Triton on Windows? And if xformers is working, why is the inference speed not any faster than without it? I get about 3 tokens/s with or without it with my 3060 12GB card.

[tonyzehs]

If it is specific to Linux, why is it trying to run Triton on Windows? And if xformers is working, why is the inference speed not any faster than without it? I get about 3 tokens/s with or without it with my 3060 12GB card.

Maybe the base code is the same across platforms, so updating/maintaining is easier for the devs? As for the speed increase, I personally found it to also be minimal, unlike Stable Diffusion. If anything, when running larger models the AI generations seem to spit out things that make less sense. (just limited personal observations, not tested in a controlled environment) here's a thread of people experimenting with it:

https://github.com/oobabooga/text-generation-webui/pull/950

[Ph0rk0z]

It was mentioned xformers may use less memory.

[Stoxis]

The env and the mamba stuff is here:

I'm using the new installation, that batch script doesn't exist anymore. How would you go about downloading the library now? Thank you!

[sparky002]

open oobabooga_windows/cmd_windows.bat. Then run pip install xformers. Then open oobabooga_windows/webui.py, line 146 run_cmd("python server.py --chat --model-menu --model_type llama --xformers") =)

[bkutasi]

I added xfromers into oobabooga_windows/text-generation-webui/requrments.txt and it installed the latest xfromers, but I get a triton error when I use the --xformer parameter. First 4 lines at launch: Gradio HTTP request redirected to localhost :) A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' bin D:\Other_projects\oobabooga\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117_nocublaslt.dll The speed doesn't change, I tried reinstalling it with the new release, but it's the same.

[barleyj21]

Yes, just today I tried to reinstall oobabooga with oobabooga_windows.zip and despite that I answered A to install for nvidia GPU, it still installed pytorch for cpu into venv. Used this https://github.com/oobabooga/text-generation-webui/issues/1252#issuecomment-1511211024 from this thread to reinstall it and then installed "pip3 install xformers" - it's now 0.0.19 version by default. Now it's working, even though it's still giving "no triton" error

UPD: yes, it's starting and loading model, but it gives error when trying to generate. Had to use --sdp-attention instead and it's working alright

[github-actions[bot]]

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

[MistakingManx]

the xformers library def needs to be installed in the env, which is managed by mamba. I was able to get it working with these steps, hopefully it helps:

1. run micromamba-cmd.bat, this will open a terminal, in the text-generation-webui's env activated already.
2. run pip install xformers
3. close that terminal, and close, then restart webui start-webui.bat

This broke my installation completely, because it was successful ;-; Well I mean it still works, but it's significantly slower "triton" doesn't exist, and I don't think it's doing what it's supposed to.