[bug]: RTX 5090 support missing

[GewoonJaap]

Is there an existing issue for this problem?

• [x] I have searched the existing issues

Operating system

Windows

GPU vendor

Nvidia (CUDA)

GPU model

RTX 5090 FE

GPU VRAM

32GB

Version number

5.6.2

Browser

Firefox

Python dependencies

{ "accelerate": "1.0.1", "compel": "2.0.2", "cuda": "12.4", "diffusers": "0.31.0", "numpy": "1.26.4", "opencv": "4.9.0.80", "onnx": "1.16.1", "pillow": "11.1.0", "python": "3.10.11", "torch": "2.4.1+cu124", "torchvision": "0.19.1+cu124", "transformers": "4.46.3", "xformers": "0.0.28.post1" }

What happened

Getting the following error while trying to generate an image:

Error while invoking session d47d1ba3-bc52-4d04-b425-ce08a91d84df, invocation 28b7c880-8463-4e2c-8ed2-4c674f25a610 (flux_text_encoder): CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

[2025-02-25 19:18:26,153]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\app\services\session_processor\session_processor_default.py", line 129, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\app\invocations\baseinvocation.py", line 300, in invoke_internal
    output = self.invoke(context)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\app\invocations\flux_text_encoder.py", line 60, in invoke
    t5_embeddings = self._t5_encode(context)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\app\invocations\flux_text_encoder.py", line 78, in _t5_encode
    with (
  File "contextlib.py", line 135, in __enter__
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\load_base.py", line 75, in model_on_device
    self._cache.lock(self._cache_record, working_mem_bytes)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\model_cache\model_cache.py", line 52, in wrapper
    return method(self, *args, **kwargs)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\model_cache\model_cache.py", line 256, in lock
    self._load_locked_model(cache_entry, working_mem_bytes)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\model_cache\model_cache.py", line 328, in _load_locked_model
    model_bytes_loaded = self._move_model_to_vram(cache_entry, vram_available + MB)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\model_cache\model_cache.py", line 350, in _move_model_to_vram
    return cache_entry.cached_model.full_load_to_vram()
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\model_manager\load\model_cache\cached_model\cached_model_only_full_load.py", line 78, in full_load_to_vram
    self._model.load_state_dict(new_state_dict, assign=True)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\nn\modules\module.py", line 2201, in load_state_dict
    load(self, state_dict)
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  [Previous line repeated 4 more times]
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\nn\modules\module.py", line 2183, in load
    module._load_from_state_dict(
  File "C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\invokeai\backend\quantization\bnb_llm_int8.py", line 58, in _load_from_state_dict
    assert weight_format == 0
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

What you expected to happen

I expected an image to be generated

How to reproduce the problem

Bootup Invoke.AI on a RTX 50xx card Generate an image Observe the error.

Additional context

Seems like a new CUDA version is being used by the RT50 series cards: https://github.com/lllyasviel/Fooocus/issues/3862#issuecomment-2662913403

Discord username

No response

[GewoonJaap]

C:\Users\Jaap\AppData\Roaming\StabilityMatrix\Packages\InvokeAI\venv\lib\site-packages\torch\cuda\__init__.py:230: UserWarning: 
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

[Andyx1976]

sadly not an option if one uses the launcher. i think comfyui has it's first monthyversary of supporting 50xx coming up.

[GewoonJaap]

sadly not an option if one uses the launcher. i think comfyui has it's first monthyversary of supporting 50xx coming up.

Why not? Using updated packages that support the new CUDA version should fixt it?

[Andyx1976]

the entire thing with the launcher is that i t does all that automatically (yes, for noobs, that's me). and it installs an "old" version. maybe If it would run with a nightly 12.8 build but idk how.

[GewoonJaap]

Ah I understand. Yeah with the StabilityMatrix launcher you have to option to override the Python modules or use a different version of invokeAI. I override the pytorch modules but then xformers start throwing compatibility errors because they also haven't updated

[GewoonJaap]

Anyplan to support RTX 50 series?

[etakarine]

Any change we will see 50 series support?

[Andyx1976]

i managed to do it with the latest pytorch and it worked fine. "Thankfully" the stupid launcher"updated a newer pytorch with the old one. Even the most basic installer questions if a newer version should be removed. So trying to remember and doing it again. i update here in a moment. Needs python of course but it was simple enough for me.

[GewoonJaap]

Can you send how you did it? Could you just pip install pytorch or did you have to use some local/other repo with the override?

[Andyx1976]

Update: even simpler solution now (5.10.1), It's here (last post in this thread, https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2816987958 ) IF you use the launcher!

(get a current one! version 1.5.0). If not, this post here might still be useful.

OLDER post for older versions/launchers and manual installations (These are outdated and not needed anymore for the current launcher version!!! Do the simpler version above or in the later post) IF you have a manual install or an older version (5.8, 5.9) they might still work:

Final update to this post: A few posts down i have a streamlined, less rambly version of how it works. https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2717003528

So, don't worry it looks way more complicated than it is (w python installed and pathed), this makes RTX50 work flawlessly with invokeai launcher edition. It REALLY should not be necessary btw. the short version: no, the pip pytorch alone didn't work, it installed but invoke gave me a cuda kernel error. But i only needed to use uv (invoke uses this in their full manual installation instructions too). which installs some other needed stuff automatically. But with that it was all it needed ( activate env, then uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall )

So here is the longer noob guide that i would have needed. sorry for the pros :) Python installed, prompt going:

First you need uv installed, replacement for pip, it does install some requirements automatically when needed, which just pip doesn't). For that at https://docs.astral.sh/uv/getting-started/installation/ they have a simple windows setup-launcher among other ways. then: copy my line below or go to https://pytorch.org/get-started/locally/ At the bottom select: nightly, windows, pip, cuda 12.8. Copy the code and add uv up front and --force-reinstall at the back. that gives you the code below.
EDIT at one point it won't be nightly build anymore or there might be a higher version, check occasionally instead of copying my link. As of march 26. it still correct)

**now go to the invoke root directory (where you should see a .venv, models, outputs and so on folder. Open a powershell or whatever does your python prompt here:

enter .venv\Scripts\activate to activate... nevermind. And then the magic formula: uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall

it should start downloading and doing things. when it's done, i can start invoke normally with the launcher and runs no issues with my 5080.

Update: see top of the post, this part has finally become unnecessary as of Invokeai Version 9.10, which installs a modern Bitsandbytes version.

-- Except for a non-crashing (but performance killing) error during launch, about a outdated bisandbytes version. Here is how you update bitsandbytes too:

pip install --force-reinstall --no-cache-dir bitsandbytes --no-deps

IF you use -uv pip- on bitsandbytes you need to do that BEFORE the pytorch one. Edit: i added --no-deps at the end, it should solve this issue of bitsandbytes installing old requirements again, this likely solves the "do it first" problem. Doesn't hurt to do it first though anyway.

i had the same issue (and working RTX5080 solution btw!) with Fluxgym and onetrainer, both work with 5080 with this. procedure.

[LeastyBeasty]

Went through this a few days ago and it's been working fine. Wanted to share, as it's slightly different from the others.

I'm on Windows 11, using RTX 5090.

First of all, I was using the Invoke Installer, and it doesn't include the pip module in the .venv, so I was having trouble trying to manually install cuda128. When I ran the pip commands, it ended up using the system pip, which installed the new torch versions to the wrong place.

You can determine where the pip is by doing where pip (on Windows like me) or which pip for unix type systems I think.

So, basically I had to go in to my directory, as others have stated, and activate the environment.

.venv\Scripts\activate

Then I had to install the missing pip module

py -m ensurepip --default-pip

To confirm it installed to the .venv, I typed where pip and verified it had it listed in the .venv path.

Now we just reinstall torch with the proper cuda version:

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall

Once that is done, start up Invoke and it should work! Hope that helps someone.

[Andyx1976]

Update: even simpler solution now (5.10.1), It's here (last post in this thread, https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2816987958 ) Use that one, it is MUCH easier.

If you use the launcher (get a current one! version 1.5.0). If not, this post here might still be useful.

OLD posts: Update, simplified solution now: As of Version 5.10 The community edition launcher installs a correct (modern) version of Bits and Bytes. So the only thing to be done is the pytorch command. uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall Update2 as of Version 5.10.1 it does NOT ANYMORE! Back to square 1. Try the easy solution, maybe it'll work for you or again in a future version. If you use the launcher, you can do it from the built in console. In the launchers very bottom left click the >_ button, then click "Start Dev console" and once you in just enter the above. It should start downloading some stuff, and then it should work with a RTX50XX

OLDER post for older versions/launchers and manual installations:

revised and updated working solution for RTX 50 support in invokeai ce ( windows)

1. You need python installed of course. Sounds scary, but isn't. It's a simple windows installer (download windows64x installer). Which version? I always see 3.10-12 recommended over newer versions?! Invoke recommends 3.11 So here it is https://www.python.org/downloads/release/python-3119/ select all defaults on install, make sure it's "added to path" in windows.
2. you need uv installed, replacement for pip, just pip didn't work for installing pytorch for me. also invoke uses and recommends using uv. For that at https://docs.astral.sh/uv/getting-started/installation/ they have a simple windows setup-launcher among other ways.

• final note: (Update: as of invoke 5.10 this step is likely not needed anymore, it automatically installs a recent version. Skip the bitsandbytes step), If your version/installation is older!: do bitsandbytes first and THEN pytorch as listed below. I had problems with Invoke 5.9.rc02. result: Do NOT install bitsandbytes via "uv pip .." use only "pip ...".

the actual fix:

3. open Powershell prompt (or your console) in invoke root folder, where the .venv folder is (shift+rightclick in folder, select open powershell) then:

.venv\Scripts\activate

SKIP the bitsandbytes step (this one) if invokeai installation 5.10 or higher! check for Bitsandbytes error during launch.

pip install --force-reinstall --no-cache-dir bitsandbytes (use only pip, NOT uv pip here!) - IF you get error "pip not found" here, just install it locally with python -m ensurepip then continue.

This is the main step: uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall installing The RTX50 supporting pytorch. here you DO need uv pip! Just pip didn't work for me (it installs but invoke has "cuda kernel" errors. .

That is it!

**And don't forget, the launcher overwrites everything with it's ancient medieval versions on every patch so far (5.10 they finally updated their bitsandbytes install) , so you need to do it again. BUT IF something goes wrong here, just use the launcher to re-install invokeai and start over. At least for that it's useful.

This works (self tested) also to add RTX50 support to fluxgym and onetrainer. Install their default, actiavte their env, do the bitsandbytes thing and then the pytorch thing.

**UPDATE: check occasionally https://pytorch.org/get-started/locally/ for a new pytorch version. At the bottom there select stable (instead of nightly) to see if 12.8 or higher is there. So far (March 26) it's still only under nightly ( selecting nightly/windows/pip/ python/12.8 gives you the download link i use above with uv added as discussed in my post further up. ) Referring to the invokeai manual install guide earlier, here it is: https://invoke-ai.github.io/InvokeAI/installation/manual/ Once you get the python and github basics, this is arguably easier than the launcher currently.

If some invoke devs see this: PLEASE update your oh so convenient launcher, that you recommend 20 times in the manual installation instructions as easier option, with a RTX50xx option! It's been MONTHS since rtx50 and working pytorch builds have been released and months since full rtx50 support on Comfyui!!

If i can figure out a working solution you should be very much more. But kinda thanks for forcing me to learn more about the whole scary python and github stuff. And yes, also thanks for Invokeai, which is an amazing tool nonetheless, easy for beginners but very very powerful once you dig a bit deeper. But a big part of it's beginner friendliness IS the launcher so PLEASE fix it.

[LeastyBeasty]

Thanks for the tip about uv.

Things have been working well except it seems like my images don't save the prompt in the metadata. When I recall metadata, everything else is saved except the prompt. Have you experienced this?

[kernkraft235]

I just used the vast.ai cuda 12.8 template and compiled xformers from source For BNB I set env var BNB_CUDA_VERSION=128 and installed v0.45.3 which is the first release with blackwell support

# we are overriding some of the requirements so I install invokeai first
pip install --pre invokeai

# now torch nightly for CUDA 12.8
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

# Now Xformers, we must build from source, it will claim it can't find torch w/o no-build-isolation
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers --no-deps --no-build-isolation

# InvokeAI ran without BNB getting updated but gave errors/warnings on start
export BNB_CUDA_VERSION=128
pip install --force-reinstall 'https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-0.45.3.dev0-py3-none-manylinux_2_24_x86_64.whl' --no-deps`

# Might as well fix pypatchmatch too
sudo apt install python3-opencv libopencv-dev
pip install pypatchmatch

It tries to install pytorch 2.6 so thats why I added no-deps Anecdotally, before fixing BNB i was getting SDXL 1024x1024 in 6 seconds. After updating BNB to a version that supports CUDA 12.8 and blackwell I noticed it was about 3 to 4 seconds. With Hyper lora I was down to 1s per image

[drexlma]

pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers --no-deps --no-build-isolation

I just used the vast.ai cuda 12.8 template and compiled xformers from source For BNB I set env var BNB_CUDA_VERSION=128 and installed v0.45.3 which is the first release with blackwell support

we are overriding some of the requirements so I install invokeai first

pip install --pre invokeai

now torch nightly for CUDA 12.8

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Now Xformers, we must build from source, it will claim it can't find torch w/o no-build-isolation

pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers --no-deps --no-build-isolation

InvokeAI ran without BNB getting updated but gave errors/warnings on start

export BNB_CUDA_VERSION=128 pip install --force-reinstall 'https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-0.45.3.dev0-py3-none-manylinux_2_24_x86_64.whl' --no-deps`

Might as well fix pypatchmatch too

sudo apt install python3-opencv libopencv-dev pip install pypatchmatch It tries to install pytorch 2.6 so thats why I added no-deps Anecdotally, before fixing BNB i was getting SDXL 1024x1024 in 6 seconds. After updating BNB to a version that supports CUDA 12.8 and blackwell I noticed it was about 3 to 4 seconds. With Hyper lora I was down to 1s per image

This works also with NVIDIA GeForce RTX 5070 Ti, i only use uv in docker

uv pip install --pre invokeai uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 uv pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers --no-deps --no-build-isolation export BNB_CUDA_VERSION=128 apt install python3-opencv libopencv-dev uv pip install pypatchmatch

then restart container

i dont need this:

pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers --no-deps --no-build-isolation

[Andyx1976]

Thanks for the tip about uv.

Things have been working well except it seems like my images don't save the prompt in the metadata. When I recall metadata, everything else is saved except the prompt. Have you experienced this?

nope. It works perfectly normal for me. Maybe reinstall invoke again? It's a new version out anyway, so do that and then pytorch again. update bitsandbytes, i added the professional way to do it in both posts.. But i honestly I doubt that prompt thingy function has anything to do with what we touched here.

[boyan-orion]

The installer should have include a selector for RTX 5000 series with pytorch 2.8.0 / Cuda 12.8 by now. I guess i'm going back to the manual installation way with pyenv or uv for now.

[Andyx1976]

but i assume the requirements that uv goes for in the full package list have these old versions too? I mean don't get me wrong, i agree, This outdated launcher months after rtx50xx release is getting ridiculous. Basically instead of saving work the launcher is kinda more work over a manual installation.
Invokeai, is such a amazing tool, easy for beginners but very, very powerful once you dig a bit deeper. But a big part of it's beginner friendliness IS the launcher so PLEASE, devs, fix it!

• I updated my personal solution post https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2717003528 the link to the pytorch selection page. There might be a new or maybe not nightly build version soon. and my link would be bad.
• Also i had problems with 5.9.rc02. tl;dr: don't install bitsandbytes with uv pip! only with pip as in the actual code in the post.** however i still get the warning "Bitsandbytes" was compliled without gpu support...
• i streamlined it a bit and made it beginner-friendly.**

[boyan-orion]

The only way I could get it to work was by doing clean manual install, then pytorch 2.8.0 over and latest bitsandbytes from the GH repo. After that had to fix a symbolic link for a library needed for patchmatch and it works flawlessly now on the latest 5.9.0 RC2

[Little-God1983]

Cannot install bitsandbytes i do this: pip install --force-reinstall --no-cache-dir bitsandbytes then a lot gets uninstalled, downloaded and reinstalled. But then? ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. torchaudio 2.6.0.dev20250326+cu128 requires torch==2.8.0.dev20250325+cu128, but you have torch 2.6.0 which is incompatible. torchvision 0.22.0.dev20250326+cu128 requires torch==2.8.0.dev20250325+cu128, but you have torch 2.6.0 which is incompatible. I dont know how to upgrade torch. I already did: uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall

[boyan-orion]

Cannot install bitsandbytes i do this: pip install --force-reinstall --no-cache-dir bitsandbytes then a lot gets uninstalled, downloaded and reinstalled. But then? ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. torchaudio 2.6.0.dev20250326+cu128 requires torch==2.8.0.dev20250325+cu128, but you have torch 2.6.0 which is incompatible. torchvision 0.22.0.dev20250326+cu128 requires torch==2.8.0.dev20250325+cu128, but you have torch 2.6.0 which is incompatible. I dont know how to upgrade torch. I already did: uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall

Try install it from the gh repo as kernkraft235 mentioned above.

[Andyx1976]

i get the error message too because i do it before the torch update (for that reason, so it doesnt overwrite the pytorch stuff again) . that is normal. but after the uv pip torch thing that very stuff gets installed, That is just my theory, why it works, but after that i can launch invoke with no issues on a 5080. If you cant it seems more like the pytorch command goes wrong.... I´m no expert and no good for trouble shooting. i found one solution that works for me and try not to stray from the path.

When you do the uv pip torch thing it should do a long series of downloads and stuff and then it works . Does it do that? If not maybe uv isntt correctly installed? Also i use my solution (in windows) from the version that the invoke launcher installs. any different base (linux) might have other ways. I let the launcher do its thing (and do it again to reset, if i have a problem). and than do my 3 lines. so far it worked on every patch But i did the same to fluxgym and onetrainer and it worked too.

[mgrugl]

@Andyx1976 @LeastyBeasty Many thanks, your tips work for my 5090. I'd assume the Invoke devs are aware of the need to support 50xx gen cards, but perhaps you would consider a PR.

[Andyx1976]

@Andyx1976 @LeastyBeasty Many thanks, your tips work for my 5090. I'd assume the Invoke devs are aware of the need to support 50xx gen cards, but perhaps you would consider a PR.

i strongly suspect they wait until that cuda version moves from nightly to stable. But a option in the launcher would be good. I'm also absolutely sure they are aware of this.

[AlexGreenEyes]

Is it possible to somehow fix this error?

Could not find the bitsandbytes CUDA binary at WindowsPath('D:/test/.venv/Lib/site-packages/bitsandbytes/libbitsandbytes_cuda128.dll')
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.

Following the advice from the commen https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2717003528

Update:

In my case, what ultimately helped resolve the issue, after performing all the previous steps, was to re-run the installation using uv with the following command:

uv pip install --force-reinstall --no-cache-dir bitsandbytes --no-deps

After this, everything worked, and I was finally able to run Flux successfully, whereas before this step, it wouldn’t launch. Interestingly, other models like SDXL and SD worked fine during this time, which might be a useful clue for others facing similar issues.

[Andyx1976]

Update working, for all RTX50 cards (i think) for 5.10.1: , it seems using in the console

uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --no-deps --force-reinstall does the trick for 5.10.1. Paste exactly that, it has to have --no-deps in it, or it will crash on launch! No bitsandbytes commands needed anymore.

Howto:

Install the version with the launcher, check the "repair" box during installation. When finished click the >_ button bottom left in the launcher, then click "Start dev Console", then paste the command above (exactly) and wait til it's done - it should start downloading about 3gb of stuff)). See you in the next update when that doesn't work anymore.

if you don't use the launcher, a older version or get a Bitsandbytes error during launch, check extended solution post: https://github.com/invoke-ai/InvokeAI/issues/7683#issuecomment-2717003528 for a longer guide.

check occasionally https://pytorch.org/get-started/locally/ for a new pytorch version. At the bottom there select stable (instead of nightly) to see if 12.8 or higher is there. So far (April 21.) it's still only under nightly ( selecting nightly/windows/pip/ python/12.8 gives you the download link i use above with uv added. change the url in the above command. Once it's listed as stable they will finally add it to the launcher.

Here is the most important tip: IF you got a WORKING installation, leave it alone! Don't update it with the launcher at least for small updates. The usually tiny changes in versions are not worth the trouble! One day they may add a rtx50 alternative install option.

Whatever is going on with Pytorch and Nvidia, fact is you don't get any other Nvidia cards anymore, so it's NOT just "a few rich people who can afford a 5080 or 90" anymore, the 5060 has the same problem.

[ecker00]

Will this work with driver 570.133 and cuda 12.8? Thinking maybe to wait until this is resolved, as suggestion here seems a bit daunting to configure.

$ docker run --rm --runtime=nvidia --gpus=all --publish 9090:9090 \
  --env-file ./.env \
  -v ./invokeai.yaml:/invokeai/invokeai.yaml \
  ghcr.io/invoke-ai/invokeai:5.10.1-cuda

Traceback (most recent call last):
  File "/opt/venv/bin/invokeai-web", line 10, in <module>
    sys.exit(run_app())
             ^^^^^^^^^
  File "/opt/invokeai/invokeai/app/run_app.py", line 37, in run_app
    torch_device_name = TorchDevice.get_torch_device_name()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/invokeai/invokeai/backend/util/devices.py", line 94, in get_torch_device_name
    return torch.cuda.get_device_name(device) if device.type == "cuda" else device.type.upper()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 491, in get_device_name
    return get_device_properties(device).name
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 523, in get_device_properties
    _lazy_init()  # will define _get_device_properties
    ^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 319, in _lazy_init
    torch._C._cuda_init()
RuntimeError: No CUDA GPUs are available

[strayge]

For those who use docker - Dockerfile with squashed layers based on v5.11.0:

FROM ghcr.io/invoke-ai/invokeai:sha-7722f47-cuda AS base
RUN UV_INDEX=https://download.pytorch.org/whl/cu128 uv pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --no-cache-dir

FROM scratch
COPY --from=base / /

ENV \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    VIRTUAL_ENV=/opt/venv \
    INVOKEAI_SRC=/opt/invokeai \
    PYTHON_VERSION=3.12 \
    UV_PYTHON=3.12 \
    UV_COMPILE_BYTECODE=1 \
    UV_MANAGED_PYTHON=1 \
    UV_LINK_MODE=copy \
    UV_PROJECT_ENVIRONMENT=/opt/venv \
    UV_INDEX="https://download.pytorch.org/whl/cu124" \
    INVOKEAI_ROOT=/invokeai \
    INVOKEAI_HOST=0.0.0.0 \
    INVOKEAI_PORT=9090 \
    PATH="/opt/venv/bin:$PATH" \
    CONTAINER_UID=${CONTAINER_UID:-1000} \
    CONTAINER_GID=${CONTAINER_GID:-1000}

ARG GPU_DRIVER=cuda

WORKDIR ${INVOKEAI_SRC}

ENTRYPOINT ["/opt/invokeai/docker-entrypoint.sh"]
CMD ["invokeai-web"]

[jvbrt4]

Running this after installing default version is the way: uv pip install --upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu128

[nelno]

Note that to get the uv pip install fix to work I had to do it from UV environment started inside my invoke AI folder. Otherwise you'll end up changing your machine's default uv environment (I guess) and not the invoke one.