InvokeAI, Linux, AMD Radeon, Rocm, no GPU support

[tarkh]

Package

InvokeAI

When did the issue occur?

Running the Package

What GPU / hardware type are you using?

CPU: AMD Ryzen 7700 GPU: Radeon 7800 XT

What happened?

GPU support did not work.

In StabilityMatrix.Core/Models/Packages/InvokeAI.cs there is a part https://github.com/LykosAI/StabilityMatrix/blob/bedc702697a572ada5a0faffbd357e58949688b1/StabilityMatrix.Core/Models/Packages/InvokeAI.cs#L178 which sets pipCommandArgs to use rocm5.6 and specific versions of other packages.

But on official InvokeAI manual install guide there's rocm6.1: https://invoke-ai.github.io/InvokeAI/installation/manual/

uv pip install invokeai==5.6.1 --python 3.11 --python-preference only-managed --index=https://download.pytorch.org/whl/rocm6.1 --force-reinstall

So with manual separate InvokeAI install everything seems to work fine.

Console output

The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
>> patchmatch.patch_match: ERROR - patchmatch failed to load or compile (/Apps/Data/Packages/InvokeAI/venv/lib/python3.10/site-packages/torch/lib/libgomp-a34b3233.so.1: version `GOMP_5.0' not found (required by /usr/lib/libvtkCommonCore.so.1)).
>> patchmatch.patch_match: INFO - Refer to https://invoke-ai.github.io/InvokeAI/installation/060_INSTALL_PATCHMATCH/ for installation instructions.
[2025-02-13 03:53:20,473]::[InvokeAI]::INFO --> Patchmatch not loaded (nonfatal)
[2025-02-13 03:53:22,133]::[InvokeAI]::INFO --> Using torch device: CPU
[2025-02-13 03:53:22,273]::[InvokeAI]::INFO --> cuDNN version: 8902
[2025-02-13 03:53:22,291]::[InvokeAI]::INFO --> InvokeAI version 5.6.1
...

Additional InvokeAI fixes

1. Install/Reinstall this packages with proper versions:

pip install "torch==2.4.1+rocm6.1" "torchvision==0.19.1+rocm6.1" "torchaudio==2.4.1+rocm6.1" --force-reinstall --extra-index-url https://download.pytorch.org/whl/rocm6.1
pip install tensorflow==2.15.0 --force-reinstall
pip install ml_dtypes==0.2.0 --force-reinstall
pip install numpy==1.26.4 --force-reinstall

2. To fix bitsandbytes issue, you need to install Multi-Backend Preview version:

cd <invokeai dir>
source venv/bin/activate
pip install --no-deps 'https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_multi-backend-refactor/bitsandbytes-0.44.1.dev0-py3-none-manylinux_2_24_x86_64.whl'

3. To fix patchmatch GOMP_5.0 error you need to link local system libgomp.so.1 to libgomp-a34b3233.so.1 in the package:

cd <invokeai dir>/venv/lib/python3.10/site-packages/torch/lib/
mv libgomp-a34b3233.so.1 libgomp-a34b3233.so.1.bkp
ln -s /usr/lib/libgomp.so.1 libgomp-a34b3233.so.1

Or if package already contains libgomp.so

cd <invokeai dir>/venv/lib/python3.10/site-packages/torch/lib/
mv libgomp.so libgomp.so.bkp
ln -s /usr/lib/libgomp.so libgomp.so

My fix of StabilityMatrix

So in StabilityMatrix.Core/Models/Packages/InvokeAI.cs i edited code around line 170:

     case TorchIndex.Rocm:
         torchInstallArgs = torchInstallArgs
             .WithTorch("==2.4.1+rocm6.1")
             .WithTorchVision("==0.19.1+rocm6.1")
             .WithTorchAudio("==2.4.1+rocm6.1")
             .WithExtraIndex("https://download.pytorch.org/whl/rocm6.1");

         Logger.Info("Starting InvokeAI install (ROCm)...");
         pipCommandArgs =
             "-e . --use-pep517 --extra-index-url https://download.pytorch.org/whl/rocm6.1";
         break;

After building and running app now InvokeAI installs/updates without issues. But after install and each update I have to:

1. Fix bitsandbytes as mentioned before
2. Fix patchmatch as mentioned before
3. Install packages with proper version in this order:

pip install tensorflow==2.15.0 --force-reinstall
pip install ml_dtypes==0.2.0 --force-reinstall
pip install numpy==1.26.4 --force-reinstall

For now I just created SH script to run after each update to automate this. But maybe it can be introduced in StabilityMatrix itself? Is there a way to have somekind of hook to run cusom commands/scripts after package updates? Anyway we will hope, that InvokeAI will manage to update and set everything properly in their packages.

Version

2.13.3

What Operating System are you using?

Arch Linux