Lukas S

Does someone has a working AutoGPTQ setup? Mine was really slow when I installed the wheel: https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.4.2/auto_gptq-0.4.2+rocm5.4.2-cp310-cp310-linux_x86_64.whl When building from source, the text generation is much faster but the output...

> @CNR0706 I managed to install a modified version of bitsandbytes for ROCm. Just follow this tutorial and you should be fine: [YT Video](https://www.youtube.com/watch?v=2cPsvwONnL8). Therefore, you can leverage all of...

> I have an AMD Radeon RX 5500 XT, is that good? My CPU spits fully completed things out within 6 seconds, when the CPU isn't stressed with OBS. Otherwise...

@henrittp, @CNR0706 Did you try setting up AutoGPTQ? Did it work for you?

> I have `AttributeError: 'NoneType' object has no attribute 'split' ` error too... @RBNXI This is caused by `bitsandbytes`. You need to install a specific Version. Take a look at...

@RBNXI What Error did you get? Make sure the repo is located on a path without spaces. This seems to cause issues sometimes. And you need the `rocm-hip-sdk` package (at...

Yes you can of course use the link from the repo directly. You probably mean this one: https://github.com/agrocylo/bitsandbytes-rocm.git

# AMD Setup Step-by-Step Guide (WIP) I finally got my setup working again (by reinstalling everything). Here is a step by step guide on how I got it running: I...

> I try to load a model and says: assert self.model is not None Errors are never ending... @RBNXI What model are you using? Which loader are you using? Usually...

@RBNXI I found this [Issue](https://github.com/RadeonOpenCompute/ROCm/issues/1698) in the ROCm Repo discussing the RX 6600. According to this the RX 6600 should work. Usually for all 6xxx cards `gfx1030` works. You could...