jmoney7823956789378

Hmm weird, what's the log say right before `Traceback (most recent call last):` ? Also, does your $PATH have /opt/rocm/bin right at the start? `export PATH=/opt/rocm/bin:$PATH`

I kept my exllama folder in /repositories and it seemed to work, but... I'm not sure about the hip modules missing.

> Do you think that it is potentially an environment/system issue with the ROCm installation and not an exllama issue? I've tried running with both the pip module installed and...

I was running GPTQ multi for 65B, it's pretty slow across the two MI60s, but you have soooo much memory to spare you could probably dump context like it's nothing....

> the other is unfortunately running PCIE Gen3x4 mode at the moment. I think... the MI60s HATE running in

Hmm, yeah maybe next step will be 22.04, unfortunately. If that doesn't work I'd say you might be getting stuck by the x8/x4 connection, but that could just be me...

> We could swap so that we'd both have a match! Ha ha 😛 > > Interestingly, one of my MI60s is 1x8-pin PCIe power connector + 1x6-pin PCIe power...

> Out of curiosity, how are you cooling your MI60s? I am cooling using 1 80mm fan per card, rigged up with 3D printed fan shrouds so that they force...

Did you try a docker yet? I'm not sure if it will help, but here's what I used to set up my docker container for oobabooga before. https://github.com/evshiron/rocm_lab

@sjstulga not sure if you're still having issues, but I wanted to point out that I was using CUDA_VISIBLE_DEVICES=0 (or 1) even when using my MI60s. I saw you have...