abysssol

Sorry, but I don't have much idea about what's going wrong. It seems likely that ollama simply doesn't support that gpu currently. This `nvml vram init failure: 18` is a...

> Sorry to re-open this issue. Don't be; this is exactly what reopening issues is meant for. I'm actually rather excited to see that this problem may have a solution...

Sorry for not getting to this today, I've been working on getting ollama 0.1.24 [merged into upstream nixpkgs](https://github.com/NixOS/nixpkgs/pull/289108).

My hope is that once ollama is available from upstream nixpkgs, all library and driver versions should match, so hopefully your gpu would work then. I changed back to vendoring...

I created a new branch, `cuda-testing`, that changes how it splices together the `cuda-toolkit` libs. Could you try building that and see if it makes a difference? ``` nix ollama...

> Apologies for not responding No worries. It's good to prioritize things that actually matter to you. I'll do the same. To put it in perspective, we're just trying to...

I'm not sure if this will work, but I think you can just override nvidia_x11 with your custom driver. Try it and tell me if it works. ``` nix {...

I'm glad to hear you got it working. Is there a reason why you're duplicating the definition of your custom driver instead of using a let binding? It seems like...

It seems to me that the version of rocblas in nixpkgs (5.7.1) doesn't support your gpu. As far as I can tell, this is purely a nixpkgs problem, not an...

Have you tried enabling rocm on a recent version of ollama from nixpkgs? The version in nixpkgs-unstable has been using version 6 of rocm libraries, which may be worth trying...