Why is this not working for the current release. UNABLE TO USE GPU

Open AnirudhJM24 opened this issue 1 year ago • 1 comments

          I got it to works just like instruction, I'm using CUDA 12.3:

set CMAKE_ARGS="-DLLAMA_CUBLAS=on" && set FORCE_CMAKE=1 && pip install --no-cache-dir llama-cpp-python==0.2.90 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu123

Originally posted by @BinhPQ2 in https://github.com/abetlen/llama-cpp-python/issues/576#issuecomment-2379861701

Oct 02 '24 16:10 AnirudhJM24

What is your CUDA version?

And you can try to update your Visual Studio Built Tool to see if it will solve the problem! https://visualstudio.microsoft.com/downloads/?q=build+tools

Oct 03 '24 09:10 BinhPQ2