sergey21000 comments

Repositories
Issues
Comments

Results 1 comments of


                                            sergey21000

CUDA wheel installs, but GPU is never used on Windows 11 (Python 3.11, CUDA 12.1, torch finds GPU)

Use a fork that provides prebuilt Windows wheels, such as [JamePeng/llama-cpp-python](https://github.com/JamePeng/llama-cpp-python/releases) ```powershell pip install https://github.com/JamePeng/llama-cpp-python/releases/download/v0.3.16-cu128-AVX2-win-20251007/llama_cpp_python-0.3.16-cp311-cp311-win_amd64.whl ``` The original `llama-cpp-python` repository does not provide Windows wheels. You can confirm this on...