llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

Published 20 hours ago • verified publisher icon abetlen

successfully installed !! CUDA/v12.6 !! Visual Studio 2022!!

Open dw5189 opened this issue 9 months ago • 3 comments

I've just successfully installed it! Here's the information for your reference.

PowerShell ：

$env:CUDA_TOOLKIT_ROOT_DIR="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6" $env:CMAKE_GENERATOR_PLATFORM="x64" $env:FORCE_CMAKE="1" $env:CMAKE_ARGS="-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89" pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade

** Visual Studio 2022 Developer PowerShell v17.10.11 ** Copyright (c) 2022 Microsoft Corporation

(base) PS C:\Users\Administrator\source\repos> conda activate CUDA126-py312 (CUDA126-py312) PS C:\Users\Administrator\source\repos> $env:CUDA_TOOLKIT_ROOT_DIR="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6" (CUDA126-py312) PS C:\Users\Administrator\source\repos> $env:CMAKE_GENERATOR_PLATFORM="x64" (CUDA126-py312) PS C:\Users\Administrator\source\repos> $env:FORCE_CMAKE="1" (CUDA126-py312) PS C:\Users\Administrator\source\repos> $env:CMAKE_ARGS="-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89" (CUDA126-py312) PS C:\Users\Administrator\source\repos> pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/, http://mirrors.aliyun.com/pypi/simple/ Collecting llama-cpp-python Downloading http://mirrors.aliyun.com/pypi/packages/a6/38/7a47b1fb1d83eaddd86ca8ddaf20f141cbc019faf7b425283d8e5ef710e5/llama_cpp_python-0.3.7.tar.gz (66.7 MB) ---------------------------------------- 66.7/66.7 MB 22.7 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting typing-extensions>=4.5.0 (from llama-cpp-python) Downloading http://mirrors.aliyun.com/pypi/packages/26/9f/ad63fc0248c5379346306f8668cda6e2e2e9c95e01216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl (37 kB) Collecting numpy>=1.20.0 (from llama-cpp-python) Downloading http://mirrors.aliyun.com/pypi/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl (12.6 MB) ---------------------------------------- 12.6/12.6 MB 23.3 MB/s eta 0:00:00 Collecting diskcache>=5.6.1 (from llama-cpp-python) Downloading http://mirrors.aliyun.com/pypi/packages/3f/27/4570e78fc0bf5ea0ca45eb1de3818a23787af9b390c0b0a0033a1b8236f9/diskcache-5.6.3-py3-none-any.whl (45 kB) Collecting jinja2>=2.11.3 (from llama-cpp-python) Downloading http://mirrors.aliyun.com/pypi/packages/bd/0f/2ba5fbcd631e3e88689309dbe978c5769e883e4b84ebfe7da30b43275c5a/jinja2-3.1.5-py3-none-any.whl (134 kB) Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python) Downloading http://mirrors.aliyun.com/pypi/packages/c1/80/a61f99dc3a936413c3ee4e1eecac96c0da5ed07ad56fd975f1a9da5bc630/MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl (15 kB) Building wheels for collected packages: llama-cpp-python Building wheel for llama-cpp-python (pyproject.toml) ... done Created wheel for llama-cpp-python: filename=llama_cpp_python-0.3.7-cp312-cp312-win_amd64.whl size=93613512 sha256=cd98aae040b2dbcc1f4653370900de27455ef65275d08543da81c53c31138a1a Stored in directory: C:\Users\Administrator\AppData\Local\Temp\pip-ephem-wheel-cache-9usio9a1\wheels\ec\61\fc\cee068315610d77f6a99c0032a74e4c8cb21c1d6e281b45bb5 Successfully built llama-cpp-python Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python Successfully installed MarkupSafe-3.0.2 diskcache-5.6.3 jinja2-3.1.5 llama-cpp-python-0.3.7 numpy-2.2.3 typing-extensions-4.12.2 (CUDA126-py312) PS C:\Users\Administrator\source\repos>

G:>conda.bat activate CUDA126-py312 ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4070 SUPER, compute capability 8.9, VMM: yes llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4070 SUPER) - 11053 MiB free llama_model_loader: loaded meta data with 29 key-value pairs and 579 tensors from E:.lmstudio\models\Qwen\Qwen2.5-Coder-14B-Instruct-GGUF\qwen2.5-coder-14b-instruct-q4_k_m.gguf (version GGUF V3 (latest))

dw5189 avatar Feb 26 '25 14:02 dw5189